Main WebsiteDiscord ChatOnline APIGitHub

Integrating With Games

Suggest Edits

Ultralight offers developers the ability to display fast, modern HTML UI within games and other GPU-based apps.

Let's dive into the main things you need to know to integrate Ultralight into an existing game or game engine.

Two Ways to Integrate

Ultralight can either:

  • Render on the CPU to a pixel buffer (using the Surface API).
  • Render on the GPU via low-level commands (using the GPUDriver API)

The pixel-buffer method is easier to integrate and gives good performance across a range of use-cases. In a game, you would simply upload the pixel buffer to a texture each time it changes.

You can always upgrade later to the GPU renderer later if you need the extra performance and are comfortable working with low-level GPU driver commands.

👍

We will be using the CPU Renderer (Surface API) in this article.

If you'd like to use the GPU Renderer instead, please see Using a Custom GPUDriver

Configuring the Library

The first thing to do is to set up our Config-- you can use this to customize many low-level rendering, layout, performance, and styling aspects of the engine.

At a minimum you'll need to specify the resource_path (the library needs this path to load bundled SSL certificates and other data).

#include <Ultralight/Ultralight.h>
  
using namespace ultralight;

void Init() {
  Config config;
  
  ///
  /// We need to tell config where our resources are so it can 
  /// load our bundled SSL certificates to make HTTPS requests.
  ///
  config.resource_path = "./resources/";
  
  ///
  /// The GPU renderer should be disabled to render Views to a 
  /// pixel-buffer (Surface).
  ///
  config.use_gpu_renderer = false;
  
  ///
  /// You can set a custom DPI scale here. Default is 1.0 (100%)
  ///
  config.device_scale = 1.0;
  
  ///
  /// Pass our configuration to the Platform singleton so that
  /// the library can use it.
  ///
  Platform::instance().set_config(config);
}

📘

Check out the Config API

There's more options to choose from, see the full Config API here.

Defining Platform Handlers

The Platform singleton can be used to provide custom handlers for things like file loading, font loading, clipboard access, and other low-level OS tasks.

All platform handlers are optional except for the FontLoader (the library will not run if one is not defined).

Using the Default Platform Handlers

To make it easier to get started, AppCore provides default implementations for FileSystem, FontLoader, and Logger for all major desktop platforms.

You can use these default implementations in your code via the following (just make sure to #include <AppCore/Platform.h>, it's not included in the main library header):

#include <Ultralight/Ultralight.h>
#include <AppCore/Platform.h>
  
using namespace ultralight;

void InitPlatform() {
  ///
  /// Use the OS's native font loader
  ///
  Platform::instance().set_font_loader(GetPlatformFontLoader());

  ///
  /// Use the OS's native file loader, with a base directory of "."
  /// All file:/// URLs will load relative to this base directory.
  ///
  Platform::instance().set_file_system(GetPlatformFileSystem("."));

  ///
  /// Use the default logger (writes to a log file)
  ///
  Platform::instance().set_logger(GetDefaultLogger("ultralight.log"));
}

Using a Custom FileSystem

When integrating into a game, you may want to provide your own FileSystem implementation to load encrypted / compressed HTML/JS/CSS assets from your game's native data loader.

To do this you would simply subclass the FileSystem interface, override the callbacks, and provide an instance of the class to Platform::instance().set_file_system(). (This instance should outlive the Renderer). For more info please see the FileSystem API.

Creating the Renderer

The Renderer singleton maintains the lifetime of the library, orchestrates all painting and updating, and is required before creating any Views.

To create the Renderer simply call Renderer::Create().

RefPtr<Renderer> renderer;

void CreateRenderer() {
  ///
  /// Create our Renderer (call this only once per application).
  /// 
  /// The Renderer singleton maintains the lifetime of the library
  /// and is required before creating any Views.
  ///
  /// You should set up the Platform handlers before this.
  ///
  renderer = Renderer::Create();
}

Creating Views

Views are sized containers for displaying HTML content, similar to a tab in a browser. You can create one via Renderer::CreateView():

RefPtr<View> view;

void CreateView() {
  ///
  /// Create an HTML view, 500 by 500 pixels large.
  ///
  view = renderer->CreateView(500, 500, false, nullptr);
  
  ///
  /// Load a raw string of HTML.
  ///
  view->LoadHTML("<h1>Hello World!</h1>");
  
  ///
  /// Notify the View it has input focus (updates appearance).
  ///
  view->Focus();
}

Using the Surface

Each View has an underlying pixel buffer Surface which the Renderer paints into.

You can get the Surface for a View via View::surface():

///
/// Get the pixel-buffer Surface for a View.
///
Surface* surface = view->surface();

The default Surface implementation is BitmapSurface, which is backed by a Bitmap.

You can access it via the following:

///
/// Get the pixel-buffer Surface for a View.
///
Surface* surface = view->surface();

///
/// Cast it to a BitmapSurface.
///
BitmapSurface* bitmap_surface = (BitmapSurface*)surface;

///
/// Get the underlying bitmap.
///
RefPtr<Bitmap> bitmap = bitmap_surface->bitmap();

Once you have the Bitmap, you can lock it to retrieve the pixel buffer and upload to a texture.

void CopyBitmapToTexture(RefPtr<Bitmap> bitmap) {
  ///
  /// Lock the Bitmap to retrieve the raw pixels.
  /// The format is BGRA, 8-bpp, premultiplied alpha.
  ///
  void* pixels = bitmap->LockPixels();

  ///
  /// Get the bitmap dimensions.
  ///
  uint32_t width = bitmap->width();
  uint32_t height = bitmap->height();
  uint32_t stride = bitmap->row_bytes();
 
  ///
  /// Psuedo-code to upload our pixels to a GPU texture.
  ///
  CopyPixelsToTexture(pixels, width, height, stride);

  ///
  /// Unlock the Bitmap when we are done.
  ///
  bitmap->UnlockPixels();
}

Updating the Renderer

You should call Renderer::Update() as often as possible to give the library a chance to process background tasks, JavaScript callbacks, and dispatch event listeners.

void UpdateLogic() {
  ///
  /// Give the library a chance to handle any pending tasks and timers.
  ///
  ///
  renderer->Update();
}

Rendering Views

You should generally call Renderer::Render() once per frame to update the Surface of each View.

Views only re-render if they actually need painting, you can check if a Surface has changed by checking if Surface::dirty_bounds() is non-empty.

void RenderOneFrame() {
  ///
  /// Render all active Views (this updates the Surface for each View).
  ///
  renderer->Render();
  
  ///
  /// Psuedo-code to loop through all active Views.
  ///
  for (auto view : view_list) {
    ///
    /// Get the Surface as a BitmapSurface (the default implementation).
    ///
    BitmapSurface* surface = (BitmapSurface*)(view->surface());
    
    ///
    /// Check if our Surface is dirty (pixels have changed).
    ///
    if (!surface->dirty_bounds().IsEmpty()) {
      ///
      /// Psuedo-code to upload Surface's bitmap to GPU texture.
      ///
      CopyBitmapToTexture(surface->bitmap());
      
      ///
      /// Clear the dirty bounds.
      ///
      surface->ClearDirtyBounds();
    }
  }
}

Passing Mouse Input

Passing mouse input to a View is pretty straightforward-- just create a MouseEvent and pass it to View::FireMouseEvent():

MouseEvent evt;
evt.type = MouseEvent::kType_MouseMoved;
evt.x = 100;
evt.y = 100;
evt.button = MouseEvent::kButton_None;

view->FireMouseEvent(evt);

Just make sure that all coordinates are localized to the View's quad in screen-space, and that they are scaled to logical units using the current device scale (if you set one).

Passing Keyboard Input

Keyboard events are broken down into three major types:

  • KeyEvent::kType_RawKeyDown -- Physical key press
  • KeyEvent::kType_KeyUp -- Physical key release
  • KeyEvent::kType_Char -- Text generated from a key press. This is typically only a single character.

📘

You should almost always use KeyEvent::kType_RawKeyDown for key presses since it lets WebCore translate these events properly.

You will need to create a KeyEvent and pass it to View::FireKeyEvent():

// Synthesize a key press event for the 'Right Arrow' key
KeyEvent evt;
evt.type = KeyEvent::kType_RawKeyDown;
evt.virtual_key_code = KeyCodes::GK_RIGHT;
evt.native_key_code = 0;
evt.modifiers = 0;

// You'll need to generate a key identifier from the virtual key code
// when synthesizing events. This function is provided in KeyEvent.h
GetKeyIdentifierFromVirtualKeyCode(evt.virtual_key_code, evt.key_identifier);

view->FireKeyEvent(evt);

In addition to key presses / key releases, you'll need to pass in the actual text generated. (For example, pressing the A key should generate the character 'a').

// Synthesize an  event for text generated from pressing the 'A' key
KeyEvent evt;
evt.type = KeyEvent::kType_Char;
evt.text = "a";
evt.unmodified_text = "a"; // If not available, set to same as evt.text

view->FireKeyEvent(evt);

Handling Keyboard Focus

When dispatching keyboard events in your game, you may only want the View to consume keyboard events when an input element has keyboard focus (usually indicated by a blinking caret).

You can check if a view has input focus via View::HasInputFocus():

if (view->HasInputFocus()) {
  ///
  /// The View has an input element with visible keyboard focus (blinking caret).
  /// Dispatch the keyboard event to the view and consume it.
  ///
  DispatchAndConsumeKeyboardEvent(view, evt);
}

📘

You should call View::focus() whenever the View gains focus (focus management and event dispatch should be handled by you). If you forget to call this, the renderer will not render any carets or other focus indicators.

Multithreading

The Ultralight API is not thread-safe at this time-- calling the API from multiple threads is not supported and will lead to subtle issues / application instability.

The library does not need to run on the main thread though-- you can create the Renderer on another thread and make all calls to the API on that thread.

Updated over 2 years ago