HTML-in-Canvas overview

Natalia Markoborodova

The HTML-in-Canvas API bridges the gap between the interactive and accessible world of the DOM and the high-performance, pixel-level control of <canvas>. The API lets you draw live DOM subtrees directly into 2D, WebGL, or WebGPU canvas contexts—while the browser retains interactivity, accessibility trees, and browser-level integrations.

Video overview of the HTML-in-Canvas API.

To learn about the benefits of HTML-in-Canvas, how it improves on earlier workarounds, and which use cases it's best suited for, read HTML-in-Canvas in Chrome 157: Interactive DOM inside 2D, WebGL, and WebGPU.

Implement your UI with HTML-in-Canvas

To integrate HTML-in-Canvas into your app, you can:

  • Create a direct implementation using the browser APIs.
  • Use a framework that supports HTML-in-Canvas.

You can also use a coding agent to assist you. Learn how to improve your coding agent experience with Modern Web Guidance.

Direct implementation

The implementation depends on the context you're using. Before you start, review the technical concepts that are common across all the contexts, such as drawable content and the paint event.

Learn how to draw the element image into a canvas and update the element's geometry for the following contexts:

Library and framework support

Several popular libraries have added experimental HTML-in-Canvas support, for WebGL and WebGPU textures as well as 2D canvas effects. Check each library's documentation for its current support status and implementation details.

Three.js

Updating matrixes manually can be tedious, so frameworks are adding built-in support. For example, Three.js provides experimental support using THREE.HTMLTexture:

const material = new THREE.MeshBasicMaterial();
material.map = new THREE.HTMLTexture(uiElement); // Pass the DOM element

const geometry = new THREE.BoxGeometry(1, 1, 1);
const mesh = new THREE.Mesh(geometry, material);
scene.add(mesh);

PlayCanvas

PlayCanvas also supports HTML-in-Canvas using their texture API:

// Wait for the 'paint' event to set the source
canvas.addEventListener('paint', () => {
  htmlTexture.setSource(htmlElement);
}, { once: true });
canvas.requestPaint();

// Keep up to date
canvas.addEventListener('paint', onPaintUpload);

const material = new pc.StandardMaterial();
material.diffuseMap = htmlTexture;
material.update();

Babylon.js

Babylon.js supports HTML-in-Canvas through the BABYLON.HtmlTexture class:

// 2. Wrap the DOM element in an HtmlTexture
const htmlTexture = new BABYLON.HtmlTexture("htmlTexture", uiElement, scene);

// 3. Assign the texture to a material
const material = new BABYLON.StandardMaterial("hudMaterial", scene);
material.diffuseTexture = htmlTexture;

// 4. Apply material to 3D mesh
const plane = BABYLON.MeshBuilder.CreatePlane("hudPlane", { width: 1.6, height: 0.9 }, scene);
plane.material = material;

// 5. Start Render Loop
engine.runRenderLoop(() => {
  scene.render();
});

PixiJS

PixiJS shipped support for HTML-in-Canvas in v8.19.0:

import { HTMLSource } from 'pixi.js/html-source';

// ... Initialize the app

// Append the HTML element as a direct child of the Pixi canvas
app.canvas.appendChild(htmlElement);

// Create the HTML-in-Canvas source and wrap it in a Sprite
const sprite = Sprite.from(new HTMLSource({ resource: form, autoUpdate: true }));
app.stage.addChild(sprite);

app.ticker.add(() => {
  // Make sure to align the HTML element by updating its transform
  const syncT = app.canvas.getElementTransform(htmlElement, finalTransform);
  if (syncT) {
    htmlElement.style.transform = syncT.toString();
  }
});

CanvasUI

CanvasUI is an open source library of UI components that use HTML-in-Canvas.

Implement HTML-in-Canvas with a coding agent

If you use a coding agent, install Modern Web Guidance, which includes the latest implementation instructions for HTML-in-Canvas. In addition to HTML-in-Canvas, Modern Web Guidance covers other modern browser APIs and best practices.

Next steps