Widgets in the scene

Flutter Scene can put a real, live Flutter widget subtree on a 3D surface. The subtree stays fully alive (state, tickers, and animations run normally), its visual output streams into a texture on scene geometry, and pointer input finds its way back in, so buttons press and sliders drag on the face of a mesh.

The panel below is ordinary Flutter, a Material surface with a Slider and InkWell color buttons. It is also ordinary scene geometry, a glossy lit surface angled toward the cube, lit by the environment and catching a screen-space reflection of the cube it controls. Drag the slider and tap the swatches.

▶ Run demo Loads a live Flutter Scene build

A live Flutter panel on a glossy lit surface, driving the cube it reflects. The slider and buttons are real widgets.

A widget on a surface

WidgetComponent is a node component that owns the whole pipeline, hosting the subtree, capturing it, and binding the capture to a material. The zero-config form creates an aspect-correct, alpha-blended quad for you.

final panel = Node()
  ..addComponent(WidgetComponent(
    child: MyControlPanel(), // any widget
    size: const Size(360, 260), // logical layout size
    worldHeight: 2.0, // quad height in world units
  ));
scene.add(panel);

The child is laid out at size logical pixels and captured at size times pixelRatio texels, so raise pixelRatio for crisper text on close-up surfaces. The scene must be rendered by a SceneView, which is what hosts the subtree invisibly and keeps it alive.

The child is fixed at construction, so drive visual changes from state inside the subtree (a StatefulWidget, a ValueListenableBuilder, an AnimatedBuilder); those rebuilds happen live in the hosted tree and each one is captured. A setState in the widget tree around the scene does not reach the hosted child. The demo’s panel works this way, it owns the slider value and reports changes outward through a callback.

Your own geometry and materials

Three setup tiers share the one component. Beyond the zero-config quad, pass any geometry with 0..1 UVs and the component still owns the material, or take over material binding entirely.

WidgetComponent(
  child: MyScreenUi(),
  size: const Size(480, 360),
  geometry: myCurvedScreen, // any surface with 0..1 UVs
  material: myMaterial,
  bind: (texture) => myMaterial.parameters.setTexture('screen_texture', texture),
)

For a surface that already exists, like a display inside an imported model, WidgetComponent.bindOnly creates no mesh at all and just delivers the capture to your bind callback whenever the texture changes.

A lit widget surface

The auto-created quad is unlit, so the widget draws at its authored colors regardless of scene lighting, the right default for readable UI. But a widget surface is scene geometry, and handing the component a lit material makes the scene treat it like any other surface. The demo’s panel is exactly this, the capture becomes the base color of a glossy PhysicallyBasedMaterial, so the environment shades it and screen-space reflections trace the cube across its face.

WidgetComponent(
  child: MyControlPanel(),
  size: const Size(360, 260),
  worldHeight: 2.0,
  material: PhysicallyBasedMaterial()
    ..metallicFactor = 0.0
    ..roughnessFactor = 0.15, // glossy; low roughness picks up sharp reflections
)

// Sharp reflections trace the rendered image (see Scene.screenSpaceReflections).
scene.screenSpaceReflections.enabled = true;

For the built-in materials the component binds the capture to baseColorTexture implicitly; any other material needs a bind callback naming the slot.

One caveat when going lit and opaque: pixels the widget leaves transparent have no color, so they read as black on an opaque material. Keep the child’s background fully opaque (the demo’s panel is a solid Material with square corners). For a floating translucent panel, stay with the default unlit alpha-blended quad instead; translucent surfaces skip the depth prepass that screen-space reflections trace against.

Input

With WidgetInput.automatic (the default), the SceneView forwards platform pointer events by raycasting them into the scene. When this component’s surface is the nearest hit, the event is delivered to the widget subtree at the hit’s UV coordinate, so gesture recognizers, buttons, drags, and scrollables work normally. Occlusion is respected, a wall in front of a panel blocks it, and while a press is held the interaction stays captured to the pressed surface even if the pointer slides off its edge (standard pointer-capture semantics).

Set WidgetInput.manual to drive input yourself, either through the component’s controller (pointerDown/pointerMove/pointerUp/tapAt at a texture-space UV) or with a ScenePointer for a crosshair or gamepad-driven cursor. See Picking and input for the pointer machinery.

Update policies

WidgetUpdatePolicy decides when the child is re-captured.

• everyFrame (the default) observes every change, including repaints inside the child’s own repaint boundaries (progress indicators, scrolling lists). Captures are throttled to one in flight, and recording reuses the child’s retained layers, so static content costs little.
• WidgetUpdatePolicy.interval(duration) captures at most that often, for dashboards and status panels.
• manual captures only when you call controller.requestCapture(), for content that changes at known moments.

Capture without a component

The widget layer under the component is usable on its own. WidgetTexture hosts a subtree from your widget tree and streams it into a WidgetTextureController’s texture, which you can bind to any material slot yourself. WidgetComponent is that plus the scene-side wiring, so reach for the raw form only when the texture consumer is not a scene surface.

Performance notes

A capture rasterizes the child and publishes it to the texture, throttled to one in flight; when the child repaints faster than captures complete, intermediate frames are skipped and the texture converges on the latest content. A static subtree settles to near-zero cost. The texture object can be replaced across captures, so always consume it through the bind callback or by listening to the controller rather than caching it once.

Key API

WidgetComponent, WidgetInput, WidgetUpdatePolicy, WidgetTexture, WidgetTextureController, and ScenePointer.