Picking and input

Interactive scenes need to answer “what did the user click on”. Flutter Scene does this by raycasting against the rendered geometry, the actual triangles on screen, with no colliders or physics setup. A hit tells you the node, the world-space point and surface normal, and even the texture coordinate under the cursor.

Try it below. Tapping casts a ray from the camera through the tapped pixel; the nearest hit gets highlighted and a marker lands on the hit point.

▶ Run demo Loads a live Flutter Scene build

Tap a shape to pick it with Scene.raycast. The marker sits at the hit's world point.

From a tap to a ray

Picking starts by turning a screen position into a world-space ray. Camera.screenPointToRay does the unprojection; give it the tap position and the size of the view it was tapped in.

GestureDetector(
  onTapDown: (details) {
    final ray = camera.screenPointToRay(details.localPosition, viewSize);
    final hit = scene.raycast(ray);
    // ...
  },
  child: SceneView(scene, camera: camera),
)

A LayoutBuilder around the SceneView is an easy way to know viewSize.

Raycasting

Scene.raycast returns the nearest SceneRaycastHit, or null when the ray misses. The hit carries everything a pick usually needs.

final hit = scene.raycast(ray);
if (hit != null) {
  hit.node; // the node whose mesh was hit
  hit.distance; // along the ray, world units
  hit.worldPoint; // the intersection point
  hit.worldNormal; // the triangle's facing at the hit
  hit.uv; // interpolated texture coordinate (null if the mesh has no UVs)
}

Scene.raycastAll returns every hit sorted nearest-first, for when you need to see through the frontmost surface. To cast against a single subtree instead of the whole scene, use raycastNode.

Filtering what a ray can hit

Three controls narrow the test set. A node with raycastable set to false never participates, which is the right tool for cursors, markers, and other helper geometry that should not occlude picks (the demo’s marker does this). layerMask tests against Node.layers for coarse groups, and a where predicate handles anything ad hoc.

final hit = scene.raycast(
  ray,
  maxDistance: 30.0,
  where: (node) => node != playerModel, // don't pick yourself
);

Invisible subtrees are skipped by default; pass includeInvisible: true to test them anyway.

Scene pointers

For a persistent pointer rather than one-shot picks, ScenePointer wraps the raycast with pointer state: hover tracking, press capture, and event forwarding. Aim it each frame with pointAt (a screen position) or pointAlong (any world-space ray, for a crosshair or a gamepad cursor), then read hit for what it is over.

final pointer = ScenePointer(scene);
pointer.pointAt(cursorPosition, camera: camera, viewSize: viewSize);
final over = pointer.hit?.node; // hover feedback

ScenePointer is also the input layer for interactive widget surfaces embedded in the scene (Flutter UI rendered onto 3D geometry); it maps the hit’s UV into the widget and forwards presses and drags with standard capture semantics. A dedicated guide on widget surfaces is on the way.

Render picking vs physics queries

Scene.raycast answers “what visible surface did the ray touch”. The physics integration’s queries (raycasts against collision shapes) answer “what does the collision world say”, which is usually what gameplay logic wants. Use render picking for selection and cursors, physics queries for line-of-sight and projectiles.

Key API

Scene.raycast, Scene.raycastAll, SceneRaycastHit, raycastNode, Camera.screenPointToRay, Node.raycastable, and ScenePointer.