One world for camera and object motion
Draw a camera path and move selected foreground regions independently. Both controls describe motion in the same 3D scene.
Composing Camera and Object Motion in 3D
Start with one image. Choreograph the camera and subjects in a shared 3D world.
Turn your motion into a generated video.
Click a scene to switch the image, 3D controls, and generated video below.
A movement on the screen can come from the subject, the camera, or both. A 2D trajectory alone leaves that relationship ambiguous.
We lift an image into a colored point cloud and give artists one shared coordinate system for the camera and foreground. The video model then turns this geometric guidance into a coherent shot.
Draw a camera path and move selected foreground regions independently. Both controls describe motion in the same 3D scene.
Each handle moves a selected group of points together. Several local handles can approximate articulated or non-rigid motion, without a skeleton or a physics simulator.
Three rasterized guidance maps tell the model where the background is, where a foreground handle is now, and which handle it is across frames and viewpoints.
Author in 3D, render the motion into image-plane maps, then condition a pretrained video generator. The controls carry geometry; the model supplies appearance and motion between them.
Estimate depth and a dynamic mask. Back-project pixels into a colored point cloud, using the first camera frame as the world frame.
Select local foreground regions in a 3D interface such as Blender. Author their trajectories and the camera path.
Project the static background and moving handles. Encode world XYZ and persistent foreground identity, with validity masks.
A frozen Wan VAE encodes the maps. A trainable side branch, residual projections, and LoRA adapters condition generation.
The Wan VAE and base diffusion weights stay frozen. The lightweight side branch, cross-branch convolutional projections, and main-branch LoRA adapters are trained. The side branch operates at half the main branch’s spatial resolution and injects geometric features through residual connections.
Move the camera, then move the handle. Watch the projected maps change. The identity color stays attached to the handle; its XYZ color changes only when its world position changes.
Conceptual illustration · no model inference
Fixed world positions. The same points keep the same colors across views.
Current world position. The color changes when the handle moves.
A fixed base color. It identifies this handle throughout the shot.
Try the camera slider: projections move, but the world-coordinate colors stay the same.
Both XYZ maps use a per-axis normalization derived once from the first-frame scene: bounding-box center o and half-range σ. Normalized X, Y, and Z become red, green, and blue. The same normalization is used in every frame.
A projected pixel says where the point appears; its XYZ value says where it is in the world. Because a moving point changes XYZ, the separate foreground identity map supplies its persistent handle label. This allows the model to interpret foreground motion relative to the static scene.
Swipe or scroll sideways to inspect the full diagram. Select it to enlarge.
Compare our result with seven baselines in one view. Browse the scenes, inspect the intended motion, and replay all outputs together.
Explore 13 selected examples from the supplied gallery. Each result follows an authored motion. Open a video to inspect it, or choose “Compare” to see its controls and baseline outputs.
Monocular depth errors and unseen surfaces can produce geometric inconsistencies, especially under large viewpoint changes, reflections, and thin structures.
Handles are better suited to local motion than to tearing, fracture, or fluid dynamics. Fine deformation can require many handles, increasing the authoring effort.
Independent trajectories do not impose physical or causal constraints. XYZ maps also use a fixed first-frame normalization range, which can be exceeded by large edits.
Project page accompanying the current manuscript snapshot · September 2026