WorldDraft: If Code Builds the World, Can Video Models Just Render It?
Abstract
Multimodal coding agents decouple 4D world construction from visual generation, repurposing video models from world generators toward generative renderers. This division of labor raises a fundamental question: if code builds the world, can video models just render it? A reliable generative renderer hinges on two complementary criteria: (i) world preservation, retaining prescribed 4D kinematics and camera paths; and (ii) visual realization, persistently binding text-specified target attributes over time. To evaluate this capability, we introduce WorldDraft, a benchmark of 432 strictly paired test cases across 48 executable 4D world families spanning six dynamic challenge categories. Under a 3 times 3 factorial design, WorldDraft orthogonally crosses draft granularity (abstract proxy to previs) and appearance alignment (neutral to conflicting) under identical dynamics. Extensive evaluations across 12 representative models reveal that: (I) world preservation and visual realization remain in stark operational tension; (II) richer draft scaffolding anchors structure but paradoxically impairs appearance fidelity; and (III) conflicting source cues trigger severe source leakage (up to 64%), exposing fragile cross-modal disentanglement. To mitigate these bottlenecks, we propose WorldDraft-Primer, a plug-and-play visual prompting strategy that consistently improves both objectives across frozen video renderers. Together, WorldDraft and WorldDraft-Primer lay a foundation for bridging programmatic 4D simulation with video generation, advancing physically grounded generative rendering.
Community
If Code Builds the World, Can Video Models Just Render It?
Calling code the world builder and the video model a renderer is a useful split, and the tension between keeping the 4D structure and getting the text-specified appearance matches what I would expect. Source leakage up to 64% under conflicting cues is striking. Does a richer draft hurt appearance because the model copies the draft textures, or because it ignores the prompt?
"Code builds the world, the renderer respects it" is the split we found useful for a 3D house model at ML Systems. Once every stud has a real dimension in code, the picture becomes the easy part. I'd be curious which models kept geometry intact under the conflicting appearance prompts.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- PhysOPSD: Code as Privilege for Physically Plausible Video Generation (2026)
- PROWBench: Do Video Models Render What the Program Specifies? (2026)
- Code Plans, Diffusion Renders: Open-Ended Generative World Modeling (2026)
- Proxy2World: Learning to Generate Worlds From Lightweight Proxies without Seeing Them (2026)
- Beyond Legibility: Benchmarking Visual Text Rendering and In-Place Editing in Unified Video Generation (2026)
- Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model (2026)
- BlenderWorld: Benchmarking Visual Agents for 3D Creation from Real-World References (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2611.16217 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper