BlenderWorld Benchmarks Visual Agents That Create 3D Content from Real-World References

The Qwen team has introduced the BlenderWorld benchmark to measure how well visual agents can complete real-world 3D creation tasks, not merely produce attractive images. The benchmark turns Blender projects into testable deliverables, giving developers a way to compare models on spatial reasoning, tool use, and iterative visual feedback.

Background: Visual Agents Move From 2D Tasks to 3D Creation

Before this release, visual agents were already used to build websites, design posters, and create slide decks. Those tasks have clear visual outcomes and often operate on flat layouts. 3D creation is harder because it requires an agent to understand depth, object relationships, materials, and functional structure while working with tools that can change a scene step by step. Existing benchmarks offered limited coverage of realistic, open-ended 3D tasks. BlenderWorld was built to address that gap by testing complete project creation rather than isolated renders.

Development: BlenderWorld Turns 3D Output Into a Testable Benchmark

The benchmark contains 223 realistic tasks across 9 domains and 22 subcategories. The domains include architecture, product design, nature, character animation, and scientific visualization. Task inputs can be text, images, videos, or structured specification files. The expected output is not a single render but a complete, editable Blender project. Because the deliverable remains editable, the benchmark can test whether an agent understands the structure behind the image, not only its final appearance.

To judge that level of detail, the team defined 2,651 task-specific, fine-grained criteria. These criteria cover 13 aspects, including geometry, spatial layout, materials, and functionality. During evaluation, an agent automatically selects and renders 8 key views. A Vision-Language Model-as-Judge evaluates the visual output, while Agent-as-Judge inspects the underlying project structure. Overall quality is also assessed across 8 dimensions, including realism and physical plausibility.

Latest Progress: GPT-6-Astra Leads, But Perfect Task Completion Remains Rare

The team evaluated 13 frontier models. GPT-6-Astra leads with a score of 79.9. Even so, the strongest model satisfies every fine-grained requirement on only 42.2% of tasks. That result shows that high scores do not yet equal reliable 3D authoring. The evaluation results point to continuing gaps in interpreting multimodal references, meeting detailed requirements, and handling physical and spatial relationships.

The benchmark's key numbers show its scope: 223 tasks, 9 domains, 22 subcategories, 2,651 criteria, 13 measured aspects, 8 rendered views, 8 overall quality dimensions, and 13 evaluated models. The interactive 3D gallery lets visitors explore, rotate, and compare creations from different models directly in a browser. The paper is available at https://arxiv.org/abs/2610.23986, the project at https://blenderworld.github.io/, and the code at https://github.com/QwenLM/BlenderWorld. The code and dataset are publicly available, and the team notes that users can star the GitHub repository if they find BlenderWorld useful.

What the Benchmark Changes

BlenderWorld shifts evaluation of visual agents from simple visual comparison to inspection of full 3D projects. For readers, the value is a clearer picture of where current models stand: they can create useful Blender scenes, but many tasks still fail when judged by geometry, material, functionality, and physical plausibility. The public benchmark and gallery make those gaps measurable across models and task types.

Meta description: Qwen team introduces BlenderWorld, a 223-task benchmark with 2,651 criteria to evaluate visual agents on realistic 3D creation, plus 13 model scores.

Tags: BlenderWorld, Qwen team, visual agents, 3D benchmark, Blender projects

Featured image: Abstract technology style scene with 3D wireframe objects, rendered views, and evaluation panels; no real people, no company logos, no text.