Velox uses spatial memory to make world models efficient.

Velox is a training-free attention router designed to make long video world model rollouts more responsive. It accelerates attention by 4.36 times and denoising by 2.43 times at chunk 48, while staying within 0.15 API-free average points of dense LingBot-Fast-Cam on the full 158-case WBench navigation split.

Background: Long Video Rollouts Meet Growing Memory Costs

Video world models turn actions and camera motion into long visual rollouts, making video a natural interface for interactive simulation. That interface creates a practical tension: responsive generation is needed, yet coherent rollouts must retain visual memory of what has already been generated. Chunked video diffusion models usually keep this context in growing KV caches, so later chunks become progressively slower. The issue is therefore a scaling problem. As the rollout grows, the amount of stored context grows, and attention must spend more time consulting that context. The reported speedups target this later-chunk bottleneck.

Development: Frustum Slabs Replace Uniform Context Retention

To address the bottleneck, the work introduces Velox, a training-free attention router for region-level context selection using frustum slab correspondences from streaming geometry. Instead of treating all stored context with the same fixed importance, Velox represents historical blocks as camera frustum slabs with depth intervals. It obtains current-block depth through recency-priority multiwarp. For well-covered regions, frustum slab overlap softly modulates each layer's model-derived block importance. Uncovered regions retain the model-derived route without geometric bias. The fixed token-block granularity matches FlashAttention-style tiled execution and reduces the attention term that grows with rollout length.

Latest Progress: Speed and Benchmark Results on H200

The construction of frustum slabs runs asynchronously with denoising on the same H200. At chunk 48, Velox accelerates attention by 4.36× and denoising by 2.43×. On the full 158-case WBench navigation split, it remains within 0.15 API-free average points of dense LingBot-Fast-Cam. These figures place the result at the intersection of speed and benchmark distance: a later-chunk speedup paired with a small measured gap to the dense comparison.

Current Stage: A Routing Layer for Interactive Video Generation

Velox sits between the memory cost of chunked video diffusion and the need for long, coherent, responsive rollouts. Its reported design uses streaming geometry to guide region-level context selection, preserves model-derived routing where geometric coverage is weak, and aligns context selection with tiled attention execution. The next point to watch is how the same frustum-slab routing behaves as rollouts become longer, and whether the asynchronous construction on H200 continues to keep attention growth under control across more chunks. More details are available on the project page.

SEO Summary

Meta description: Velox, a training-free attention router, speeds video world model attention by 4.36x and denoising by 2.43x at chunk 48 while staying within 0.15 points.

Tags: Velox, video world models, attention routing, WBench, H200

Featured image: Abstract technology style visual showing camera frustum slabs and attention routing, with no real people, no company logos, and no text.