ViRDM: Mastering Representation Distribution Matching for Few-Step Causal Video Generation

Meta description: ViRDM four-step causal video post-training recipe reports 84.87 VBench Total, 20 generator updates, and single 80GB A100 GPU support released.

Tags: ViRDM, causal video generation, few-step post-training, VBench, A100

Featured image: abstract technology illustration of a timeline with video frames, neural representation distributions, and GPU memory graphs; no people, logos, or text.

ViRDM video post-training has moved from an image-inspired idea to a concrete few-step causal recipe after the release of training code, inference code, model weights, and reference artifacts. The project matters because it reports 84.87 VBench Total after only 20 generator updates, with approximately 2 hours of post-training on 8×A100, and it also supports training on a single 80GB A100. That combination lowers a common barrier: few-step causal video adaptation has often depended on DMD pipelines that use more compute and memory.

Background: Image RDM Points Toward Video Adaptation

The starting point was the strong image-generation result associated with FD Loss and RDM. In that setting, training can focus on matching a precomputed representation distribution and updating only the generator. The question ViRDM asked was whether the same principle could carry into few-step causal video generation, where the model must produce temporally coherent frames while still using a limited number of denoising steps.

A direct transfer would remove the online diffusion teacher and learned critic commonly used in DMD-based methods such as Self Forcing and Causal Forcing. That change is attractive for post-training because it can simplify the update loop. It is also risky: video generation adds temporal structure, longer computation graphs, and motion fidelity constraints that do not appear in one-step or few-step image generation.

Development: Memory, Initialization, and Motion Dynamics

The first obstacle was engineering rather than modeling. Multi-step causal video generation has a much longer backward path than one-step image generation. Backpropagating through the video VAE is particularly memory-intensive, so the ViRDM team had to solve out-of-memory issues before the training recipe could be evaluated. This is where the single 80GB A100 launcher becomes relevant: it shows that the final pipeline can fit within a setup that common DMD pipelines find challenging.

The second obstacle was population size and initialization. Image RDM uses very large generated populations and can start directly from a pretrained bidirectional model. ViRDM could not simply copy those settings for few-step causal video generation. The team had to rethink how many generated samples are practical and how a causal video model should be initialized when the target is a short-horizon, step-limited generator.

The third obstacle was motion. Matching feature distributions is not the same as preserving dynamic behavior. Even when video representation features were used, matching those distributions alone did not fully resolve the degraded dynamics issue. The release frames this as one of the key lessons: representation matching must be adapted to the temporal demands of video, not treated as a drop-in image solution.

Latest Progress: A Four-Step Causal Recipe and Public Artifacts

After ablated experiments, ViRDM arrived at a final four-step causal recipe. The paper evaluates it against DMD-based baselines and reports better results, with substantially less post-training time and lower peak memory. The headline numbers are 84.87 VBench Total, only 20 generator updates, and approximately 2 hours of post-training on 8×A100. The single 80GB A100 training path is presented as a supported option rather than a theoretical possibility.

The public release includes training and inference code, model weights, precomputed reference artifacts, reproduction configurations, evaluation scripts, and a single-80GB-GPU training launcher. The website and demos are available at https://neu-vi.github.io/ViRDM/, the code at https://github.com/neu-vi/ViRDM, and the paper at https://arxiv.org/abs/2609.28923. The authors thank the FD Loss, RDM, Self Forcing, Causal Forcing, and other related work for providing the foundation.

At this stage, ViRDM is best read as a completed recipe with evidence and artifacts, not as a universal replacement for DMD pipelines. The next signal to watch is whether follow-up video post-training work can reproduce the 20-update, 84.87 VBench Total setting while adapting the same representation-matching logic to new backbones, longer horizons, or different video VAE stacks.