Teaching Unified Models to Reflect Natively via Interleaved Reinforcement Learning

Unified multimodal models that can both see and draw images now have a concrete path to repairing their own generations, and UMM-Reflection shows that the path matters. The method trains a single model to diagnose an image, revise it, inspect the result, and revise again, instead of treating reflection and rendering as separate problems. On BAGEL, full-loop reinforcement learning lifts GenEval by 12.05 points over supervised fine-tuning, with gains also reaching WISE, OneIG-Bench, and T2I-CompBench++.

Background: Self-repair Needs a Joint Learning Loop

A unified multimodal model can take an image as input and produce an image as output. In principle, that dual role lets the model check its own generation: it can identify a flaw, write a revision, render a new image, then evaluate whether the fix worked.

The difficulty is timing. A reflection note may sound useful, but it only proves useful after the image is rendered. If reflection text and image generation are trained separately, the model may learn confident diagnoses that do not improve outputs. Supervised fine-tuning on reflection trajectories gives the system a cold start, but it does not reliably discover high-success repair paths. Reinforcement learning that optimizes only the renderer, or only one model head, leaves much of the possible gain untapped because the loop is broken into disconnected rewards.

Development: UMM-Reflection Trains the Whole Repair Trajectory

UMM-Reflection applies reinforcement learning to complete reflection trajectories inside one unified model. A trajectory is the chain from initial image to diagnosis, revision, rendered result, and follow-up reflection.

The design uses sibling trajectories that share one initial image. Because the starting point is fixed, the model can compare different reflection strategies against the same visual problem. The group-relative advantage evaluates which strategy is better, while one trajectory-level advantage updates both the reflection tokens and the flow-based revisions. This avoids the combinatorial blow-up of assigning credit round by round, where each step would need separate blame or reward.

The result is a method where learning credit flows across rounds and across the two roles of the same model: the reflection role and the image-generation role. Unlike single-round editing or pipelines with an external critic, the model does not need a verifier at inference. The repair loop is learned inside the unified system.

Latest Progress: BAGEL Scores Rise and Transfer Across Benchmarks

The reported gains come from BAGEL. UMM-Reflection improves GenEval by 12.05 points over SFT. That number is the central evidence that joint reflection-and-rendering training can move a unified model beyond a supervised cold start.

The gains also transfer beyond the main benchmark. WISE rises by 10.97 points, OneIG-Bench by 3.48 points, and T2I-CompBench++ by 4.63 points. These are separate evaluation scores, so the pattern suggests the improvement is not limited to one narrow test set. The result does not claim a universal fix for all multimodal image tasks, but it shows that full-loop RL can improve the same model across multiple benchmarks.

For readers, the increment is practical: the method treats multimodal self-correction as a single optimization problem. That matters for systems expected to produce and audit visual outputs without a separate checker, because each round of repair can influence both the quality of the next image and the quality of future reflections.

Current Stage: Full-Loop RL Becomes the Next Training Layer

UMM-Reflection sits between supervised reflection training and external critic pipelines. It keeps the model unified, uses complete repair trajectories, and assigns advantage at the trajectory level. The BAGEL numbers show that the approach can produce large benchmark gains, especially on GenEval, while preserving transfer to other evaluation suites.

The next question is how far this full-loop pattern can generalize across model families, image domains, and longer revision chains. The reported evidence is tied to the named benchmarks and the 12.05, 10.97, 3.48, and 4.63 point gains, so the strongest claim is that joint reflection-and-rendering RL is now a measurable step forward for unified multimodal image repair.

Meta description: BAGEL UMM-Reflection adds 12.05 GenEval points over SFT and transfers gains to WISE, OneIG-Bench and T2I-CompBench++ through full-loop RL training.

Tags: UMM-Reflection, BAGEL, unified multimodal models, GenEval, reinforcement learning

Featured image: Abstract technology style, geometric neural network nodes and image-generation loop symbols, no real people, no company logos, no text.