VibeEdit: Canvas-Instructed Image Editing

Text-guided image editing has a well-known failure mode: describing what you want changed is easy, but pointing to exactly which object should change is not, particularly when several objects in a scene look alike. A new research effort introduces an editing interface that sidesteps that problem by letting users draw spatial marks and attach short notes directly onto the image. Together, those annotations form a "canvas instruction" that specifies both the location and the nature of the edit. The accompanying editor, VibeEdit, executes object addition, removal, replacement, attribute modification, and movement from those annotations alone, without requiring a separate text prompt.

Background: Why Text Prompts Struggle With Ambiguous Targets

Conventional text-instructed editors rely on language to carry all the intent. That works when a scene contains one obvious candidate, but degrades when a user needs to distinguish between similar objects. Ambiguity in the prompt forces the model to guess, and errors in target selection propagate through the rest of the edit. The researchers frame spatial marking as a more natural way to disambiguate, since humans already gesture and point when describing changes. By combining marks with optional short notes, the interface keeps the specification compact while removing the guesswork about which region is meant.

How VibeEdit Was Built

The team constructed 1.55 million source-target edit pairs, each paired with object masks and structured edit descriptions. Canvas instructions were rendered from this data during training, so the model learns to interpret marks and notes as editing directives rather than as image content to reproduce. The editor itself adapts Qwen-Image-Edit with layer-decoupled conditioning, which encodes source images and canvas instructions through separate pathways. Training proceeded in two stages: region-weighted supervised fine-tuning, followed by rubric-guided reinforcement learning. Those stages target three goals — completing the requested edit, maintaining local edit quality, and preserving the regions the user did not touch.

Benchmark Results Against a Text-Instructed Baseline

Evaluation used an independently constructed, human-curated benchmark of 419 cases, deliberately weighted toward target selection among similar objects. VibeEdit reached a VLM rubric score of 79.9 and an outside-region PSNR of 32.8 dB. The strongest text-instructed baseline in the evaluation, FireRed, scored 67.4 on the rubric and 24.0 dB on outside-region PSNR. The outside-region metric matters because it measures whether untouched parts of the image survive the edit intact — a persistent weakness of prompt-driven tools.

What to Watch Next

The 419-case benchmark is small and built by the same researchers who built the editor, so independent replication would strengthen the comparison. The more consequential question is whether canvas-style instructions become a standard input mode for editing tools, or remain a research interface. For now, the numbers suggest that specifying where explicitly, rather than hoping a prompt implies it, produces measurable gains in both edit accuracy and preservation of everything else in the frame.