StreamVibe Uses Proactive Video Agents to Anticipate Evolving User Intent

Meta description: StreamVibe-Bench covers 4.1K samples, and StreamVibe-VL adapts to implicit and dynamic intents, lifting answer quality by 10.7% and interaction by 12.3%.

Tags: StreamVibe-Bench, vision-language models, proactive AI, user intent, streaming inference

Featured image: abstract flowing video frames around a network of intent nodes, no text, logos, or people.

Vision-language models are shifting from offline passive understanding toward online proactive interaction. StreamVibe-Bench introduces a testbed for user intent understanding and interaction, built from 4.1K evaluation samples and 85K training samples spanning explicit, implicit, and dynamic intents. The benchmark is built for cases where a system must act before an explicit request arrives and keep its judgment current as the user’s goal changes.

Background: passive understanding is not enough

Prevailing methods largely rely on explicit requests. In a real interface, however, a user may be navigating a website without typing or speaking the next goal. The model must infer the implicit intent, such as where the user should click next, and then update that inference as the intent evolves. These limitations make such models less suitable for intent-aware interaction in real-world scenarios.

StreamVibe-Bench reframes the problem as a continuous one. Instead of treating user intent as a single prompt, it separates explicit, implicit, and dynamic intents. That structure gives a testbed for evaluating when a system should respond, what it should respond to, and how well it can follow intent over time.

Development: StreamVibe-VL infers intent from live observation

Building on the benchmark, StreamVibe-VL is a streaming proactive model. It infers implicit user intents from continuous observations and dynamically adapts its interactions as those intents evolve. This changes the model’s task from answering a fixed question to maintaining a running interpretation of the user’s likely next step.

The result is not only about content accuracy. Online interaction also depends on timing. A reply that arrives too early or too late can miss the moment when assistance is useful. The experiments show that StreamVibe-VL preserves strong general video understanding capabilities while improving response timing and answer quality by 10.7% over existing baselines.

Latest progress: StreamVibe-Harness adds persistent intent memory

StreamVibe-Harness addresses another bottleneck: sustained interaction. It maintains user intents as persistent states with flexible lifecycles through hierarchical memory, enabling it to track the latest user intents and environmental states.

This matters because a user’s goal can appear, pause, shift, or reappear as the environment changes. A single inference can become stale quickly. Hierarchical memory gives the system a way to keep relevant intent states alive while still responding to new observations.

What the results show

The experiments reveal three findings. First, StreamVibe-VL improves response timing and answer quality by 10.7% while preserving strong general video understanding. Second, existing proactive models remain limited in understanding user intents and determining when to respond, especially under implicit and dynamic intents. Third, StreamVibe-Harness further improves response consistency and adaptation during sustained interactions, resulting in a 12.3% gain in proactive interaction performance.

The common thread is that online proactive interaction needs both perception and state management. Recognizing a scene is not enough; a system must also keep track of what the user is likely trying to do and update that track as the scene changes.

What to watch next

StreamVibe-Bench provides the measurement layer, while StreamVibe-VL and StreamVibe-Harness provide a model and memory path. The 4.1K evaluation and 85K training samples set the starting point for testing whether streaming models can move from controlled benchmarks to sustained, intent-aware interaction.