DeltaStreamer aims to achieve human-like agentic streaming video understanding.

Title: DeltaStreamer Builds Video Memory for Questions That Arrive Too Late

Meta description: DeltaStreamer is an open-source video agent that observes frames, organizes memory, and revisits visual evidence when questions arrive after key moments pass.

Tags: DeltaStreamer, open-source video agent, streaming memory, OVO-Bench, StreamingBench

Featured image: Abstract technology-style illustration of continuous video frames forming a searchable memory stream, with no real people, no company logos, and no text.

Background: Streaming video creates a memory gap

DeltaStreamer is an open-source agent designed for a difficult timing problem: video streams keep arriving, but questions may come later. It continuously watches video, remembers what happened, and returns to visual evidence when memory falls short. The core issue is that a streaming system cannot know which details will matter before a question appears. By the time a user asks about an earlier event, the crucial frame may already be gone.

This matters for systems that must answer about ongoing visual input rather than a fixed clip chosen after the fact. Instead of treating each incoming frame only as a new image, the agent builds a running record of the scene. It compares consecutive sampled frames and records both scene descriptions and visual changes. That gives the system a basis for later reasoning about what changed, not just what is currently visible.

The source describes a practical asymmetry: video keeps arriving, while questions can come at any time. A system that only describes the current frame may lose the ability to answer later about an event that has already moved out of view. DeltaStreamer's response is to record both the scene and its changes, creating material that can be organized and queried after the fact.

Development: Three capabilities turn frames into evidence

The project connects three capabilities in sequence. The first is observation. DeltaStreamer samples frames and compares consecutive ones, capturing descriptions of the scene and changes between them. The second is memory organization. The observations are stored as structured textual memory, and each memory includes pointers back to the original visual evidence. The third is on-demand investigation. When a query arrives, the agent formulates a query-driven plan, retrieves relevant memories, and revisits past frames if the stored text lacks an important detail.

Because the memory is textual, it is easier to search and plan with than raw frames. Because it retains pointers to frames, it can fall back to visual evidence when a detail matters. The query-driven plan is the bridge between those two states: a question triggers retrieval first, and the agent decides whether the stored memory is sufficient or whether the original frames must be revisited.

This architecture addresses a practical gap in answering questions about video. A memory record can help the agent reason quickly, but it may miss fine visual information. The pointers allow the agent to return to the original frames instead of relying only on a summary. The demo exposes that process: a user can upload a video, ask questions, and explore the agent's observations, memory, and planning process. The code and a locally deployable interactive demo are available.

Latest progress: Benchmarks, public code, and a coming application

The project reports evaluations on OVO-Bench and StreamingBench. It says DeltaStreamer shows stronger overall performance than multiple existing approaches. No individual benchmark scores are included in the source material, so the claim is presented as an overall comparison rather than a numeric result. The benchmark names appear in the evaluation claim, but the source does not provide scores.

The public project page is https://bbbbchan.github.io/DeltaStreamer/. The code and demo are available at https://github.com/BBBBchan/DeltaStreamer. The repository invites feedback, including a GitHub star or an upvote. The project also states that a new application built on DeltaStreamer is coming soon.

For now, the contribution is the workflow: continuous observation, structured memory with visual pointers, and query-driven retrieval of past frames. The next milestone to watch is how the announced application uses that foundation.