Google DeepMind's Dream-RSI lets AI agents learn from replaying their past experiences like dreams

AI agents tackling complex problems face a massive search space where success depends on knowing which approaches to pursue, test in parallel, or drop. Google DeepMind researchers have unveiled Dream-RSI, a new method that improves those exploration decisions — without touching the underlying model. The technique could make agent-based problem-solving dramatically cheaper and more effective.

Replaying Search History Instead of Re-Running It

Current exploration strategies fall into two camps. Fixed strategies cannot learn from experience, so agents keep stumbling into the same dead ends. Adaptive strategies avoid that rigidity but require many costly attempts to prove their worth.

Dream-RSI sidesteps both problems by reusing data from completed searches. As an agent works, it records every attempt and its outcome. Later, alternative strategies can be tested against that stored search tree — no new model calls or evaluations needed. The researchers liken it to navigating an unfamiliar city: once you have a mental map, you can plot a new route without revisiting every street.

The team calls this replay process "dreaming." The agent runs through thousands of strategy variations offline, picks the best one, and applies it to the next live search. The loop repeats: search, dream, improve, search again — with only the strategy evolving, never the model itself.

Fewer Attempts, Better Results

The team evaluated Dream-RSI with Gemini 3.1 Pro and Gemini 3.7 Flash on eight tasks across three domains, benchmarking against identical baselines with fixed search strategies.

In one task, the system wrote the fastest possible program for a statistical calculation used in genomics and finance. The result beat established libraries sklearn and glmnet on all six test datasets. With Gemini 3.1 Pro, average runtime dropped from 3,587 to 2,931 milliseconds, while attempts fell from 550 to 317. A competing system, SimpleTES, required 51,200 runs to achieve comparable results.

On GPU kernel optimization, Dream-RSI matched baseline performance with up to 2.43x fewer runs, or achieved up to 2.09x higher performance within the same compute budget. Math optimization tasks showed the same pattern of equal or better results at lower cost.

Explicit Instructions Can Backfire

A follow-up experiment tested a different use of search histories: condensing them into explicit instructions guiding where to search. On one GPU task, this instructed version performed worse than the replay-based version. The researchers believe overly specific directions can over-narrow the search space, blocking broader exploration.

The analysis also revealed how the learned strategy manages effort dynamically. It initially reduces attempts as performance improves, then ramps search effort back up when progress stalls — a shift that coincided with renewed gains. Code and further details are available on GitHub.

Part of a Broader Recursive Self-Improvement Trend

Dream-RSI arrives amid rising interest in recursive self-improvement, a field Anthropic CEO Dario Amodei recently flagged when warning about the accelerating pace of AI research.

Google DeepMind's AlphaEvolve, introduced in 2025, works on a similar principle: Gemini Flash proposes code, Gemini Pro analyzes it, and an evolutionary algorithm selects the best versions. Dream-RSI operates one level above, optimizing the search strategy itself. AutoTTS takes a related path, using a coding agent to discover algorithms in simulation that govern when a language model should start, extend, or abandon reasoning chains — beating hand-designed methods at lower compute.

Google Research's WikiSkill offers an alternative reuse mechanism, converting recorded successes and failures into wiki-based instructions. But Dream-RSI's follow-up analysis suggests such explicit instructions can constrain open-ended exploration. Meta pushes the concept further with Hyperagents, which can rewrite their own improvement mechanisms. How these approaches compare in practice will be worth watching as agent search costs come under increasing scrutiny.

Meta description: Google DeepMind's Dream-RSI lets AI agents test thousands of search strategies offline by replaying recorded searches, cutting compute and improving results.

Tags: Google DeepMind, Dream-RSI, AI agents, recursive self-improvement, Gemini

Featured image: Abstract visualization of branching search paths and decision trees in deep blue and violet tones, digital technology style, no people, no logos, no text.