Training Language Models for Belief Revision in Effective Long-Horizon Decision-Making
ABBEL is a new training framework that lets LLM agents maintain memory-efficient natural-language belief states instead of full interaction histories. An auxiliary belief grading RL task rewards beliefs that preserve task-relevant information, with an autoencoding-style reconstruction grader when domain heuristics are unavailable. On CollabBench, ABBEL recovered about half the gap to full-context policies while using far fewer peak tokens and only 50 training steps, suggesting paths toward multi-memory agent systems.
Key Takeaways
- ABBEL is a new training framework that lets LLM agents compress entire interaction histories into short natural-language "belief states," addressing the practical limit that long-horizon agents (hundreds to thousands of steps) can't keep full context forever.
- Conventional context compaction (self-summarization), used by systems like Cursor's composer 2.5 and Grandcode, degrades performance mid-task because high-quality training data and effective human simulators are hard to build.
- ABBEL's key innovation is "belief grading"—an auxiliary RL task that rewards beliefs preserving the right information, using either domain heuristics or an autoencoding-inspired grader that scores how well observations can be reconstructed from a belief.
- Results show ABBEL recovers roughly 50% of the performance gap between summarized-context and full-context policies (e.g., 0.48 vs. 0.52 test pass rate on CollabBench) while using less than half the peak tokens and training in half the steps.
- The authors envision multi-memory architectures where belief states act as working memory alongside other mechanisms like test-time training or adapter memories.
Researchers behind ABBEL, a new training framework for memory-efficient LLM agents, argue that natural-language belief states—short summaries that replace an agent's full interaction history—can be supervised directly, closing roughly half of the performance gap left by conventional recursive summarization. The work, detailed in the arXiv paper "ABBEL: Learning Natural-Language Belief States for Memory-Efficient Interaction" (arXiv:2512.20111, 2026), targets a real production pain point: agents that assist humans over hundreds or thousands of steps cannot keep entire interaction histories in context forever.
Why compaction degrades performance
The standard workaround today is context compaction—having a model summarize its own history as it works. Cursor's composer 2.5 model uses compaction during training, and Grandcode (DeepReinforce et al., 2026), the first system to consistently beat all human competitors in online coding competitions, still relies on context summarization despite running on the efficient-attention model Qwen 3.5-397B.
Yet compaction carries a hidden cost. Cursor's model servers continue to recommend users avoid compaction mid-task with their coding assistants. In Combination Lock, a Wordle-like game allowing up to 16 guesses, a context-summary model improved over reinforcement-learning fine-tuning but never closed the gap to full-context policies.
The researchers attribute this degradation partly to the difficulty of building effective human simulators for high-quality training environments, especially in domains like collaborative code generation where good data is scarce.
How belief grading works
ABBEL borrows from recursive Bayesian estimation: the model periodically updates a natural-language belief state from prior belief, action, and observation, then acts conditioned only on that belief. The key addition is belief grading—an auxiliary RL task rewarding beliefs that preserve the right information. When domain heuristics are hard to define (for example, "shorter is better, but closer to reconstructing the git diff is better"), a general autoencoding-inspired grader treats the current model as both encoder and decoder, scoring each belief by how well the latest observation can be reconstructed from it.
Results: half the gap, half the training steps
On CollabBench (from Sweet-RL, Zhou et al., 2025), where agents ask clarifying questions and submit functions scored against hidden unit tests, ABBEL with reconstruction belief grading (ABBEL-rec-BG) achieved a 0.48 test pass rate and 0.36 success rate versus 0.52 and 0.39 for full context—recovering about 50% of the gap while using 6.01×10² peak tokens versus 14.08×10², and training in 50 steps instead of 100. Without belief grading, ABBEL reached only 0.46 and 0.31. In Combination Lock, a domain-knowledge grader let ABBEL approach or exceed full-context learning efficiency, and in multi-objective QA (from MEM1, Zhang et al., 2025), a peak belief length penalty cut memory with minimal performance loss.
The team points toward multi-memory systems where belief states serve as working memory alongside test-time training or adapter memories. The research was advised by Alane Suhr and Kartik Goyal.
Comments
No comments yet. Be the first to comment.