TimeEvo: Self-Evolving a Time Series Agent Through Failure-Driven Learning

Headline: TimeEvo Replaces 21 Static Tools With Runtime Admission

Meta description: TimeEvo replaces static tool libraries by diagnosing gaps, adding evidence-only tools through a paired gate across ten tasks and three backbones.

Tags: TimeEvo, time series QA, AI agents, tool libraries, runtime evaluation

Featured image: abstract time series dashboard with linked tool nodes, no people, logos, or text.

TimeEvo changes how time series agents acquire external tools by building a tool library from diagnosed failures rather than relying on a fixed human-curated set. This matters because a tool's usefulness emerges during execution, question by question, while the tool set is created beforehand and scored with one aggregate average.

Background: Tools Chosen Before Execution

Agents for time series data solve analytical questions by invoking outside functions, such as routines that summarize, forecast, detect changes, or compute statistics from a sequence of observations. In the setup described, the list of tools is fixed before the agent runs. Human experts supply that list, and the agent uses whichever tools it finds among them. This design makes tool selection predictable, but it also hides a practical problem: the value of a tool depends on the question being asked, not only on the tool's reputation.

A fixed library can be useful on some tasks and harmful on others because it encourages the agent to reach for familiar operations even when a simpler or different measurement would be better. The authors call this failure Human-Agent Tool Misalignment. They illustrate it with a library of 21 expert-curated tools. That library helps some tasks while hurting others; it also drops anomaly accuracy, the score for detecting irregular patterns, under every backbone tested. A backbone here means the underlying language model that hosts the agent and calls tools.

Development: 21 Tools and One Silent Revision Round

The second failure mode appears after the agent has produced answers. The authors describe a round of generic self-revision, where the agent is asked to review and adjust its own responses without a task-specific reason to change them. In one such round, 147 answers changed and 56 of those changes broke the answer, while the final score moved by less than a point. They call this Silent Harm because the visible score barely changes, even though many answers were altered and many of those changes were wrong.

Both failures share one root cause. Usefulness is decided per question during execution, yet the available tools are assembled beforehand and evaluated through one overall average. That gap gives a misleading picture: an agent may look acceptable on aggregate while making avoidable errors on specific anomaly-detection tasks or while quietly damaging correct answers during self-revision.

Latest Progress: Building Libraries From Diagnosed Gaps

TimeEvo addresses the gap by changing how the library is formed. Instead of starting with a fixed set of expert tools, it begins by diagnosing where the agent fails. The diagnosed failures are clustered into capability gaps, meaning categories of missing measurement or reasoning ability. For each gap, TimeEvo plans a measurement, then synthesizes evidence-only tools that can fill that gap. Evidence-only tools are restricted to producing observable support for the question rather than guessing an answer directly.

The candidate library is not accepted simply because it looks useful. It must pass a paired admission gate, a selection step that admits the candidate library only after a paired comparison. This makes library growth tied to runtime need, not to a single pre-run average.

Current Stage: Empty Libraries, Ten Tasks, Three Backbones

The experiments cover ten time series QA tasks, question answering over sequences of observations, and three backbones. TimeEvo begins from an empty library, meaning it does not rely on the earlier 21 expert-curated tools as a starting point. Under those conditions, it raises accuracy across all ten tasks and all three backbones. The authors also report portability: a library grown on a cheap model still gains when it is installed into stronger ones.

The code is available at https://github.com/Muyiiiii/TimeEvo.

The work now points to a narrower but practical question for agent builders: which tools should enter a library only after they have survived a paired runtime test, and whether that test can keep anomaly accuracy from dropping when agents self-revise answers.