TokenCast: Predicting Token Usage During LLM Agent Execution

Meta description: TokenCast forecasts LLM agent token consumption from call counts, context growth, and cost residuals, lowering error and reducing agent token spend.

Tags: TokenCast, LLM agents, token forecasting, agent budget control, SWE-bench Verified


TokenCast is a token consumption forecasting method for LLM (large language model) agent execution, and its reported results point to a practical way to reduce one of the messiest costs in agentic software: deciding how many tokens a task will use before the task ends. The method addresses a pattern that is common in agent systems. The same task can consume over an order of magnitude more tokens across runs, because an agent's next steps depend on tool feedback, and the growing context inflates the input size of every later call.

Background: Why Agent Runs Make Token Use Hard to Forecast

An LLM agent usually runs as a loop: the model receives a prompt, chooses an action, calls a tool or returns an answer, then sees the result and plans another step. Each round adds tokens to the conversation history. A search result, error message, file listing, or API response may become part of the next prompt. This makes execution costs harder to estimate than single-shot model calls.

TokenCast is positioned around that problem. Rather than treating token use as a single opaque output, it breaks execution into segments and models the factors that make later calls more expensive.

Development Process: Turning Token Usage Into Composable Segments

The core idea is a composable cost representation for each execution segment. The segment features listed in the source are call count, net input-length change, and cost residual. Call count captures how many model interactions have occurred. Net input-length change reflects how much context has grown or been trimmed between calls. Cost residual represents the portion of usage not explained by those simple variables.

The method also uses an exact composition identity that captures the extra input cost of re-reading earlier context in every later call. That detail matters because context accumulation is a major driver of agent spend: a long task may pay for the same background again and again.

On top of that representation, TokenCast uses a staged prefix-suffix predictor. It combines direct forecasts with compositional forecasts, then refreshes estimates from newly observed execution evidence. In other words, the prediction is not assumed to be final at the start of the run; it can be updated as the agent produces more steps. Calibrated quantile models provide prediction intervals, giving a range rather than a single point estimate. The reported design also uses no additional LLM calls, which keeps the forecasting layer lightweight. In the evaluation setting, the mean cumulative prediction time is 32.8 ms per run on SWE-bench Verified.

Latest Progress: Accuracy, Intervals, and Budget Replay Results

The reported accuracy results are comparative. Across 4 task suites and 6 agent models, TokenCast reduces mean absolute error against the strongest comparator by 14.5% on average over 96 evaluated combinations. This is not a claim of perfect prediction; it is a measured improvement over a baseline under multiple settings.

The budget-control result is more operational. In an offline budget-control replay, TokenCast uses 21.3% fewer tokens on average than a fixed-budget policy at matched trace completion. A fixed-budget policy is a simple control: allocate a set amount and stop or proceed according to that allocation. The replay result suggests that a better token forecast can let a system make more informed choices about when to continue, trim context, or end a run, while still completing the same trace.

Current Stage: What the Numbers Show About Agent Cost Control

At this stage, TokenCast appears as a forecasting and budgeting technique evaluated across multiple task suites, agent models, and an offline replay. The key numbers show improved error and lower token use under matched completion. What remains to watch is whether such forecasts move into live agent schedulers, where prediction intervals could shape context management, call limits, and budget allocation during execution. The broader signal is that agent cost control is becoming a modelable problem: token use can be decomposed, estimated, and updated as an agent runs, rather than treated as an unavoidable surprise at the end of a task.