Intelligence Comes Free — So What's Next? <br> Data Systems Built for, Composed of, and Created by Agents

Inference costs for GPT-4-class AI capabilities have collapsed from roughly $30 per million tokens in early 2023 to under $1 today, with some providers pushing below $0.10. A new perspective paper led by Aditya G. Parameswaran, Associate Professor of EECS and co-director of the EPIC Data Lab at UC Berkeley, argues this collapse is about to transform the data systems field. The paper, part landscape survey and part research agenda, identifies three intertwined challenges: data systems for agents, of agents, and by agents.

How Inference Costs Fell 9x to 900x Per Year

Across benchmarks, inference prices have declined between 9x and 900x per year, with a median drop near 50x. Even frontier models are getting dramatically cheaper each generation, and open-source models are following closely behind. The authors argue that while "Nobel-Prize-winning genius-level" intelligence may not exist yet, intelligence sufficient for the vast majority of knowledge work is already available—and getting cheaper by the month. They call the coming period the era of virtually free intelligence.

Several of the research directions in the paper, including agentic speculation, structured memory, and synthesizing custom data systems from scratch, draw on the authors' own ongoing work.

Data Systems For Agents: Handling Agentic Speculation

An agent querying a database behaves nothing like a human or a BI tool. Instead, it engages in what the authors call agentic speculation: a high-volume, heterogeneous stream of work spanning schema introspection, columnar exploration, and partial-to-full query formulation. A single user request—say, root-cause analysis of why coffee sales in Berkeley dropped this year, or exploratory cohort analysis of which user segments are most likely to churn next quarter—could generate 1000s of individual SQL queries as multiple agents explore the hypothesis space.

Much of that work is redundant. On a text-to-SQL benchmark with multiple agents attempting each task, only 10-20% of sub-plans were distinct, meaning 80-90% of sub-queries duplicate effort. Interestingly, task success rates rise significantly with more agentic attempts, so the redundancy helps agents even though it wastes system resources.

The paper suggests agent-first systems could reuse results across overlapping sub-plans, drawing on decades-old multi-query optimization and shared scans techniques. Systems could also "satisfice"—return approximate answers good enough for agents to progress, leveraging Approximate Query Processing (AQP) literature—or stream intermediate results so agents can decide whether more is needed. Other ideas include batch query interfaces with per-query approximation requirements, DBT-style Jinja macros as looping primitives, and making data systems proactive: offering latency estimates instead of executing expensive queries, or pre-building materialized and virtual views as agent context.

Data Systems Of Agents: Memory, Coordination, and Failure

As agents take on most knowledge work, they need a substrate to manage state over long-running tasks, coordinate, and handle failures—separate from the inference stack, which is already abstracted away through APIs from OpenAI or Anthropic, or serving frameworks for open-weight models. Today that substrate is handled by harnesses like Claude Code and Codex, plus mechanisms to store and retrieve memory.

The current wisdom—files are all you need—has agents writing unstructured markdown files searched via grep or embedding-based retrieval. The authors argue this breaks at scale: stuffing retrieved fragments into limited context windows will fail, and serializing large databases or codebases into context is often infeasible. Knowledge graphs suffer the same limitation for lack of structured search. Their answer is structured memory: organizing memory across attributes (columns, tables, operation types, plus open-ended corrective instructions) that can be matched precisely—for example, "when performing date-time operations, use fiscal year as opposed to calendar year conventions." Retrieved memory should also be corrective, since raw agent traces with mistakes induce repeated errors. Mining multi-agent traces could even enable recursive self-improvement.

Concurrency poses another challenge: when thousands of agents edit shared state, most speculative transactions must roll back while one "correct" result persists—raising questions about multiversioning, copy-on-write semantics, exactly-once semantics, and CRDTs. A key failure mode to avoid is "livelock," where endless compensating actions block progress. Durable execution systems like Temporal exist, but scaling to thousands of agents remains unproven, as do mechanisms for agents to negotiate consensus, such as four developer agents agreeing on a shared schema.

Data Systems By Agents: Trusting AI-Built Systems

The third challenge: agents are rapidly becoming capable of synthesizing entire data systems in one go, enabling custom rebuilds for each new workload. The catch is verification—ensuring generated systems match intended behavior. Trustworthy synthesis, the authors contend, is the gate on this future.

Where This Goes Next

The paper closes by examining how the three challenges intersect as agents and data systems intertwine. With inference costs still falling toward the median 50x annual decline, the central question for the database community is no longer whether agentic workloads arrive, but whether data systems can be redesigned fast enough to serve, host, and verify them.