Do AI Agents Enable Open-Ended Scientific Discovery? Evidence from Station

A new system called Station has demonstrated that AI agents can independently conduct open-ended scientific research, recovering 62.7% of predefined sub-discoveries on average across three tasks derived from ICLR oral papers. The result substantially outperforms the strongest tested AI Scientist-v2 configuration, which recovered 20.6%, and Codex Multiagent-v2, which managed 15.4%. The research team built these tasks by providing agents with original research questions and experimental setups while withholding the published findings and disabling web access, forcing genuine discovery rather than retrieval.

Background: Why Open-Ended Discovery Is Hard for AI

Most AI research agents today operate in constrained settings: they search literature, run predefined experiments, or optimize known objectives. Open-ended scientific discovery is different. It requires choosing which questions to pursue, deciding when a line of inquiry has dead ends, and building cumulative knowledge across experiments. These are precisely the capabilities that current agent architectures struggle with. The Station team designed their benchmark to test exactly this gap, using real ICLR oral papers as ground truth because those papers represent work that the human research community judged both novel and significant.

Development: How Station Approaches the Problem

Station agents operate with several distinctive design choices. They independently select research directions rather than following a fixed experimental plan. They conduct experiments, and they build on reviewed findings stored in a shared knowledge base. The system incorporates Supervisor guidance and periodic Meta Reflection, mechanisms intended to encourage sustained investigation instead of early pivots to easier questions. This matters because agents left unconstrained tend to drift toward trivial or already-solved problems. The comparison against AI Scientist-v2 and Codex Multiagent-v2 matches cumulative experiment time, though the team notes that token budgets and model teams differ across configurations, a caveat worth keeping in mind when interpreting the headline numbers.

Latest Progress: Matching Concurrent Human Research

Beyond the three ICLR-derived tasks, Station was tested on two additional exploratory tasks. In both cases, agents produced findings that matched concurrent human research. In one case involving subliminal learning, Station discovered that restricting LoRA training to early layers could restore failed trait transfer, a result that aligned with human findings published around the same time. LoRA, or Low-Rank Adaptation, is a technique for fine-tuning large models efficiently by training only a small number of additional parameters. The fact that an agent independently arrived at a comparable conclusion suggests the approach can generate genuinely novel, verifiable insights rather than merely reproducing known results.

What Remains Unsolved

The team is releasing the code and full research records, including failures, so others can inspect both the discoveries and the dead ends. They identify scientific judgment as a key remaining challenge. Agents still spend substantial effort on questions human researchers find uninteresting, meaning the system lacks the tacit taste that guides human scientists toward promising problems. For readers tracking AI research agents, the release offers a rare look at raw research records rather than cherry-picked successes. The paper is available at arxiv.org/abs/2610.08927, with code at github.com/dualverse-ai/station-open-reseach and research records at dualverse-ai.github.io/station-open-reseach_data. The next milestone will be whether agents can improve their question-selection judgment, not just their experimental execution.