Insights into Agentic Black-Box Optimization: Benchmarking LLM Agents

Meta description: AgenticBBO-Bench tests LLM agents across five black-box optimization domains. Agents beat tested numerical optimizers in four domains, and code is available.

Tags: AgenticBBO-Bench, LLM agents, black-box optimization, benchmark, numerical optimizers

Featured image: Abstract technology illustration of optimization landscapes, connected evaluation paths, and agent decision loops, with no people, logos, or text.

Background

Black-box optimization asks a system to improve a solution without seeing the internal formula for the objective. Instead, it evaluates candidate answers, receives scores, and decides what to try next. That pattern appears when direct gradients or analytical shortcuts are unavailable, so the quality of search strategy becomes central. AgenticBBO-Bench was introduced to compare LLM agents against numerical optimizers in a unified setting.

Development Process

The benchmark covers five black-box optimization domains. It tests whether language-based agents can use task semantics, tool actions, and iterative feedback to search better solutions. The comparison also checks a practical assumption: more tools may not always produce better optimization. In the reported results, task semantics were broadly useful, while specific priors were less reliable.

Latest Progress

LLM agents outperformed the best tested numerical optimizers in four of the five domains. The project also reports that numerical optimizers can build on gains from agent-generated search trajectories. The code is available at github.com/lamda-bbo/agentic-bbo. The next signal to watch is how far agent trajectories can strengthen conventional optimizers beyond this benchmark.