Hugging Face’s policy chief says built-in AI evaluators are not a simple fix

Meta description: Hugging Face chief policy officer Irene Solaiman says AI benchmarking needs reproducibility, clearer evaluator standards, and better harness testing.

Tags: AI benchmarking, Hugging Face, TechCrunch Disrupt, Anthropic, Accenture

Featured image: Abstract technology style with no real people, no company logos, and no text.

Background: Benchmarking Becomes a Reputation Test

AI benchmarking sits at the center of how labs compare products and defend their reputations. These evaluations are presented as neutral and accurate systems designed to measure whether one AI product is better than another. Yet Hugging Face chief policy officer Irene Solaiman told a TechCrunch Disrupt discussion last week that the methods behind them remain complicated and still maturing. For the industry to rely on benchmarking in the future, she argued, testing must become a more consistent science.

Development: Reproducibility and Embedded Evaluators Enter the Debate

The question is whether current tests can be trusted. Solaiman said trust is a weighted term, and a major part of trusting results is whether an evaluation can be reproduced. She tied reproducibility to access to the model itself, noting that open weights matter greatly in this area.

That access issue has long made reproducibility one of the stickier parts of benchmarking and a significant industry debate. Solaiman added that evaluations are an evolving science and that the field does not yet know how to meaningfully compare one test to another. She said tests can start to become ineffective relatively quickly, and the ecosystem of evaluations is not as thriving as it needs to be.

Over the last several months, embedded evaluators have become a popular idea. The concept places supposedly neutral third-party observers inside AI organizations. It gained attention as AI labs grappled with a rash of safety incidents that raised concerns about potential harms from their technology. Hugging Face was famously the target of one such incident. Instead of active government regulation to identify and mitigate model safety concerns, labs have pushed for third-party solutions.

Doubts remain about how neutral or effective those evaluators will be. Anthropic's decision to hire Accenture as its first evaluator spurred a good amount of backlash, with some critics mocking the idea that the consulting firm was qualified for the task. Solaiman said there is now a hunger for an embedded evaluator ecosystem, but the qualifications that determine who is an effective evaluator across what tasks need much more standardization.

Latest Progress: Evaluations Expand Beyond the Model

The evaluation space is also shifting from testing only the model to testing the entire system around it, including the harness and the agent. The harness is the tool a user employs to access the model, and Solaiman said it can have a large effect on performance. A weaker model paired with a better harness could easily outperform a frontier model.

She described agentic evaluations as really hard and really expensive. The field currently has more evaluations for models than for harnesses, and it is developing better evaluations for the holistic agent because that is where many users are getting value.

Current Stage: A Free-for-All Before Standardization

The current stage of AI benchmarking is close to a free-for-all. If benchmarking is to remain a useful tool for industry leaders, the discussion at TechCrunch Disrupt last week suggested a more uniform, industry-wide approach may be necessary. What to watch next is whether reproducibility, evaluator qualifications, and harness and agent testing move toward shared standards.