Home โ€บ Blog โ€บ Evaluation harnesses
Inside the ML Layer ยท Part 6 ยท Evaluation

Measuring the Machine: The Evaluation Harnesses That Decide What Ships

Every training run, RL experiment and release decision ends with a number. That number comes from an evaluation harness: software that formats prompts, calls the model, extracts answers, scores them and adds them up. Small choices at each step move scores by several points, benchmarks saturate within a year or two, and agent evaluations now need containers and sandboxes like production systems. This part explains how harnesses work, why the same model gets different scores, and how to read a leaderboard without being fooled.

FL
FrontierAGI Team
โš ๏ธ How this was researched (as of 27 Oct 2026) Sources are the papers and documentation for lm-evaluation-harness, HELM, Inspect, SWE-bench, GPQA and Humanity's Last Exam, Hugging Face's Open LLM Leaderboard announcements, the Leaderboard Illusion paper, LMArena funding coverage and Anthropic's paper on error bars for evals. Disclosure: Claude, made by Anthropic, helps write this site; we cite one Anthropic paper on statistics and mention Claude models only where third-party leaderboards list them. Recent leaderboard scores come from trackers that disagree and are marked โš ๏ธ. The score-sensitivity demo, confidence calculator and pass@k calculator are our own illustrations. See Reading the numbers.

1. Case file

๐Ÿ“
The evaluation layer, October 2026Layer 3 of 8 in the ML layer map
Main open harnesseslm-eval-harness ยท HELM ยท Inspect ยท lighteval ยท OpenCompass
Open LLM LeaderboardRetired 13 Mar 2025 after 13K+ models
LMArena$100M seed (May 2025) ยท $150M at $1.7B (Jan 2026)
Leaderboard Illusion (Apr 2025)27 private Llama 4 variants tested before release
GPQA Diamond198 questions ยท top scores ~94% โš ๏ธ
Humanity's Last Examtop scores ~45โ€“55%, trackers disagree โš ๏ธ

2. Timeline

Evaluating language models, 2018 โ†’ 2026

Tap a milestone. Colour: benchmarks, harnesses, arenas and leaderboards, agent and safety evals.

Tap a milestone.

3. What happens in one eval run

A multiple-choice question, end to end

Step through the five stages a harness runs for every question.


      

4. Same model, different scores

Each stage involves a choice, and each choice moves the score. A base model scored by comparing log-probabilities of the options can look very different from the same model asked to write its answer. A strict answer-extraction regex can mark correct answers wrong. Few-shot examples help base models and can hurt chat models. This is why papers that report "MMLU 85" often cannot be compared.

Move the knobs

An illustration of a hypothetical chat model whose "true" ability is 80%. Effect sizes are invented but in the range harness papers report. Not data about any real model.

๐Ÿ’ก The fix: report the harnessGood practice is to publish the harness, its version, the prompt template, the number of shots, the sampling settings and the extraction rule alongside the score. lm-evaluation-harness and Inspect both produce logs that record these; the Llama 3 paper and many system cards now describe their settings.

5. The harnesses

The same task, two harnesses


      

6. Feature matrix

Our reading of project documentation. โ— built in ยท โ— partial ยท โ€” not a focus. Tap a cell.

Tap a cell.

7. Benchmarks saturate

From launch to ceiling

Best reported score near each benchmark's launch, and a recent top score. Launch figures from the benchmark papers; recent figures from third-party trackers that disagree (hatched, โš ๏ธ).

Benchmarks follow a life cycle: launched with headroom, adopted, optimised against, saturated, replaced. MMLU (2020) gave way to MMLU-Pro and GPQA; Hugging Face rebuilt its Open LLM Leaderboard around harder tests in 2024 and then retired it in March 2025, saying it was "slowly becoming obsolete" and could encourage people "to hill climb in irrelevant directions". Two forces speed saturation: genuine progress, and contamination, when test questions leak into training data. Harnesses respond with private held-out sets, canary strings that let labs detect leaked data, and regularly refreshed question pools.

8. Statistics: is it really better?

Confidence interval calculator

95% intervals using the normal approximation to a binomial. Questions are treated as independent; clustered questions widen the interval further.

Presets:

A 3-point gain on a 200-question benchmark is usually inside the noise. Anthropic's 2024 paper "Adding Error Bars to Evals" (Miller) argued that labs should report standard errors, cluster them when questions share a source passage, and compare models with paired tests on the same questions, which is more sensitive than comparing two separate intervals.

9. Agent evals are infrastructure

Agent benchmarks such as SWE-bench (real GitHub issues, judged by the repository's own tests), OSWorld (computer use) and terminal tasks need a fresh container per attempt, tool access, time limits and logs of every step. A single run can take minutes to hours and cost real money in tokens; variance between runs is large. Harnesses like Inspect ship sandboxing (Docker and Kubernetes) and agent scaffolds, and SWE-bench runs each task in its own Docker image. In 2024 OpenAI released SWE-bench Verified, a 500-task subset screened by human engineers because some original tasks were under-specified or had faulty tests.

pass@k vs pass^k

If an agent solves a task with probability p per attempt: pass@k = chance at least one of k tries succeeds (research benchmarks); pass^k = chance all k succeed (reliability, as ฯ„-bench reports). Independent attempts assumed.

10. Human preference and arenas

Chatbot Arena (May 2023, UC Berkeley's LMSYS) asks users to compare two anonymous models and fits a Bradleyโ€“Terry model (an Elo-style rating) to millions of votes. It became the most-watched leaderboard, spun out as LMArena, raised $100 million at a $600 million valuation in May 2025 and $150 million at a $1.7 billion valuation in January 2026, and sells evaluation services to labs. Its critics matter for anyone reading the rankings:

11. Inside the labs

During trainingSmall, fast eval suites run on checkpoints to catch regressions and choose data mixes; the curves decide when to stop or roll back (Producing Intelligence Part 7).
Before releaseLarge capability suites, internal held-out sets, human evaluations and red-teaming feed the release decision and the model card.
Safety and dangerous capabilitiesEvaluations for cyber, bio and chemical uplift and autonomy, under frontier safety frameworks; results summarised in system cards.
Third partiesGovernment institutes (UK AISI, which built Inspect; US CAISI) and nonprofits such as METR, whose time-horizon measure tracks how long a task agents can finish.

12. Choosing a harness

Which harness fits?

An editorial guide.

VerdictEvaluation is the AI factory's quality control, and it is software with as many design choices as the model. Use lm-evaluation-harness or lighteval to compare open base models, Inspect for agentic and safety evaluations, and HELM when you need many metrics beyond accuracy. Whatever you use, publish the settings, report error bars, prefer paired comparisons, and treat any single leaderboard rank with suspicion. The evals that matter most to frontier labs are the private ones you will never see.

13. Reading the numbers

Primary
Open LLM Leaderboard retired 13 Mar 2025, 13K+ models evaluated (Hugging Face). Leaderboard Illusion: 27 Llama 4 private variants; Google 19.2%, OpenAI 20.4%, 83 open models 29.7% of data (arXiv 2504.20879, preprint, disputed by LMArena). GPQA Diamond: 198 questions. SWE-bench Verified: 500 tasks (OpenAI, 2024). Launch scores: MMLU GPT-3 43.9% (2020); GPQA GPT-4 ~39% (2023); HLE frontier models under 10% (Jan 2025); SWE-bench Claude 2 1.96% (2023).
Reported / secondary โš ๏ธ
LMArena: $100M at $600M (May 2025), $150M at $1.7B post-money (Jan 2026), ~$30M revenue run-rate (Dec 2025) per press. Recent tops: GPQA Diamond ~94%, HLE ~45โ€“55% (trackers disagree); MMLU ~90%+.
Our illustrations
Score-sensitivity knobs (invented effect sizes), intervals (independent questions, normal approximation), pass@k (independent attempts) and the chooser.