Deep Dive ยท 4 of 5
Competitive Programming and Retrieval Agents
The field handbook used AlphaCode as a case study for how generation gives way to selection, and covered retrieval agents in a few paragraphs. This deep dive expands both threads โ AlphaCode's two generations, and two techniques for making search itself part of the reasoning loop โ with verified citations and original diagrams.
On the citations below: AlphaCode's identity (Science journal, DOI, full author list) is confirmed at high confidence. AlphaCode 2 was never published as a peer-reviewed paper or arXiv preprint โ it's a DeepMind technical report with no arXiv ID; its full ~29-person author list wasn't independently confirmed, though the report's existence, publisher, and headline results are corroborated across multiple sources. Search-O1 and Search-R1 are both high confidence, including Search-O1's EMNLP 2025 acceptance. No genuine companion video was found for Search-O1 or Search-R1 specifically โ don't assume one exists.
Two Threads, One Pattern
AlphaCode
AlphaCode 2
Search-O1
Search-R1
Comparing the Four
Confidence Notes & Gaps
Full Reference List
Two Threads, One Pattern
AlphaCode and its sequel are a case study in generation giving way to selection as the field handbook describes it. Search-O1 and Search-R1 are a case study in the same shift applied to a different resource: instead of asking "which of many generated programs is correct," they ask "when, during reasoning, is it actually worth pausing to look something up." Both threads are really the same lesson from different angles โ more of a resource (candidate programs, search queries) only helps once something decides when and how much of it to actually use.
AlphaCode: Brute-Force Generation, Then Filter and Cluster
Competition-Level Code Generation with AlphaCode HIGH
Samples up to roughly one million candidate programs per problem from a transformer trained on competitive-programming data, filters out candidates that fail the problem's public example test cases, then clusters the behaviorally similar survivors so the final handful of submissions aren't near-duplicates of each other. Evaluated across ten real Codeforces contests with 5,000+ participants each, AlphaCode achieved an average ranking around the top 54.3% โ an estimated Codeforces rating of 1238, roughly the level of an average recently-active competitor, not an expert.
๐ arXiv:2203.07814
๐บ A DeepMind co-author talk exists (David Choi, via a seminar series listing) โ host/date not independently confirmed
AlphaCode's original pipeline: scale of generation compensates for weak per-candidate reliability, with clustering preventing wasted, near-duplicate submissions.
AlphaCode 2: Learned Scoring Replaces Brute Force
AlphaCode 2 Technical Report MEDIUM-HIGH โ report, not a peer-reviewed paper
Built on a fine-tuned Gemini Pro model rather than the original's smaller transformer, still generating up to roughly a million C++ samples per problem, but now pruning them via filtering, behavioral clustering, and a learned scoring model that ranks surviving candidates โ shifting weight from pure brute-force volume toward selection quality. Evaluated on 12 Codeforces contests (77 problems, 8,000+ competitors), it reportedly solved 43% of problems and reached roughly the 85th percentile, up from the original AlphaCode's ~54th-percentile-from-the-top result โ moving from "average competitor" to roughly the Expert/Candidate Master boundary on Codeforces.
๐ Official technical report (PDF)
๐บ DeepMind published an official announcement/demo video alongside the report
What actually changed between the two: not the core idea (still generate many, then narrow down) but where the narrowing effort goes. AlphaCode's narrowing was almost entirely mechanical โ pass/fail on public tests, then behavioral clustering. AlphaCode 2 adds a fourth, learned step on top: a model trained specifically to judge which surviving candidates are actually most likely to be correct, rather than relying on clustering diversity alone to pick a good final set. That's the field handbook's "generation gives way to selection" pattern showing up as a literal architecture change between two versions of the same system.
Search-O1: Search as a Conditional Action, Not a First Step
Search-o1: Agentic Search-Enhanced Large Reasoning Models HIGH
Equips a long-reasoning model with a search action it can trigger conditionally, mid-chain, rather than retrieving documents once upfront the way standard RAG does. When the model detects it's missing knowledge it needs, it issues a query; a separate "Reason-in-Documents" module digests the retrieved passages before folding them back into the ongoing reasoning chain, which keeps irrelevant retrieved material from diluting the model's context the way naive RAG can. Evaluated across science, math, and coding reasoning tasks plus six open-domain QA benchmarks, with reported gains over baselines.
๐ arXiv:2501.05366
๐บ No specific dedicated talk/interview independently confirmed
Search-O1 only searches when the model itself detects a gap, and filters what comes back before it re-enters the reasoning chain.
Search-R1: Learning the Search Decision via RL
Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning HIGH
Where Search-O1 prompts a model to search when it notices a gap, Search-R1 trains that decision directly with reinforcement learning โ using only an outcome-based reward, with no supervised examples of when-to-search โ so the model learns when and how to issue multiple search queries interleaved with its own reasoning, retrieving in real time rather than following a fixed retrieval pattern. Retrieved-token masking keeps the RL training stable despite the retrieved text not being something the model itself generated. Reported gains over various RAG baselines across seven QA benchmarks: roughly 41% with a 7B model and 20% with a 3B model.
๐ arXiv:2503.09516
๐บ No specific dedicated talk/interview independently confirmed
Search-R1 trains when-to-search directly via RL โ the search decision is learned, not just prompted.
Comparing the Four
| System | Resource being managed | How the decision gets made | Headline result |
|---|---|---|---|
| AlphaCode | Which of ~1M generated programs to submit | Mechanical: test-filter + behavioral clustering | ~54th percentile on Codeforces (average competitor) |
| AlphaCode 2 | Same, plus which candidates actually look best | Mechanical filtering + a learned scoring model | ~85th percentile on Codeforces |
| Search-O1 | When to pause reasoning and search | Prompted: model notices a gap and triggers search | Gains across science/math/coding reasoning + 6 QA benchmarks |
| Search-R1 | Same as Search-O1 | Learned via RL: no supervised search-trace examples | ~41% (7B) / ~20% (3B) gains over RAG baselines across 7 QA benchmarks |
๐ฏ The Bottom Line
Look at the progression within each pair and the same move repeats: AlphaCode โ AlphaCode 2 replaces a mechanical selection rule with a learned one; Search-O1 โ Search-R1 replaces a prompted decision rule with a learned one. In both cases, "learned" beat "mechanical/prompted" by a wide margin โ AlphaCode 2's roughly 30-percentile jump, Search-R1's reported edge over prompted search-triggering baselines. The pattern worth carrying forward: whenever a system has to decide something repeatedly (which candidate to trust, when to search), training that decision usually outperforms hand-writing a rule for it, provided you have a reliable enough reward signal to train against โ which loops back to the verification theme running through every deep dive in this series.
Finishing the deep dives? Deep Dive 5 โ Evaluation and the Open Problems covers METR's time-horizon methodology, GDPval, and DeepScholar-Bench, plus the three problems nobody in this series has solved yet.
โ ๏ธ Confidence Notes and Gaps
AlphaCode's identity and results are high confidence, confirmed against its Science publication and DOI. AlphaCode 2 was never peer-reviewed or given an arXiv ID โ it exists only as a DeepMind technical report; its headline results (43% solve rate, 85th percentile) are corroborated across multiple independent sources, but its full author roster was not independently confirmed. Search-O1 and Search-R1 are both high confidence on identity and core claims; neither has a confirmed dedicated companion video, and this article does not claim one exists for either. None of the four items were verified against a direct arxiv.org fetch in this research environment โ check primary sources before citing specific figures formally.
๐ Full Reference List
Self-Improving AI Agents: A Field Handbook (base article)
Deep Dive 1 โ Agentic Feedback Loops in Practice
Deep Dive 2 โ Planning as Search
Deep Dive 3 โ Training on What Search Finds
AlphaCode โ arXiv:2203.07814
AlphaCode 2 โ official technical report
Search-O1 โ arXiv:2501.05366
Search-R1 โ arXiv:2503.09516