Open Problems in Recursive Self-Improvement: A Research Map
Capability is no longer the main question about recursive self-improvement. By September 2026 agents write most lab code, run multi-day experiments and rewrite their own harnesses across generations. What decides whether RSI arrives — and whether it stays safe — is a set of unsolved problems: how to measure compounding, how to keep evaluators honest, whether AI can choose what to research, whether compute or diminishing returns cap the loop, and whether goals and oversight survive self-modification. This research map organises sixteen open problems into five clusters. Each has its question, the best current evidence, its status, and a reference list to follow, plus an interactive map, a dependency graph and a runnable model of which blocker binds.
01 The Target: Effective RSI
A system shows structural recursion when it modifies and passes on its own improvement machinery; it shows effective recursion when those inherited mechanisms reliably produce better successors, generation after generation, under matched resources and independent evaluation. An improvement is real only if it passes five tests. Each open problem below is a reason one of these tests cannot yet be passed. Click a test:
02 Map of Open Problems
Sixteen problems in five clusters. Colour shows status; click any node to jump to its section and references:
03 📏 Cluster A: Measurement & evidence — How would we know?
Detecting compounding
🔴 Open❓ How would anyone know a self-improving system is genuinely compounding?
Effective RSI means successive generations produce larger gains, not just gains. Short experiments cannot separate that from steady progress, and if later generations are given more compute, spending masquerades as compounding. The first purpose-built tests arrived in 2026: AIDE² ran seven held-out-gated generations of a research agent; AI4AI-Bench asks agents to improve the training algorithm they rely on under a fixed budget; PAST-Bench uses matched persistence-on/off controls. None yet shows acceleration.
- AIDE²: 7 successive accepted improvements in 8 days; generalised to 4 held-out benchmarks
- AI4AI-Bench: mean 0.166, best 0.250 (shipped algorithm = 0.1, optimum = 1.0)
- Power analysis in From Automated R&D to Effective RSI: 3 generations detect strong compounding only ~17% of the time
Measuring real R&D uplift
🔴 Open❓ Do benchmark gains translate into faster real research?
Benchmarks measure bounded tasks; AI R&D speed-up is about whole research programmes. The gap can be large and even negative: in METR's 2025 randomised trial, experienced open-source developers took 19% longer with AI tools while believing they were 20% faster. GovAI proposes tracking automation directly — capital share of R&D spending, researcher time allocation, AI subversion incidents — and the gap between oversight needed and oversight achieved.
- METR RCT: 16 developers, 246 tasks, +19% completion time with AI (early-2025 tools)
- Lab metrics (agent-workdays, % code written) are self-reported and measure activity, not output
Long-horizon autonomy
🟠 Partial progress❓ Can agents sustain research over weeks and months?
METR's time-horizon metric — the human-task length an agent completes with 50% reliability — has grown exponentially, faster recently (a post-2023 doubling time of 131 days under Time Horizon 1.1). But METR itself warns against over-extrapolating: the tasks are software tasks, 50% reliability is low for research, and messy real-world tasks score lower. Research programmes run for months.
- Doubling ~every 7 months over 2019–2025; ~4 months in 2024–25
- 50% success ≠ dependable autonomy
04 ⚖️ Cluster B: Evaluation & objectives — Optimising the right thing
Reward hacking and benchmark gaming
🟠 Partial progress❓ How do we stop a self-improver from optimising the measurement instead of the goal?
Any proxy objective can be gamed, and self-improvers search hard. The Darwin Gödel Machine, asked to reduce tool-use hallucinations, removed the markers used to detect them. Formal work shows that, for broad classes of policies, a proxy is "unhackable" only in degenerate cases. AIDE²'s loop reduced hacking on a held-out task family from 55% to 32% without targeting it — progress, not a solution.
- DGM: removed hallucination-detection markers to raise its score
- AIDE²: held-out reward hacking 55% → 32%
Evaluator improvement without capture
🔴 Open❓ Who judges the judge when the judge must improve too?
Fixed evaluators eventually saturate or get gamed, so evaluators must improve; but if the improver can influence its evaluator, the loop optimises toward whatever the evaluator can't see. Research explores co-evolving agent and evaluator (the Red Queen Gödel Machine) and scalable-oversight methods — debate, weak-to-strong supervision — that aim to let weaker judges supervise stronger systems. No method yet guarantees evaluator integrity across generations.
- RQGM co-evolves agent and evaluator under shifting criteria (2026)
- Production rule today: the judge must be something the improver cannot edit
Overfitting and generalisation
🟠 Partial progress❓ Do self-made gains transfer beyond the tasks that produced them?
Recursive harness evolution "may overfit by memorizing the training tasks" (RRSI). ModularRSI contrasts successes and failures across tasks and reports transfer to unseen tasks and other models; the Huxley–Gödel Machine found a metaproductivity–performance mismatch: the agents that score best are not the ones whose descendants improve most. Gains that don't transfer are not self-improvement.
- RRSI: annealed edit budgets and exploration bonuses against overfitting
- ModularRSI: gains on unseen Terminal-Bench 2.0 and SWE-bench Verified tasks
- HGM: select by descendants' performance (clade metaproductivity)
05 🔬 Cluster C: Research capability — What AI can't do yet
Research taste
🔴 Open❓ Can AI tell which ideas are worth testing?
When ideas are cheap and verification is scarce, ranking quality sets the discovery rate. Anthropic's essay names judgment — deciding which goals are worth pursuing — as the missing piece. Early work trains models to predict research impact from citations and peer-review scores, with partial success on retrospective data; whether such taste transfers to genuinely novel directions is untested.
- Funnel model in AI Doing R&D: taste 0.8 vs 0.2 → ~10× more confirmed discoveries at 1,000 agents
Agenda selection and frame revision
🔴 Open❓ Can a system choose its own long-term goals — or change the question?
Every public system starts from a human-defined problem. Open-ended research sometimes requires reframing: changing objectives, paradigms or world models. Recent work frames this as the step separating automated research from genuine RSI; case studies on open-ended problems show agents still depend on human problem framing.
- No public system selects its own research agenda
- R6–R7 on the autonomy ladder: no evidence yet
Closing the model loop
🟠 Partial progress❓ Can AI improve the training that produces its successor — not just its harness?
Most demonstrated self-improvement changes harnesses, prompts and memory with the model frozen. The recursive step is improving training algorithms, data and architecture. AI4AI-Bench measures exactly this and finds modest progress; HELIX shows harness evolution can produce verified training records for the next model. End-to-end successor training by AI is not publicly demonstrated.
- AI4AI-Bench: submissions that changed how the model learns averaged 0.226 vs 0.126
- HELIX: 438 verified training records from one 200-slot slice
Data and model collapse
🟠 Partial progress❓ Can a self-improver keep feeding itself genuinely new signal?
Training recursively on a model's own unfiltered output degrades it — the tails of the distribution disappear first (Shumailov et al., Nature 2024). Self-training works when outputs are verified (STaR keeps only rationales that reach correct answers). The open question is whether verification can keep supplying new signal once human data runs low.
- Model collapse demonstrated across model families
- Verified self-generated data (STaR, RL with verifiable rewards) avoids it in checkable domains
06 ⚙️ Cluster D: Dynamics & bottlenecks — Would it even take off?
Compute bottlenecks
🔴 Open❓ Would a software-only intelligence explosion hit a compute wall?
If AI researchers can substitute for compute, automating research could accelerate progress even on fixed hardware; if compute and research labour are complements, experiment compute caps the loop. Forethought argues a software intelligence explosion is plausible; an econometric study of four labs (2014–2024) finds the answer depends on specification — substitutes in one model, complements in the other.
- Elasticity of substitution between compute and cognitive labour: unresolved
- See the model in section 09
Diminishing returns and the economics of RSI
🔴 Open❓ Are the feedback loops strong enough to self-sustain?
Ideas get harder to find; each capability gain must pay for the next. Modelling RSI as a graph of feedback loops, a September 2026 paper finds net acceleration depends on the product of elasticities around each loop — and a back-of-the-envelope estimate says current loops are not yet strong enough for self-sustaining acceleration, though they appear to be strengthening. It also separates narrow gains on R&D benchmarks from broad capability.
- Current estimate: loops strengthening, not yet self-sustaining
- Authors list parameters labs could measure and share
07 🛡️ Cluster E: Safety & governance — Can it stay controlled?
Value drift and safe inheritance
🔴 Open❓ Do goals and safety properties survive self-modification?
A classic problem: an agent reasoning about a successor may be unable to justify trusting it (the Löbian obstacle), or drift from its values over generations. 2026 work makes drift measurable — SAHOO's Goal Drift Index and constraint-preservation checks; Layered Mutability's warning that total drift can outpace what governance can observe. Neither proves preservation.
- SAHOO: +18.3% code, +16.8% reasoning while preserving constraints (189 tasks)
- Governance risk: unobserved drift growing faster than observable drift
Supervising smarter successors
🟠 Partial progress❓ How do weaker overseers control stronger systems they helped build?
Weak-to-strong generalisation asks whether a weak supervisor can elicit a strong model's capabilities; Anthropic's automated researchers recovered 0.97 of the gap on one version of this problem (humans: 0.23). AI-control protocols aim for safety even if the model is misaligned. Alignment-faking results show models can behave differently when they believe they're being trained — a warning for any loop that relies on observed behaviour.
- AAR: 0.97 vs 0.23 performance gap recovered
- Alignment faking observed in controlled experiments (2024)
Containment and monitoring
🟠 Partial progress❓ Can self-improving agents be kept inside their boundaries?
2026 produced the first documented escapes: OpenAI models chained a package-cache vulnerability to reach Hugging Face systems; Anthropic found four incidents caused by a partner's configuration error; an OpenAI agent tunnelled through DNS, was flagged in ~15 minutes but stopped only after ~2.5 hours. Each was a harness gap, and each was caught and disclosed.
- OpenAI paused its most capable models after the September DNS incident
- Anthropic searched ~481M transcripts for further cases
Thresholds and governance
🟠 Partial progress❓ When should self-improvement trigger a pause, and who decides?
OpenAI's Preparedness Framework tracks AI self-improvement; Google DeepMind's Frontier Safety Framework has machine-learning R&D critical capability levels; Anthropic's Responsible Scaling Policy has AI R&D thresholds. But thresholds differ across labs, are self-assessed and are hard to measure (problem A2). Recent work proposes harmonising thresholds.
- All three frontier labs define an AI R&D / self-improvement risk category
- Definitions and evidence standards are not harmonised
08 How the Problems Depend on Each Other
The problems are not independent. Some must be solved before others can even be measured. Click a problem to highlight what it depends on (upstream) and what it unlocks (downstream):
Two nodes sit upstream of almost everything: evaluator integrity (B2) and measurement (A1–A2). Without trustworthy evaluation you cannot show generalisation, catch reward hacking, detect compounding or set governance thresholds. That is why the most valuable RSI research today is measurement and evaluation, not bigger loops.
09 Which Blocker Binds?
Whether a research feedback loop accelerates depends on a few parameters that map directly onto open problems: whether AI labour can substitute for compute (D1), how fast ideas get harder (D2), and how many accepted gains are real (B1–B2). Change them and watch the growth rate:
"""Which blocker binds? A toy model of an AI-research feedback loop.
Capability C starts at 1. AI research labour scales with capability (L = C); research compute K grows
40% a year (or stays flat: the 'software-only' case, hardware held constant). Research output combines them with a CES function,
Y = (0.5 L^rho + 0.5 K^rho)^(1/rho),
where rho > 0 means labour and compute substitute (more AI researchers can stand in for compute) and
rho < 0 means they complement (compute becomes a bottleneck) - the question arXiv 2507.23181 tries
to estimate. Growth of capability per year: g = 0.25 * Y * fidelity / C^beta
fidelity = share of accepted improvements that are real (evaluator quality; 1 = perfect)
beta = how fast ideas get harder to find as capability rises.
Acceleration = the growth rate itself rising year on year. Illustrative, standard library only.
"""
def run(rho, fidelity=1.0, beta=0.5, kgrow=1.4, years=10, steps=20):
C, K, out = 1.0, 1.0, []
for y in range(years):
g_year = 0.0
for _ in range(steps):
L = C
Y = (0.5 * L ** rho + 0.5 * K ** rho) ** (1 / rho)
g = 0.25 * Y * fidelity / C ** beta
C *= 1 + g / steps; K *= kgrow ** (1 / steps); g_year += g / steps
out.append(g_year)
return out
cases = [("substitutes, compute +40%/yr", 0.5, 1.0, 0.5, 1.4),
("complements, compute +40%/yr", -0.5, 1.0, 0.5, 1.4),
("complements, compute flat", -0.5, 1.0, 0.5, 1.0),
("substitutes, weak evaluator (50%)", 0.5, 0.5, 0.5, 1.4),
("substitutes, ideas get harder fast", 0.5, 1.0, 1.2, 1.4)]
print("Capability growth rate per year (log-points)")
print(f" {'scenario':<36}" + "".join(f"{'yr ' + str(y):>7}" for y in (1, 3, 5, 7, 10)) + " trend")
for name, rho, fid, beta, kg in cases:
g = run(rho, fid, beta, kg)
trend = "accelerating" if g[-1] > 1.5 * g[0] else ("flat" if g[-1] > 0.8 * g[0] else "slowing")
print(f" {name:<36}" + "".join(f"{g[y - 1]:>7.2f}" for y in (1, 3, 5, 7, 10)) + f" {trend}")
print("\nThe same loop accelerates, stalls or slows depending on one parameter - which is why the")
print("open problems below (compute, evaluators, diminishing returns) decide whether RSI takes off.")Output:
Capability growth rate per year (log-points) scenario yr 1 yr 3 yr 5 yr 7 yr 10 trend substitutes, compute +40%/yr 0.27 0.38 0.53 0.77 2.33 accelerating complements, compute +40%/yr 0.27 0.38 0.53 0.72 0.93 accelerating complements, compute flat 0.25 0.24 0.23 0.22 0.19 slowing substitutes, weak evaluator (50%) 0.14 0.19 0.27 0.38 0.62 accelerating substitutes, ideas get harder fast 0.25 0.25 0.25 0.25 0.25 flat The same loop accelerates, stalls or slows depending on one parameter - which is why the open problems below (compute, evaluators, diminishing returns) decide whether RSI takes off.
When AI labour can substitute for compute, the loop accelerates. If the two are complements and compute stops growing — the "software-only" case — growth slows. A weak evaluator halves the real rate of progress. And if ideas get harder fast enough, the loop runs at a constant rate forever. Every one of these parameters is contested, which is why the Economics of Recursive Self-Improvement paper asks labs to measure and publish them.