From Automated R&D to Effective RSI: Frontier, Evidence and Open Problems
AI already codes experiments, searches literature, generates hypotheses, runs benchmarks, evolves algorithms and distils models. The question is no longer whether AI does R&D, but whether an improvement mechanism can produce successors that are reliably better — generation after generation, under matched resources and independent evaluation. This final part of the self-improvement arc sorts what is already real from what is unproven, explains the difference between structural and effective recursion, reviews the first experiments designed to test it (AIDE², AI4AI-Bench, PAST-Bench), shows with a runnable power analysis why a few generations can't settle the question, and maps the twelve open problems that still block genuine recursive self-improvement.
01 Current Reality: AI Already Doing R&D
Eight core research capabilities are established or partially established in bounded settings. Click a tile for the strongest public example and how mature it is:
These work today in focused domains and controlled environments. None amounts to full recursive self-improvement: each is a component that a recursive system would need, run under human direction.
02 The Autonomy Ladder and Its Evidence
The R0–R7 ladder (introduced in AI Doing R&D) is useful only with an honest evidence column. Public evidence is strong at R0–R3, limited demonstrations exist at R4–R5, and R6–R7 have no evidence yet. Hover a rung:
03 The Core Frontier Loop
The frontier setup is an end-to-end AI R&D cycle — goal, literature, hypotheses, experiment design, code and training runs, results, analysis and verification, research memory, replanning — run by many agent teams in parallel, sharing what they learn through a common memory. Watch the teams explore; lines show ideas cross-pollinating via memory:
Parallelism buys exploration, specialisation and speed; shared memory turns one team's failure into every team's lesson. This is the architecture of OpenAI's ~10,000-agent maths run and Anthropic's automated alignment researchers (with a shared forum). What it does not buy by itself is direction — someone still sets the goal at the top.
04 Where the Evidence Is Strongest
AI R&D progress is most reliable when grounded in external, formal signals. The same five-level ranking from the verification hierarchy, now with the R&D result each level has supported:
| Grounding | Reliability | Example R&D result it supports |
|---|---|---|
| 1 · Formal verification / proofs | Highest | Lean-checked proofs (OpenAI's Navier–Stokes claim — though what was proved is still debated) |
| 2 · Executable experiments / tests / benchmarks | High | AlphaEvolve's faster kernels; ERA's benchmark-beating methods; AIDE²'s held-out gains |
| 3 · Simulators / controlled environments | Moderate–high | Sandboxed training runs; AI4AI-Bench's fixed re-run protocol |
| 4 · Learned judges / reward models | Moderate | Automated paper reviewing (The AI Scientist-v2's self-review) |
| 5 · Self-judgment | Lowest, easy to game | An agent reporting its own improvement — not evidence |
05 Structural vs Effective Recursion
The key distinction. Structural recursion: the improvement mechanism itself is modified and inherited — Improver 1 builds System 2 and an improved Improver 2, which builds System 3. Many systems already do this. Effective recursion: those inherited mechanisms reliably produce better successors — consistent, measurable gains across multiple generations, evaluated independently, under matched resources, not just overfitting. Toggle and play:
✔ Structural recursion
- Improves its own algorithms, prompts or architecture
- Changes persist to successors
- Mechanism-level improvement
✔ Effective recursion adds
- Consistent, measurable gains
- Works across multiple generations
- Evaluated independently
- Not just overfitting or benchmark gaming
- Under matched compute, data and time
06 The First Tests of Effective RSI
In August–September 2026 three preprints tried to measure the recursive link directly rather than assert it. Each tests a different piece:
Together they show both how far things have come and how far they have to go. AIDE² is the strongest public multi-generation result — seven accepted successive improvements, each gated on held-out data, generalising to external benchmarks — yet its gains are to the research agent, not the model, and it doesn't show that later improvements come faster or larger. AI4AI-Bench shows agents can improve training algorithms, but only modestly (best 0.25 on a scale where the shipped algorithm is 0.1 and the optimum 1.0). PAST-Bench shows persistence helps every model — unevenly, and not always via the intended pathway.
07 How Would We Know?
Suppose a lab runs its self-improving system for a few generations and sees gains. Is that compounding? Statistically, telling "gains accelerate" from "gains are constant" needs more generations than most experiments run — and if later generations also get more compute, fake acceleration appears easily. Move the sliders:
"""How would we know? Detecting *effective* recursive self-improvement.
Effective RSI means each generation's improvement mechanism produces a successor whose gain is
LARGER than the last one's: gains accelerate. We simulate experiments: G generations, S independent
seeded runs per generation. Gain of generation k = 1.0 + accel * k + noise (sd 1.0), in benchmark
points. We regress gain on k and call acceleration "detected" if the slope is positive at the
one-sided 5% level (t-test). Also: if each generation also gets more compute than the last and
gains are not resource-normalised, extra compute adds 0.3 * k points and masquerades as acceleration.
2,000 simulated experiments per cell. Standard library only.
"""
import random, math
T95 = [6.314, 2.920, 2.353, 2.132, 2.015, 1.943, 1.895, 1.860, 1.833, 1.812, 1.796, 1.782, 1.771, 1.761,
1.753, 1.746, 1.740, 1.734, 1.729, 1.725, 1.721, 1.717, 1.714, 1.711, 1.708, 1.706, 1.703, 1.701,
1.699, 1.697] # one-sided 95% t critical values, df 1..30
def detect(G, S, accel, compute_drift, rng):
xs, ys = [], []
for k in range(G):
for _ in range(S):
xs.append(k); ys.append(1.0 + (accel + compute_drift) * k + rng.gauss(0, 1.0))
n = len(xs); mx = sum(xs) / n; my = sum(ys) / n
sxx = sum((x - mx) ** 2 for x in xs)
b = sum((x - mx) * (y - my) for x, y in zip(xs, ys)) / sxx
resid = sum((y - my - b * (x - mx)) ** 2 for x, y in zip(xs, ys))
se = math.sqrt(resid / (n - 2) / sxx)
df = n - 2
return b / se > (T95[df - 1] if df <= 30 else 1.645)
def power(G, S, accel, drift=0.0, trials=2000):
rng = random.Random(G * 100 + S * 10 + int(accel * 10) + int(drift * 7))
return sum(detect(G, S, accel, drift, rng) for _ in range(trials)) / trials
print("Chance of detecting accelerating gains (3 seeded runs per generation)\n")
print(f" {'generations':>11} {'no accel':>10} {'weak 0.1':>10} {'strong 0.3':>11}")
for G in [2, 3, 5, 7, 10]:
print(f" {G:>11} {power(G, 3, 0.0):>10.0%} {power(G, 3, 0.1):>10.0%} {power(G, 3, 0.3):>11.0%}")
print("\n'no accel' is the false-positive rate: it should stay near 5%.")
print(f"\nNo real acceleration, but compute grows each generation (not normalised), 7 generations:"
f" {power(7, 3, 0.0, 0.3):.0%} 'detect' acceleration")
print("\nFew generations cannot tell compounding from constant gains; unmatched compute fakes it.")Output:
Chance of detecting accelerating gains (3 seeded runs per generation)
generations no accel weak 0.1 strong 0.3
2 6% 6% 9%
3 5% 7% 17%
5 5% 12% 48%
7 5% 22% 84%
10 5% 44% 100%
'no accel' is the false-positive rate: it should stay near 5%.
No real acceleration, but compute grows each generation (not normalised), 7 generations: 85% 'detect' acceleration
Few generations cannot tell compounding from constant gains; unmatched compute fakes it.With three generations, even strong compounding is detected only 17% of the time; with seven it's 84%; weak compounding needs well over ten. And a design that lets each generation use more compute "detects" acceleration 85% of the time when there is none. This is why the definition of effective RSI insists on multiple generations, matched resources and independent evaluation: without them, the claim can't be distinguished from noise or from spending more.
08 Twelve Open Problems Blocking Genuine RSI
Technical, evaluation and governance challenges remain. Filter by type and click a card for why it matters and who is working on it:
09 What Researchers Should Focus on Now
A practical near-term roadmap toward safe, effective RSI — each step makes the next measurable. Press play:
10 Bottom Line
✅ Already real (demonstrated today)
- AI-assisted R&D: search, coding, analysis
- Automated experiments and benchmarking
- Harness and workflow auto-improvement
- Parallel agent research
- Teacher–student distillation and synthetic data
🟠 Actively emerging (rapid progress)
- Autonomous research loops in bounded domains
- Model–harness co-evolution
- Evaluator-aware improvement
- Long-horizon research agents and research organisations
- Multi-generation self-improvement of research agents (AIDE²)
❓ Still unproven (open frontier)
- Robust effective recursive systems (R7)
- Autonomous agenda-setting
- Open-ended recursive improvement
- Sustained multi-generation capability gains that compound
- Safe and controllable RSI in the real world
Next: Open Problems in Recursive Self-Improvement: A Research Map →