Home โ€บ Blog โ€บ How Self-Improving AI Works
Practical RSI ยท Field Map ยท Snapshot: September 2026 ๐Ÿงฌ

How Self-Improving Agentic Systems Actually Work: Mechanisms, Architecture and the Path to Verified Persistent Improvement

"Self-improving AI" covers everything from a model critiquing its own answer to a system that redesigns how it learns. This field map untangles it. We define what counts, then build the full architecture: an experience-evaluation-routing loop that writes lessons into five different substrates โ€” the model, the harness, the evaluator, the research process and the data โ€” behind a validation gate. We cover the improvement router that decides where each lesson belongs, eight mechanisms with the papers that introduced them, the fast harness loop and slow model loop and why together they form a flywheel (with a runnable model), the verification hierarchy, safe inheritance, an autonomy ladder, and five tests for whether an improvement is real.

How to read this: a synthesis article tying together this section's Recursive Harness Self-Improvement, RSI evidence review and AI Doing R&D. The architecture is a conceptual framework that organises published work โ€” no single system implements all of it. Mechanism examples link to their papers in References; the flywheel simulation is illustrative.

01 What Counts as Self-Improvement?

Three distinctions come first. Self-modification is any change a system makes to itself โ€” editing a prompt, its memory, its weights. Self-improvement is a change that measurably increases capability. Recursive self-improvement is improvement to the mechanism that creates future improvements, so gains compound. Self-modification โ‰  self-improvement: most self-edits make things worse, which is why measurement sits at the heart of everything below. Within self-improvement there are five levels โ€” click one:

โ–ถ Five levels From one better answer to a better way of improving

02 The Core Architecture: a Closed Improvement Loop

Every self-improving system, however simple, runs some version of one loop: experience (tasks, interactions, feedback) โ†’ evaluation (what worked, what failed) โ†’ an improvement router that decides what to change and where โ†’ a change to one of five substrates โ†’ a validation gate โ†’ a persistent update โ†’ the next experience. Press play, and click a substrate to see what can change inside it:

โ–ถ End-to-end improvement loop Five substrates, one gate
Click a substrate.

The five substrates differ enormously in how fast, cheap and safe they are to change. Editing a prompt or a memory entry takes seconds and is trivially reversible; changing a harness (ฮฃ) โ€” tools, workflows, context policy โ€” takes hours; changing an evaluator (V) is dangerous because it moves the goalposts; changing the model (ฮธ) takes days to months of training and is hard to inspect or undo. Much of the art of self-improvement is putting each lesson in the cheapest substrate that can hold it.

03 The Improvement Router: Where Should the Lesson Be Stored?

A failure is a signal; the router decides which substrate should absorb it. Store a one-off fact in weights and you waste training; store a missing reasoning skill in a prompt and it never generalises. Pick a signal and watch where it goes โ€” then try routing it yourself:

โ–ถ Route the lesson Click a signal, then a substrate
Pick a signal on the left.

04 Eight Mechanisms of Self-Improvement

Eight complementary mechanisms drive progress across the substrates, each with a lineage of published work. Click one to see how it works, its substrate, a landmark paper and its main risk:

โ–ถ Mechanisms Animated mini-diagram per mechanism

05 Two Speeds and the Flywheel

Practical systems run two loops. The fast loop โ€” harness evolution โ€” changes prompts, retrieval, memory, tools and workflows in hours to days. The slow loop โ€” model consolidation โ€” distils accumulated verified trajectories into the model via fine-tuning, RL or self-training over weeks to months. The fast loop explores; the slow loop makes the best discoveries durable. Toggle the loops and watch success rates over a year:

โ–ถ Fast loop ร— slow loop Same model as the script below
"""Two-speed self-improvement: fast harness loop + slow model loop.

State: model skill m and harness quality h, both in [0, 1]. Task success s = m + (1 - m) * 0.6 * h
(a harness can recover part, not all, of what the model lacks).
Fast loop (weekly):   h moves 15% of the way toward its ceiling, and the ceiling rises with the model
                      (h_max = 0.5 + 0.5 m): a stronger model can use richer tools and workflows.
Slow loop (every 8 weeks): verified trajectories train the model, m += 0.2 * s * (1 - m) - better
                      success means more and better training data. After a model update 20% of the
                      harness tuning no longer fits and must be redone (h *= 0.8).
Illustrative dynamics, not fitted to data. Standard library only.
"""
def run(fast, slow, weeks=52):
    m, h = 0.30, 0.20
    out = []
    for w in range(weeks + 1):
        s = m + (1 - m) * 0.6 * h
        if w % 13 == 0: out.append(s)
        if fast: h += 0.15 * ((0.5 + 0.5 * m) - h)
        if slow and w > 0 and w % 8 == 0:
            m += 0.2 * s * (1 - m); h *= 0.8
    return out, m

print("Task success rate by week")
print(f"   {'loop':<26}" + "".join(f"{'wk ' + str(w):>8}" for w in (0, 13, 26, 39, 52)) + f"{'final m':>9}")
for name, f, s in [("static system", 0, 0), ("fast loop only (harness)", 1, 0),
                   ("slow loop only (model)", 0, 1), ("both (flywheel)", 1, 1)]:
    out, m = run(f, s)
    print(f"   {name:<26}" + "".join(f"{v:>8.1%}" for v in out) + f"{m:>9.2f}")
print("\nThe fast loop gains quickly then plateaus at what the model can use; the slow loop compounds")
print("slowly. Together each raises the other's ceiling - the flywheel.")

Output:

Task success rate by week
   loop                          wk 0   wk 13   wk 26   wk 39   wk 52  final m
   static system                38.4%   38.4%   38.4%   38.4%   38.4%     0.30
   fast loop only (harness)     38.4%   55.0%   57.0%   57.3%   57.3%     0.30
   slow loop only (model)       38.4%   41.6%   49.4%   53.8%   62.9%     0.62
   both (flywheel)              38.4%   58.2%   68.4%   76.4%   83.2%     0.71

The fast loop gains quickly then plateaus at what the model can use; the slow loop compounds
slowly. Together each raises the other's ceiling - the flywheel.

The harness-only loop jumps fast (38% โ†’ 55% in a quarter) and then stalls: it can only recover what the current model is able to use. The model-only loop compounds but slowly. Together, each raises the other's ceiling โ€” better harness โ†’ better trajectories โ†’ better training data โ†’ better model โ†’ room for a more ambitious harness โ€” reaching 83%. This modelโ€“harness co-evolution is what HELIX made explicit, and it is widely seen as the most practical precursor to recursive self-improvement. (Real dynamics are unknown; HELIX reports only a single co-evolution round.)

06 The Verification Hierarchy

Every loop above is only as good as its evaluator. Guarantees get stronger as feedback becomes more external, formal and objective โ€” and trust in self-improvement weakens as verification becomes less so. Hover a level:

โ–ถ From proof to self-judgment Stronger at the top
Hover a level.

This is why self-improvement arrived first in code and mathematics, and why the riskiest design choice is letting the system improve its own evaluator. Some research deliberately co-evolves agent and evaluator (the 2026 Red Queen Gรถdel Machine) โ€” but in production the rule is the opposite: the judge must be something the improver cannot edit.

07 Safe Inheritance and Validation

Self-modification must be versioned and verified to avoid persistent mistakes: treat self-improving agents like continuously deployed software. Each candidate passes a sandbox, regression tests, out-of-distribution tests, safety checks and an independent verifier, then a canary rollout, and is promoted or rolled back. Play several generations:

โ–ถ Generations of a self-improving agent Each candidate must pass every check

08 Autonomy and Maturity Ladder

Six levels describe how much of the improvement process the system runs itself. Drag the slider:

โ–ถ L0 โ†’ L5 Who decides what to improve?

At the top, a useful distinction: structural L5 means the improvement mechanism itself is modified and inherited (new learners, evaluators, search strategies or objectives). Effective L5 means those inherited mechanisms reliably create better successors under matched resources and independent evaluation. Several research systems reach structural L5 in narrow settings; none has shown effective L5.

09 What Makes Improvement Real?

A change counts as real progress only if it meets five criteria. Pick a claim to test it:

โ–ถ Five tests Illustrative assessments

10 What Is Real Today vs the Frontier

โœ… Established today

  • Self-refinement and reflection (self-critique, chain-of-thought)
  • Memory and skill accumulation (external memory, RAG, skill libraries)
  • Harness optimisation (prompts, tools, workflows)
  • Bounded automated experimentation (prompt search, tool use)
  • AI-assisted AI R&D (idea generation, code, reviews)

๐ŸŸก Emerging frontier

  • Modelโ€“harness co-evolution (HELIX)
  • Automated research organisations (multi-agent research systems)
  • Evaluator improvement (better tests, reward models, verifiers)
  • Large-scale parallel research agents (thousands exploring at once)

๐Ÿ”ด Unproven / open

  • Robust multi-generation effective RSI
  • Autonomous agenda selection (choosing long-term goals)
  • Autonomous frame revision (changing objectives or paradigms)
  • Open-ended improvement under independent verification

Current signals, 2025โ€“2026, covered in depth in this section: OpenAI's automated research intern; Claude leading 26% of Anthropic's measured AI R&D tasks and collaborating on more than 90%; more than 80% of Anthropic's merged code AI-authored; AlphaEvolve's algorithm discovery loop; Co-Scientist, ERA, Robin and The AI Scientist as research systems; and DGM, HGM and HELIX as self-improving agents and harnesses. The practical research focus now runs left to right: fast harness adaptation (active today) โ†’ verified persistent learning โ†’ modelโ€“harness co-evolution โ†’ early meta-improvement (the frontier).

The field in one paragraph
Self-improving agentic systems are best understood as a closed loop โ€” experience, evaluation, routing, change, validation, persistence โ€” writing lessons into five substrates of very different cost and risk: the harness, memory and skills cheaply and reversibly; the evaluator and research process carefully; the model slowly and durably. The fast harness loop and slow model loop reinforce each other in a flywheel, and that co-evolution is the most practical path toward recursive self-improvement. What holds it all together is verification: improvements are real only if they persist, generalise, can be attributed, are independently verified and are efficient โ€” and trust falls as the verifier moves from proofs and tests toward the system's own judgment. Everything up to harness optimisation and AI-assisted R&D is established; co-evolution and automated research organisations are emerging; open-ended, verified, multi-generation RSI remains unproven.

Next: From Automated R&D to Effective RSI: Frontier, Evidence and Open Problems โ†’

โš ๏ธ Notes

About the source infographics (two parts, merged here into one article): (1) the architecture, levels, router and ladders are a conceptual framework, not a standard taxonomy โ€” "structural vs effective L5" and the five substrates (ฮธ, ฮฃ, V, M, E) are the infographic's own terms, useful but not established vocabulary; (2) the flywheel is a hypothesis โ€” published evidence is one co-evolution round (HELIX), not the repeated compounding the diagram implies; (3) the signals panel's figures (OpenAI intern; Anthropic >80% merged code and 26% "AI leads" / >90% "AI collaborates") check out against primary sources and are self-reported; (4) we add the named systems behind each mechanism โ€” Reflexion, ExpeL, Voyager, DSPy and Promptbreeder, SWE-agent, ADAS, the Darwin and Huxleyโ€“Gรถdel Machines, and STaR โ€” and a caution the infographic omits: some research (the Red Queen Gรถdel Machine) deliberately co-evolves the evaluator, which production systems should not allow. No numerical errors were found.

๐Ÿ”— References