A Research Org for Every RSI Level
Phase 6 opens the research engine. Gopala needs to explore every level of self-improvement, because the next moat could be anywhere and competitors are exploring them too, without letting the riskiest level anywhere near customers. This part is built to be explored, not read: a knowledge graph of the whole research org with a guided tour, a sandbox where you staff it yourself, and two scripts that set the rules.
rsi_portfolio.py and boundary_check.py were run (real output below). Paper counts per level come from the site's RSI Evolution Map catalog (3 Nov 2026). Values, probabilities, risks, people and experiments are the story's scenario. New format: short text, more to explore; open the ๐ drawers for depth. Disclosure: Claude, made by Anthropic, helps write this site.
1. Mission log: months 26โ30
- Which levels of self-improvement should a 40-person company research, and which must never reach the product?
- Five tracks, one per level; allocation set by
rsi_portfolio.py; boundaries enforced byboundary_check.py.
- The L5 track asks for more compute (section 7).
2. The map: one research org, five levels
๐ธ๏ธ The research org as a knowledge graph
Levels (top row) are explored by tracks (middle); guards in green, resources in grey. Paper counts come from the site's RSI catalog on 3 Nov 2026; people and compute come from rsi_portfolio.py below.
- Three things to notice.
- Four tracks reach the product, all through the gate.
- The L5 track has a red "never" edge to the product: its only home is the sandbox.
- Half the people sit on L4, the moat.
3. The ladder in one picture
Five levels of self-improvement and how much research exists at each
Counts from the CC0 Awesome-RSI catalog used by the site's RSI map (524 papers); the catalog undercounts recent months.
- Higher levels: fewer papers, bigger prizes, bigger risks.
- That is the shape of the research bet: most known results are low on the ladder, most of the unknown upside is high.
4. Five tracks, one per level
"""rsi_portfolio.py โ split 10 researchers and the research compute across
five tracks, one per RSI level (L1-L5), under fixed safety rules.
Every number is the story's SCENARIO estimate, not a measurement:
value = expected moat value if the track succeeds (0-10, team judgement)
p = chance of a usable result within 12 months
risk = how much harm a failure could do outside a sandbox (0-10)
Rules (from the boundary policy in this part):
* L5 runs only in an isolated sandbox and gets at most 10% of compute;
* every track at L3 or above has a safety reviewer attached;
* a track needs at least one researcher to exist.
The allocator tries every split of researchers and keeps the one with the
best risk-adjusted value: sum(value * p * people^0.6) - 0.15 * sum(risk * share).
"""
from itertools import product
TRACKS = [ # level, name, value, p, risk
(1, "L1 eval & training automation", 3, 0.9, 1),
(2, "L2 harness and strategy search", 6, 0.6, 2),
(3, "L3 curriculum from failures", 7, 0.5, 4),
(4, "L4 skill library (the product)", 9, 0.7, 5),
(5, "L5 meta-improvement (sandbox)", 8, 0.15, 9),
]
PEOPLE, L5_COMPUTE_CAP = 10, 0.10
def score(split):
total = sum(split)
if total != PEOPLE or min(split) < 1:
return None
share = [s / total for s in split]
if share[4] > L5_COMPUTE_CAP + 1e-9: # compute follows people in this model
return None
gain = sum(v * p * s ** 0.6 for (_, _, v, p, _), s in zip(TRACKS, split))
harm = 0.15 * sum(r * sh * 10 for (_, _, _, _, r), sh in zip(TRACKS, share))
return gain - harm
def main():
best = max((s for s in product(range(1, 7), repeat=5) if score(s) is not None), key=score)
print(f"{'track':34} {'people':>6} {'compute':>8} reviewer")
for (lvl, name, *_), n in zip(TRACKS, best):
print(f"{name:34} {n:6} {n / PEOPLE:8.0%} {'safety' if lvl >= 3 else '-'}")
print(f"risk-adjusted value {score(best):.2f}")
even = (2, 2, 2, 2, 2)
print(f"even split (2 each) is not allowed: L5 would get {2 / PEOPLE:.0%} of compute" if score(even) is None else f"even split value {score(even):.2f}")
heavy = (1, 1, 2, 5, 1)
print(f"all-in on the product (1,1,2,5,1) value {score(heavy):.2f}")
if __name__ == "__main__":
main()Where the 10 researchers sit (allocator's best)
Real output of rsi_portfolio.py.
๐ Go deeper: why not split evenly, or go all-in on the product?
- An even split breaks a rule.
- Two people on L5 would mean 20% of compute, twice the cap.
- All-in on the product is close but worse.
- 1-1-2-5-1 scores 22.60 against 22.95: one person moved from L3 to L2 buys more expected value at lower risk.
- The model is deliberately simple.
- Value per person grows with diminishing returns (people to the power 0.6); risk is weighted by compute share. Change the inputs and the answer changes, which is the point of writing them down.
5. Sandbox: you run the research org
๐ฎ Sandbox: staff the research org (same model as rsi_portfolio.py)
Each bar: expected value the track adds (+) and the risk it carries (โ). Scenario estimates, not measurements.
- Try this.
- Put 2 on L5: blocked by the compute cap.
- Put 6 on L4 and 0 on L1: blocked, every track needs a person.
- Find a split that beats 22.95 (the allocator says you cannot).
6. Safety boundaries rise with the level
What each level needs before an experiment may run
The story's policy, modelled on the Season 1 gate.
"""boundary_check.py โ check proposed experiments against the safety
boundary for their RSI level before they are allowed to run.
Policy (the story's; modelled on the gate in Season 1, Part 11):
L1-L2 normal cluster; results reach the product only through the gate
L3 + a safety reviewer signs the task-generation rules
L4 + changes to product skills need two gate approvals
L5 + isolated sandbox, no network, no write access to its own evaluator,
a kill switch tested this week, and never touches the product
Experiments are SCENARIO proposals from the research tracks.
"""
RULES = {
1: [],
2: [],
3: ["reviewer"],
4: ["reviewer", "two_approvals"],
5: ["reviewer", "sandbox", "no_network", "evaluator_readonly", "kill_switch", "no_product"],
}
PROPOSALS = [
("E-101", 1, "auto-tune eval sampling", set()),
("E-102", 2, "search over agent harness prompts", set()),
("E-103", 3, "generate tasks from customer failures", {"reviewer"}),
("E-104", 4, "promote 40 new skills to product", {"reviewer"}),
("E-105", 5, "let the loop edit its own reward checker", {"reviewer", "sandbox", "no_network", "kill_switch", "no_product"}),
("E-106", 5, "agent proposes new research directions", {"reviewer", "sandbox", "no_network", "evaluator_readonly", "kill_switch", "no_product"}),
]
def main():
ok = 0
for eid, lvl, what, has in PROPOSALS:
missing = [r for r in RULES[lvl] if r not in has]
ok += not missing
print(f"{eid} L{lvl} {what:42} {'RUN' if not missing else 'BLOCKED: missing ' + ', '.join(missing)}")
print(f"{ok} of {len(PROPOSALS)} may run")
if __name__ == "__main__":
main()๐ Go deeper: why the read-only evaluator matters most
- E-105 wanted the loop to edit its own reward checker.
- That is the classic way a self-improving system learns to fool its own test (Season 1, Part 11: a weak verifier let the toy model cheat 98% of the time).
- So at L5 the evaluator is read-only, always.
- The experiment can study how a loop would change its judge, without ever letting it.
7. Month 28: the L5 track asks for more (scenario)
- The L5 researcher shows a sandbox result: a loop that rewrote its own search procedure solved 30% more held-out tasks. She asks for 25% of research compute.
How the request was decided
Story events.
- What happened next.
- The L2 track re-implemented the useful part as a fixed, human-designed search procedure: an L2 change that can pass the gate.
- The lesson: L5 is where ideas are found; lower levels are where they are made safe to ship.
๐ Go deeper: can L5 ever reach a product like a coding agent?
- Yes, in a bounded form, and that is the plan for Season 2, Part 16 ("The harness as product").
- L4 products learn what (memories, skills); an L5 product would also change how it learns: its own checks, its rule for keeping skills, its harness, its experiments on itself.
- Use cases a coding agent could offer.
- A harness tuned per repository (try workflow variants on replayed tasks, keep what raises the merge rate).
- Writing its own extra checks for bugs it keeps making; pruning skills that never help; practising on past bugs overnight; learning when a cheaper model is enough.
- Why it is not shipped today.
- A system that can edit its judge can make the judge easier (the Darwin Gรถdel Machine authors reported such cases); behaviour drift is hard to audit; prompt injection could change the agent for good.
- Maestro's bounded-L5 rule.
- A read-only judge, sandbox replay first, every self-change shown as a diff with before/after scores for a human to approve, versioned and reversible, and never touching permissions, the judge or safety rules: the Season 1 gate, one level up.
8. Five mistakes
- Researching only the level you sell (L4) and getting surprised at L5.
- Letting the riskiest track grow because its demos are the most exciting.
- Any loop that can write to its own evaluator.
- Tracks above L2 without a named safety reviewer.
- No written rule for how sandbox ideas move down the ladder.
9. Delegation box
- The cap on L5 and the boundary policy
- Final allocation decisions
- Reading every L5 report
- Running the allocator and boundary checks on every proposal
- Drafting track reports for researchers to check
- Watching sandbox telemetry
- An external lab to red-team the sandbox
- Advisers from academic RSI groups
10. Budget and risk tracker
| Item | Status after Part 5 | Note |
|---|---|---|
| Research org | 5 tracks, 10 researchers | From rsi_portfolio.py |
| Boundary checks | 4 of 6 proposals may run | From boundary_check.py |
| Product safety | Protected | L5 has no path to the product |
| Research focus | Watch | L5 demos pull attention |
| Moat risk | Watch | L4 is half the team |
11. The series
The Maestro, Season 2: 23 parts in five phases (Season 1: 18 parts)
12. Reading the numbers
- Primary / standard
- The RSI level taxonomy and paper counts from the CC0 Awesome-RSI catalog (survey arXiv:2609.11873) via the site's RSI map.
- Secondary โ ๏ธ
- None in this part.
- Our assumptions
- Track values, probabilities, risks, people, compute, experiments and the month-28 events are the story's scenario.