The Maestro, Season 2 ยท Part 9 ยท Phase 6 ยท ๐ CEO hat
When Research Says No
The most exciting direction in Phase 6, agents that work for hours on one task, gets a fair test with written kill criteria. Six months later it fails three of them. This part closes Phase 6 with the kill itself: the criteria, the numbers, why the $1.9M already spent does not matter, what happens to the team and the knowledge, and why Gopala celebrates it.
๐ผ CEO๐ฌ Research director๐งโ๐ซ Manager
โ ๏ธ How to read this (as of 3 Nov 2026)
kill_decision.py and sunk_cost.py were run (real output below). The whole episode is the story's scenario: criteria, results, costs and probabilities are invented for teaching. Disclosure: Claude, made by Anthropic, helps write this site.
Mission log: months 27โ34
The map: how a direction is stopped
The kill criteria at month 33 (scenario)
Why the $1.9M does not count
Sandbox: try to save it
What a good kill leaves behind
Five mistakes
Delegation box
Budget and risk tracker
The series
Reading the numbers
1. Mission log: months 27โ34
Week 108 ยท The test
- Long-horizon agents get two researchers, six months and four written criteria.
Week 132 ยท Results
- Three of four criteria fail (
kill_decision.py).
Week 133 ยท The kill
- One meeting; team moved to environments; write-up published internally.
2. The map: how a direction is stopped
๐ธ๏ธ From exciting idea to a clean kill
Failed criteria in red, guards in green. Every number comes from the two scripts below.
- Three things to notice.
- The decision follows from rules written six months earlier.
- The money already spent has an edge marked "ignored".
- The kill produces three useful outputs: people, knowledge and culture.
3. The kill criteria at month 33 (scenario)
Scenario
- Month 33: the long-horizon direction reports its results against the criteria agreed at month 27.
kill_decision.py ยท Python 3, no dependencies; scenario results
"""kill_decision.py โ the long-horizon agents direction against the kill
criteria written before it started (month 27). SCENARIO results, month 33.
A direction is killed if ANY pre-registered criterion fails at its checkpoint.
"""
CRITERIA = [ # name, target, observed, higher is better
("held-out tasks solved, 4h+ horizon (%)", 25, 11, True),
("gain over the L4 skill library baseline (pts)", 3.0, 0.4, True),
("cost per solved task vs product ($)", 2.0, 6.8, False),
("customers who would pay in interviews (of 12)", 4, 5, True),
]
def main():
fails = 0
for name, target, obs, up in CRITERIA:
ok = obs >= target if up else obs <= target
fails += not ok
print(f"{name:48} target {target:>5} observed {obs:>5} {'pass' if ok else 'FAIL'}")
print("decision:", "KILL the direction" if fails else "continue", f"({fails} of {len(CRITERIA)} criteria failed)")
if __name__ == "__main__":
main()$ python3 kill_decision.py # criteria fixed at month 27
held-out tasks solved, 4h+ horizon (%) target 25 observed 11 FAIL
gain over the L4 skill library baseline (pts) target 3.0 observed 0.4 FAIL
cost per solved task vs product ($) target 2.0 observed 6.8 FAIL
customers who would pay in interviews (of 12) target 4 observed 5 pass
decision: KILL the direction (3 of 4 criteria failed)
Observed vs target
Bar = observed as a share of target (cost: target รท observed). Real output.
4. Why the $1.9M does not count
sunk_cost.py ยท Python 3, no dependencies; scenario estimates
"""sunk_cost.py โ why the $1.9M already spent should not decide anything.
Compares the expected value of continuing six more months against stopping
and moving the team to the environments track. Probabilities and values are
the story's SCENARIO estimates; the money already spent appears in neither
option, because it is gone either way.
"""
SPENT = 1.9 # $M already spent (sunk)
CONT_COST = 2.4 # $M for six more months
P_SUCCESS = 0.12 # chance the direction recovers, given month-33 evidence
WIN_VALUE = 12.0 # $M value if it does
ENV_VALUE = 3.1 # $M expected value of moving the same people to environments
def main():
cont = P_SUCCESS * WIN_VALUE - CONT_COST
print(f"already spent (ignored): ${SPENT}M")
print(f"continue: {P_SUCCESS:.0%} x ${WIN_VALUE}M - ${CONT_COST}M = ${cont:+.2f}M")
print(f"stop and redeploy: ${ENV_VALUE:+.2f}M")
print(f"break-even chance of success to justify continuing: {(CONT_COST + ENV_VALUE) / WIN_VALUE:.0%}")
if __name__ == "__main__":
main()$ python3 sunk_cost.py # compare only the future
already spent (ignored): $1.9M
continue: 12% x $12.0M - $2.4M = $-0.96M
stop and redeploy: $+3.10M
break-even chance of success to justify continuing: 46%
The only comparison that matters
Real output. Continuing needs a 46% chance of success; the evidence says 12%.
๐ Go deeper: the sunk-cost trap, in one line
- Money already spent is the same in both futures, so it cannot tell you which future is better.
- Teams often keep a project alive "so the money was not wasted"; that spends more money to avoid admitting the first was lost.
5. Sandbox: try to save it
๐ฎ Sandbox: try to save the project
Same numbers as kill_decision.py and sunk_cost.py; scenario estimates.
- Try this.
- Relax the criteria by 90%: the direction finally passes, yet continuing is still worth less than stopping.
- Raise the chance of recovery to 50%: only then does continuing beat redeploying.
- This is why the criteria are written first: relaxing them after the fact lets worse options through.
6. What a good kill leaves behind
Three outputs of the kill
๐๏ธPeopletwo researchers join environments
๐Knowledgewrite-up and three reusable parts
๐Culturethe kill is thanked in public
Story events.
- Why celebrate a failure?
- If killing a project feels like punishment, people stop proposing bold ideas or start hiding bad results.
- Phase 6 ends with a research engine that explores widely, kills quickly and ships only through the gate.
7. Five mistakes
- No kill criteria, so every review becomes a debate.
- Relaxing the criteria once the results are in.
- Counting money already spent.
- Dissolving the team instead of moving it.
- Killing quietly, so nothing is learned.
8. Delegation box
๐ Gopala himself
- The kill decision and the all-hands message
๐ค AI agents
- Assembling results against the criteria
- Drafting the write-up for the team to check
๐ค Bought, rented or borrowed
- The independent director as witness to the criteria
9. Budget and risk tracker
| Item | Status after Part 9 | Note |
|---|---|---|
| Long-horizon agents | Killed at month 33 | 3 of 4 criteria failed |
| Money | $1.9M spent, $2.4M saved | From sunk_cost.py |
| Research engine | Healthy | Explores, kills, ships via the gate |
| Morale | Watch | Kill celebrated; team redeployed |
10. The series
The Maestro, Season 2: 23 parts in five phases (Season 1: 18 parts)
VerdictPhase 6 ends with a "no": the most exciting direction fails its own criteria and is stopped in one meeting, without counting sunk money and without losing the people or the knowledge. That is what a working research engine looks like. Next, Phase 7: sizing and building Maestro-2.
11. Reading the numbers
- Primary / standard
- Sunk-cost reasoning and pre-registered criteria are standard decision practice.
- Secondary โ ๏ธ
- None in this part.
- Our assumptions
- The direction, criteria, results, costs and probabilities are the story's scenario.