Sizing Maestro-2
Phase 7 opens. Month 34: the research engine has said no to the exciting bet and yes to environments. Now the company has to decide whether to train a second model from scratch, and if so how big. This part follows Gopala and his research director through the first decision of any frontier run: what exactly are we buying with twenty million dollars of compute?
size_m2.py and triage.py were run (real output below). The scaling-law constants are the published Chinchilla fit; failure counts, prices, MFU and penalty weights are scenario assumptions. Disclosure: Claude, made by Anthropic, helps write this site.
1. Month 34: the question nobody wanted to ask
The board meeting ended at six. By seven, the whiteboard in the small meeting room still said what Ines, the lead investor, had written on it in green: Maestro-2: yes or no?
Gopala had expected the question. He had not expected how quickly the room divided. The sales lead wanted a bigger model because a rival had just announced one. The finance lead wanted to know why a company with a working product would spend a third of its Series A on a single training run. Mira Kovaฤ, the research director he had hired in month 24, said nothing until everyone else had finished. Then she said: Before we size a model, let us find out what the model is for.
She left with a laptop and four hundred failed customer tasks.
- What is being decided.
- Whether to pre-train a new base model (Maestro-2), or keep improving Maestro-1 with post-training, context and harness work.
- If yes: how many active parameters, how many training tokens, and how much compute.
- Why it matters.
- A frontier-style pre-training run is the single largest bet a model company makes: here roughly $15โ20M, a quarter of the Series A.
- It is also slow: from sizing to a usable model takes the best part of a year.
2. The map: from failures to a model size
๐ธ๏ธ Why train, and how big
Tap any node. Data in teal, ideas in purple, risks in red, the chosen option in green. The guided tour follows the story.
- Read it top to bottom.
- Top row: what the failures say.
- Middle: the two formulas every pre-training team uses.
- Bottom: four candidate shapes and the two costs that decide between them.
3. Train only for what nothing else can fix
Mira came back on Thursday with a spreadsheet and a short sentence. Half of these are not the model's fault.
One hundred and forty-eight failures were context: the agent had not been told how a customer's repository was laid out, or had pulled a stale skill from the library. One hundred and twelve were the harness: retry loops that never gave up, plans so long they forgot their own first step. Only the remaining hundred and forty were genuinely beyond Maestro-1, and even a better model would fix only some of those.
"""triage.py โ model, harness or context? Triage before you train.
400 failed customer tasks from the last quarter were labelled by the layer that
would fix them (the model + harness + context frame). Fixes to context and
harness are cheap and fast; only "model" failures justify a new pre-training
run. Counts, fix rates and costs are SCENARIO numbers from the story.
"""
FAILS = [ # (cause, layer, count, share a fix would recover)
("missing repo conventions in context", "context", 92, 0.80),
("stale or wrong skill retrieved", "context", 56, 0.70),
("tool-call loop / bad retry policy", "harness", 71, 0.75),
("plan too long, no checkpoints", "harness", 41, 0.60),
("multi-file reasoning over 4h tasks", "model", 63, 0.55),
("unfamiliar languages (Rust, Kotlin)", "model", 47, 0.65),
("long-context recall past 128k", "model", 30, 0.60)]
COST = {"context": ("2 weeks", 0.05), "harness": ("6 weeks", 0.3), "model": ("9 months", 22.0)} # time, $M
def main():
tot = sum(f[2] for f in FAILS)
for layer in ("context", "harness", "model"):
rows = [f for f in FAILS if f[1] == layer]; n = sum(r[2] for r in rows); rec = sum(r[2] * r[3] for r in rows)
print(f"{layer:8} {n:4} failures ({n / tot:4.0%}) fix recovers {rec:5.1f} tasks in {COST[layer][0]:9} for ${COST[layer][1]:5.2f}M")
cheap = sum(f[2] * f[3] for f in FAILS if f[1] != "model"); model = sum(f[2] * f[3] for f in FAILS if f[1] == "model")
print(f"\nfix context + harness first: {cheap:.0f} of {tot} failures ({cheap / tot:.0%})")
print(f"only a new model fixes: {model:.0f} of {tot} ({model / tot:.0%}) -> train Maestro-2 only for these")
if __name__ == "__main__":
main()Where 400 failures come from
Real output of triage.py. The context and harness fixes went into the product the same month.
- The decision this produced.
- Fix context and harness first: 191 of 400 failures recovered for about $0.35M.
- Train Maestro-2 only for the 83 failures (21%) that need a stronger model: long multi-file reasoning, unfamiliar languages, recall past 128k tokens.
- That turns a vague goal (a bigger model) into a measurable brief that defines the evaluation set for Part 14.
4. The two formulas behind every model size
Tomรกs Reyes, who had run Maestro-1's pre-training, drew two lines on the whiteboard under Ines's question. One tells us what it costs. The other tells us what we get. Everything else is taste.
๐ Go deeper: why use a dense-model law for a mixture-of-experts model?
- Because there is no better public one at this scale, and the team only needs it to rank options.
- Using active parameters for N ignores that a mixture-of-experts model stores more knowledge in its inactive experts, so the law probably undersells all four options equally.
- Before the real run, the team re-fits the law on its own small runs (Part 13).
- Season 1, Part 2 showed what happens when you trust a borrowed fit: predicted 1.410, got 1.499.
5. Four shapes, one budget
"""size_m2.py โ sizing Maestro-2 under a compute budget.
Training compute uses the standard estimate FLOPs ~= 6 x N x D (N = parameters
active per token, D = training tokens). Loss is predicted with the Chinchilla
fit of Hoffmann et al. (2022): L = E + A/N^a + B/D^b. Applying a dense-model
fit to a mixture-of-experts model by using ACTIVE parameters is a rough
approximation, used here only to rank options. Prices, MFU and the candidate
shapes and the two penalty weights are SCENARIO assumptions. Maestro-1's row
counts the final run only.
"""
E, A, B, a, b = 1.69, 406.4, 410.7, 0.34, 0.28 # Hoffmann et al. 2022, fit 3
PEAK, MFU, PRICE = 989e12, 0.40, 2.40 # H100 dense BF16 FLOP/s; our MFU; $/GPU-hour
def loss(n, d): return E + A / n ** a + B / d ** b
def main():
cands = [("Maestro-1 (today)", 11.7e9, 10e12), ("B1: 24B active, 60T", 24e9, 60e12),
("B2: 32B active, 50T", 32e9, 50e12), ("B3: 40B active, 48T", 40e9, 48e12),
("B4: 70B active, 25T", 70e9, 25e12)]
print(f"{'option':22} {'FLOPs':>9} {'tok/param':>9} {'GPU-hours':>10} {'cost $M':>8} {'pred. loss':>10}")
for name, n, d in cands:
f = 6 * n * d; gh = f / (PEAK * MFU) / 3600
print(f"{name:22} {f:9.2e} {d / n:9.0f} {gh / 1e6:9.2f}M {gh * PRICE / 1e6:8.1f} {loss(n, d):10.3f}")
# score = predicted loss + training-cost penalty + serving-cost penalty (active params drive serving cost)
print(f"\n{'option':22} {'loss':>6} {'+train':>7} {'+serve':>7} {'score':>7}")
sc = []
for name, n, d in cands[1:]:
tr, sv = 0.002 * 6 * n * d / 1e24, 0.0006 * n / 1e9
sc.append((loss(n, d) + tr + sv, name)); print(f"{name:22} {loss(n, d):6.3f} {tr:7.4f} {sv:7.4f} {sc[-1][0]:7.4f}")
print(f"\nchosen (lowest score): {min(sc)[1]}")
if __name__ == "__main__":
main()Predicted loss: lower is better
Real output. Bars are drawn on an absolute scale, so the differences look small; at this level, 0.01 in loss is a noticeable change in capability.
Final score = loss + training cost + serving cost
Real output. B1, B2 and B3 are almost tied: the decision is close, which is normal.
Ines read the table twice. So the biggest model is the smartest, and you are not building it.
The biggest model is the smartest on day one, Gopala said. We pay for its size on every token a customer sends, for as long as it runs. B4 would make us a research lab with a serving bill. B2 keeps us a company.
- Why B2.
- It wins once serving is priced in: training is paid once, serving every day.
- It sits at 96% of the EU compute line on pre-training alone, which looked like a comfortable margin.
6. Sandbox: size your own Maestro-2
๐ฎ Sandbox: size your own Maestro-2
Same formulas and constants as size_m2.py: Chinchilla loss fit, 6ND compute, 40% MFU, $2.40 per GPU-hour, our two penalty weights.
- Try this.
- Push active parameters to 70 and tokens to 25 (option B4): the best loss, but the score gets worse.
- Try 40B ร 50T: over the EU line before any post-training.
- Find a shape that beats B2's 1.8973. Around 28โ32B active and 50โ60T tokens comes close, which is why the choice is a judgement call.
7. Five mistakes
- Sizing a model before asking what its failures are.
- Choosing on predicted loss alone and forgetting the serving bill.
- Trusting a borrowed scaling law without re-fitting it on your own runs.
- Matching a rival's parameter count for the press release.
- Treating a 4% margin under a regulatory line as safe (Part 12).
8. Delegation box
- The train-or-not decision and the final shape
- Explaining the choice to the board
- Labelling failure traces
- Sweeping shapes with the scaling law
- Board and investors as a challenge panel
9. Budget and risk tracker
| Item | Status after Part 10 | Note |
|---|---|---|
| Decision | Train Maestro-2 (B2) | 32B active, 50T tokens |
| Compute | 9.60e24 FLOPs | 96% of the EU line |
| Cheap fixes | Shipped | 191 of 400 failures recovered |
| Regulatory margin | Thin | 4% under the line |
10. The series
The Maestro, Season 2: 23 parts in five phases (Season 1: 18 parts)
11. Reading the numbers
- Primary / standard
- Chinchilla scaling-law fit (Hoffmann et al., 2022); the 6ND compute estimate (Kaplan et al., 2020).
- Secondary โ ๏ธ
- None in this part.
- Our assumptions
- Failure counts, prices, MFU, penalty weights and all story events are the scenario.