Home โ€บ Blog โ€บ The Maestro, Season 2 ยท Part 17
The Maestro, Season 2 ยท Part 17 ยท Phase 8 ยท ๐Ÿ”Œ CTO hat

Model-Agnostic or Not?

If the harness is the product, why insist on running only your own model? This part follows an argument that splits the company: the researchers who built Maestro-2 against the product team, who want to route each task to whichever model does it best, including a rival's. The answer turns out to be a router, a rule, and a plan for the day the other provider changes its terms.

FL
FrontierAGI Team
๐Ÿ’ผ CEO๐Ÿงญ Product๐Ÿ’ฐ Finance
โš ๏ธ How to read this (as of 4 Nov 2026) Gopala's company is a teaching scenario. router.py and lockin_risk.py were run (real output below). Quality rates and costs are scenario numbers; "frontier API" stands for any external frontier model, and its prices are placeholders, not a real vendor's. Ideas from the owner's startup notes (Sosin, Chase) are labelled. Disclosure: Claude, made by Anthropic, helps write this site.

1. Month 43: the argument

๐Ÿ“– The story

Tomรกs took it personally, and everyone could see it. He had spent nine months on Maestro-2. Now Sofia was proposing that fifteen percent of the company's traffic go to a model built by a frontier lab that was, in some markets, a direct competitor.

We built a model so we would not depend on them, he said.

We built a model so we would be good at our customers' work, Sofia said. On Rust and Kotlin, they are better. Our customers do not care whose weights did it. They care whether it merges.

Gopala let them argue for twenty minutes. Then he asked Arjun for one number: what happens to the business if the other company doubles its price, or turns us off?

๐Ÿ”ฌ In plain words: model-agnostic A model-agnostic product can switch which model handles a task without customers noticing. It needs a router (which model for which task), evaluations for every route, and the discipline to keep the parts you own, such as data, skills and the gate, independent of any one model.

2. The map: one task, three possible models

๐Ÿ•ธ๏ธ How a task is routed, and what could go wrong

or click any node

Models in the middle row; the rule that chooses; the shock and the fallback; and what the company owns whichever model it uses.

3. The router

router.py ยท Python 3, no dependencies; scenario quality and costs
"""router.py โ€” model-agnostic or not?

Each task type can go to Maestro-2 (own model), an external frontier API, or a
small open model. Quality (merge rate) and cost per task are SCENARIO numbers;
external prices are placeholders, not any vendor's list price. The router picks,
per task type, the cheapest option within 2 points of the best quality.
"""
OPT = {"Maestro-2": {}, "frontier API": {}, "small open": {}}
TASKS = [  # type, share of volume, {option: (merge rate, $ per task)}
    ("repo-specific fixes", 0.45, {"Maestro-2": (0.61, 0.08), "frontier API": (0.55, 0.22), "small open": (0.38, 0.02)}),
    ("dependency upgrades", 0.20, {"Maestro-2": (0.72, 0.06), "frontier API": (0.73, 0.18), "small open": (0.66, 0.02)}),
    ("new-language code", 0.15, {"Maestro-2": (0.48, 0.10), "frontier API": (0.63, 0.25), "small open": (0.30, 0.03)}),
    ("docs and tests", 0.20, {"Maestro-2": (0.80, 0.05), "frontier API": (0.81, 0.15), "small open": (0.79, 0.01)})]

def main():
    q = c = 0; share = {k: 0 for k in OPT}
    for t, w, o in TASKS:
        best = max(v[0] for v in o.values())
        pick = min((k for k, v in o.items() if v[0] >= best - 0.02), key=lambda k: o[k][1])
        q += w * o[pick][0]; c += w * o[pick][1]; share[pick] += w
        print(f"{t:22} -> {pick:12} merge {o[pick][0]:.0%}  ${o[pick][1]:.2f}")
    m2q = sum(w * o['Maestro-2'][0] for _, w, o in TASKS); m2c = sum(w * o['Maestro-2'][1] for _, w, o in TASKS)
    print(f"\nrouted:      merge {q:.1%}  ${c:.3f}/task  volume " + ", ".join(f"{k} {v:.0%}" for k, v in share.items()))
    print(f"Maestro-2 only: merge {m2q:.1%}  ${m2c:.3f}/task")

if __name__ == "__main__":
    main()
$ python3 router.py # cheapest option within 2 points of the best repo-specific fixes -> Maestro-2 merge 61% $0.08 dependency upgrades -> Maestro-2 merge 72% $0.06 new-language code -> frontier API merge 63% $0.25 docs and tests -> small open merge 79% $0.01 routed: merge 67.1% $0.087/task volume Maestro-2 65%, frontier API 15%, small open 20% Maestro-2 only: merge 65.1% $0.073/task

Where tasks go

Maestro-2 ยท 65% of volumerepo-specific fixes and dependency upgrades: our skills and environments win
Small open model ยท 20%docs and tests: 79% merged at $0.01
Frontier API ยท 15%new-language code: 63% merged vs 48% on Maestro-2

Real output. Routed: 67.1% merged at $0.087 per task; Maestro-2 alone: 65.1% at $0.073.

4. The day the other provider changes its terms

lockin_risk.py ยท Python 3, no dependencies; scenario shocks
"""lockin_risk.py โ€” what if the external provider changes the terms?

Stress-test the routed plan: the frontier API raises prices 2x, deprecates the
model (quality drops while re-tuning), or is cut off. Gross margin per task is
compared with a plan that keeps a fallback route warm. All numbers SCENARIO.
"""
PRICE, BASE_COST, EXT_SHARE, EXT_COST = 0.28, 0.087, 0.15, 0.25   # platform price/task; routed cost (router.py); share on API

def margin(cost): return (PRICE - cost) / PRICE

def main():
    print(f"{'shock':36} {'cost/task':>9} {'margin':>7}")
    for name, extra in (("none", 0), ("API price x2", EXT_SHARE * EXT_COST), ("API cut off, fallback to Maestro-2", EXT_SHARE * (0.10 - EXT_COST))):
        c = BASE_COST + extra; print(f"{name:36} {c:9.3f} {margin(c):7.1%}")
    print("\ncut-off also costs quality on new-language tasks (63% -> 48% merge) until Maestro-3")

if __name__ == "__main__":
    main()
$ python3 lockin_risk.py # price doubling and cut-off shock cost/task margin none 0.087 68.9% API price x2 0.124 55.5% API cut off, fallback to Maestro-2 0.065 77.0% cut-off also costs quality on new-language tasks (63% -> 48% merge) until Maestro-3

Gross margin per platform task

Fallback to Maestro-2 (cut-off)77.0% ยท lower quality
No shock68.9%
External API price x255.5%

Real output. A cut-off is cheaper but costs quality on new-language tasks (63% โ†’ 48% merged) until Maestro-3.

๐Ÿ“– The story

Arjun's answer settled it. A doubled price would cost thirteen points of margin; a cut-off would cost quality on one task type, not the business. Both were survivable, as long as the fallback was real: tested every week, not just written in a document.

They shipped the router with three rules. Every route evaluated on the same sealed sets. Every external route with a warm fallback. And no customer data sent to any external model without that customer opting in.

5. Sandbox: build the router

๐ŸŽฎ Sandbox: build the router

Same table and rule as router.py. External prices are placeholders, not any provider's list price.

6. Five mistakes

7. Delegation box

๐Ÿ™‹ Gopala himself
  • Which external providers are allowed at all
  • The customer opt-in policy
๐Ÿค– AI agents
  • Weekly route evaluations
  • Fallback drills
๐Ÿค Bought, rented or borrowed
  • An external frontier API for one task type
  • Hosting for the small open model

8. Budget and risk tracker

ItemStatus after Part 17Note
RoutingLive65% / 20% / 15%
Quality67.1% merged+2.0 points vs Maestro-2 only
Provider riskManagedmargin 55.5% if the API price doubles
FallbackDrilled weeklycut-off survivable

9. The series

The Maestro, Season 2: 23 parts in five phases (Season 1: 18 parts)

VerdictMaestro became model-agnostic where it paid: 15% of tasks go to an external frontier model that is better at them, 20% to a small open model that is good enough, and the rest to Maestro-2. What makes it safe is not the router but what the company owns independently of any model, and a fallback that is tested, not promised. Next: selling all this to large enterprises.

10. Reading the numbers

Primary / standard
None beyond the methods.
Secondary โš ๏ธ
None in this part.
Our assumptions
Quality rates, prices, shocks and story events are the scenario. Owner's notes (Sosin, Chase) are summaries; Sosin's views are an investor's opinion.