Home โ€บ Blog โ€บ The Maestro, Season 2 ยท Part 19
The Maestro, Season 2 ยท Part 19 ยท Phase 8 ยท ๐Ÿ“ˆ CEO hat

Adoption Metrics That Matter

Every chart on the company dashboard was going up and to the right. Then a design partner asked a question nobody could answer: how many hours did Maestro actually save us this quarter? This part closes Phase 8 with the uncomfortable discovery that the company had been measuring activity, and the one number it chose instead.

FL
FrontierAGI Team
๐Ÿ’ผ CEO๐Ÿงญ Product๐Ÿ’ฐ Finance
โš ๏ธ How to read this (as of 4 Nov 2026) Gopala's company is a teaching scenario. metrics.py and cohort_retention.py were run (real output below). All product and revenue data are scenario numbers; net revenue retention is a standard SaaS measure. Ideas from the owner's startup notes are labelled. Disclosure: Claude, made by Anthropic, helps write this site.

1. Month 46: up and to the right

๐Ÿ“– The story

The quarterly review opened with Sofia's favourite slide. Sessions up sixty percent. Tokens processed up eighty-seven. A new logo wall with eleven enterprise customers on it.

Ines, the lead investor, waited until the end. Then she read out an email from the CTO of their largest design partner. It was polite and very short: Can you tell us how many engineering hours Maestro saved us this quarter? Our CFO is asking.

Nobody in the room could answer. Mira went back to her desk and did not leave until she could.

๐Ÿ”ฌ In plain words: vanity metric A vanity metric grows when the product is busy, not necessarily when it is useful: sessions, sign-ups, tokens. A value metric grows only when customers get what they pay for: here, code that is merged and stays merged.

2. The map: activity vs value

๐Ÿ•ธ๏ธ Which numbers measure value

or click any node

Red: activity that can rise while value falls. Green: value. The north star at the bottom.

3. What the dashboard did not show

metrics.py ยท Python 3, no dependencies; scenario product data
"""metrics.py โ€” vanity metrics vs value metrics.

Two quarters of product data. Sessions and tokens went up; the metrics tied to
customer value tell a different story. Data are SCENARIO numbers.
"""
Q = {"Q1": dict(sessions=410_000, tokens_b=38.0, tasks=2_100_000, merged=1_090_000, reverted=41_000, hours_saved=520_000),
     "Q2": dict(sessions=655_000, tokens_b=71.0, tasks=2_900_000, merged=1_310_000, reverted=88_000, hours_saved=560_000)}

def main():
    a, b = Q["Q1"], Q["Q2"]
    for k in ("sessions", "tokens_b"): print(f"vanity  {k:12} {a[k]:>12,} -> {b[k]:>12,}  ({b[k] / a[k] - 1:+.0%})")
    acc = lambda q: q["merged"] / q["tasks"]; kept = lambda q: (q["merged"] - q["reverted"]) / q["merged"]
    print(f"value   merge rate   {acc(a):12.1%} -> {acc(b):12.1%}")
    print(f"value   kept 30 days {kept(a):12.1%} -> {kept(b):12.1%}")
    print(f"value   hours saved  {a['hours_saved']:>12,} -> {b['hours_saved']:>12,}  ({b['hours_saved'] / a['hours_saved'] - 1:+.0%})")
    print(f"\nhours saved per 1k sessions: {a['hours_saved'] / a['sessions'] * 1000:.0f} -> {b['hours_saved'] / b['sessions'] * 1000:.0f}")

if __name__ == "__main__":
    main()
$ python3 metrics.py # vanity vs value, Q1 โ†’ Q2 vanity sessions 410,000 -> 655,000 (+60%) vanity tokens_b 38.0 -> 71.0 (+87%) value merge rate 51.9% -> 45.2% value kept 30 days 96.2% -> 93.3% value hours saved 520,000 -> 560,000 (+8%) hours saved per 1k sessions: 1268 -> 855

Q1 โ†’ Q2

Tokens processed+87%
Sessions+60%
Hours saved+8%
Merge rate51.9% โ†’ 45.2%
Value per 1k sessions1,268 โ†’ 855 hours

Real output (merge rate and value per session shown as relative change). Negative bars are drawn at their size; red means down.

๐Ÿ“– The story

Mira's answer arrived at two in the morning, in a document titled We are busier, not better. New platform customers were throwing harder, messier tasks at Maestro. The agent tried more, flailed more, and burned more tokens doing it. Merged work still grew, but only by eight percent in hours saved, against sixty percent more sessions.

The good news, she wrote, is that every one of these failures is a trace. We know exactly where to look.

4. The number investors trust

๐Ÿ”ฌ In plain words: net revenue retention Take every customer who signed in the same period and compare what they pay now with what they paid at the start. Above 100% means the growth of customers who stay outweighs everyone who leaves. For a product sold per task, it is the clearest sign that customers get more value over time.
cohort_retention.py ยท Python 3, no dependencies; scenario revenue
"""cohort_retention.py โ€” net revenue retention by customer cohort.

Monthly recurring revenue ($k) per cohort at month 0, 6 and 12 after signing.
Net revenue retention (NRR) = revenue a cohort pays now / what it paid at
start, including expansion and churn. Data are SCENARIO numbers.
"""
COH = {"PR-bot cohort (month 14)": (300, 318, 345), "Platform early (month 30)": (520, 690, 860), "Enterprise (month 38)": (900, 1150, None)}

def main():
    for n, (m0, m6, m12) in COH.items():
        line = f"{n:28} start ${m0}k  m6 {m6 / m0:5.0%}"
        if m12: line += f"  m12 {m12 / m0:5.0%}"
        print(line)
    print("\nNRR above 100% means existing customers grow faster than any churn: the platform cohort is the business")

if __name__ == "__main__":
    main()
$ python3 cohort_retention.py # NRR by cohort PR-bot cohort (month 14) start $300k m6 106% m12 115% Platform early (month 30) start $520k m6 133% m12 165% Enterprise (month 38) start $900k m6 128% NRR above 100% means existing customers grow faster than any churn: the platform cohort is the business

Revenue from each cohort vs its first month

Platform early (month 30) ยท 12 months165%
Enterprise (month 38) ยท 6 months128%
PR-bot cohort (month 14) ยท 12 months115%

Real output. Platform customers expand fastest: they build more agents once the first ones work.

๐Ÿ“– The story

The cohort table did what the dashboard could not. The PR-bot customers from Season 1 were steady. The platform customers were growing their spend by two-thirds a year. The enterprise cohort was too young to judge, and already above 125%.

The company changed its north star that week: merged changes still in place after thirty days, per customer, per month. Sessions came off the board slide. Nobody missed them.

5. Sandbox: grow the right number

๐ŸŽฎ Sandbox: grow the right number

Scales Q2 from metrics.py: tasks per session, 93.3% kept and about 0.46 hours saved per kept change are held fixed. Scenario numbers.

6. Five mistakes

7. Delegation box

๐Ÿ™‹ Gopala himself
  • The north-star metric and the board dashboard
๐Ÿค– AI agents
  • Turning failing traces into evaluation sets
  • Sampling hours-saved estimates with customers
๐Ÿค Bought, rented or borrowed
  • Customer finance teams for hours-saved validation

8. Budget and risk tracker

ItemStatus after Part 19Note
North starMerged changes kept 30 daysper customer, per month
NRRPlatform 165%enterprise 128% at 6 months
Merge rate45.2%, fallingharder tasks from new users
Phase 8Completeproduct grown up

9. The series

The Maestro, Season 2: 23 parts in five phases (Season 1: 18 parts)

VerdictUsage grew 60% while value per session fell by a third: the company had been measuring activity. It now reports one north star, merged changes still in place after 30 days, and watches net revenue retention, where platform customers grow their spend by 65% a year. Phase 8 ends here; Phase 9 is about competing at the frontier.

10. Reading the numbers

Primary / standard
Net revenue retention as a standard SaaS metric.
Secondary โš ๏ธ
None in this part.
Our assumptions
All product, revenue and cohort data and all story events are the scenario. Owner's notes (Chase, Matias) are summaries, not transcripts.