Home โ€บ Blog โ€บ The Maestro ยท Part 17
The Maestro ยท Part 17 ยท Phase 4 ยท ๐Ÿ”„ Operator hat

Operations and the Flywheel

Shipping is a beginning. From now on Maestro runs every day for paying clients: models are updated every two weeks by the loop, the service must stay up, costs must stay down, incidents must be handled, and the verified-skill flywheel must keep turning without drifting. This part covers service-level objectives and error budgets (with a burn-rate script), progressive rollouts and rollback, the weekly operating rhythm, a simulation of whether the flywheel compounds or stalls, cost operations, and how one maestro hands parts of the job to a small team.

FL
FrontierAGI Team
๐Ÿ”„ Operator๐Ÿ› ๏ธ SRE๐Ÿ”ฌ Researcher๐Ÿ’ป Coder
โš ๏ธ How to read this (as of 30 Oct 2026) Error budgets and multi-window burn-rate alerts are from Google's SRE books; DORA metrics from the DORA research programme. burn_rate.py and flywheel.py were run (real output below) on simulated error rates and scenario parameters. rollout.yaml was not executed for this article. Operating results are the scenario. Disclosure: Claude, made by Anthropic, helps write this site.

1. Mission log: months 15โ€“18

Month 15 ยท SLOs
  • Defines service-level objectives with partners; writes burn_rate.py; sets alerting.
Months 15โ€“17 ยท Rhythm
  • Weekly operations review; two-week loop cycles shipping through canaries; one severity-1 incident (section 6).
Month 18 ยท Handing over
  • First two hires take on-call and the release pipeline; Gopala keeps the gate and the data agreements.

2. Promises in numbers: SLOs

Maestro's service-level objectives

โœ… Availability99.5% of requests succeed over 30 days
โฑ๏ธ Interactive latencyFirst token within 1.5 s for 95% of requests
๐ŸŽฏ Task qualityAccepted-change rate within its baseline interval
๐Ÿ’ฐ CostServing cost per solved task below target

Our objectives for the story; the first two are standard SRE practice, the last two product-specific.

3. Alerting on the budget, not on every error

burn_rate.py ยท Python 3, no dependencies; simulated error rates
"""burn_rate.py โ€” SLO error budgets and multi-window burn-rate alerts.

SLO: 99.5% of agent requests succeed (non-5xx, under the latency limit)
over 30 days. Burn rate = error rate / allowed error rate. The Google SRE
Workbook suggests paging when the budget burns fast in both a long and a
short window (e.g. 14.4x over 1 h and 5 min). Error rates are SIMULATED.
"""
SLO = 0.995
ALLOWED = 1 - SLO
RULES = [("page", 14.4, "1h", "5m"), ("page", 6.0, "6h", "30m"), ("ticket", 1.0, "3d", "6h")]
WINDOWS = {  # scenario error rates per window during an incident
    "5m": 0.09, "30m": 0.05, "1h": 0.08, "6h": 0.02, "3d": 0.004,
}

def main():
    print(f"30-day budget: {ALLOWED:.1%} of requests = {ALLOWED*30*24*60:.0f} minutes of full outage")
    for sev, factor, long_w, short_w in RULES:
        bl = WINDOWS[long_w] / ALLOWED; bs = WINDOWS[short_w] / ALLOWED
        fire = bl >= factor and bs >= factor
        print(f"{sev:6s} if burn >= {factor:4.1f}x over {long_w:>3s} and {short_w:>3s}: "
              f"{bl:5.1f}x / {bs:5.1f}x -> {'FIRE' if fire else 'quiet'}")
    hours_left = 30 * 24 * ALLOWED / WINDOWS["1h"]
    print(f"at the current 1 h error rate the whole month's budget lasts {hours_left:.1f} hours")

if __name__ == "__main__":
    main()
$ python3 burn_rate.py # error rates during a simulated incident 30-day budget: 0.5% of requests = 216 minutes of full outage page if burn >= 14.4x over 1h and 5m: 16.0x / 18.0x -> FIRE page if burn >= 6.0x over 6h and 30m: 4.0x / 10.0x -> quiet ticket if burn >= 1.0x over 3d and 6h: 0.8x / 4.0x -> quiet at the current 1 h error rate the whole month's budget lasts 45.0 hours

4. Try the burn rate

Error budget calculator

5. Shipping a new model safely

Progressive rollout of a loop update

๐ŸšฆGatePart 11: fixed rules + human sign-off
๐ŸคCanary 5%Opted-in partners, 24 h
๐Ÿ”AnalysisSLOs and acceptance vs the old model
๐Ÿ“ˆ25%48 h, analysis again
โœ…100%Old version kept warm for instant rollback

Any failed analysis rolls back automatically.

rollout.yaml ยท canary rollout sketch ยท not executed for this article
# rollout.yaml โ€” progressive delivery of a new Maestro-1 version (sketch in the
# style of a Kubernetes canary controller such as Argo Rollouts; field names
# follow its documentation, check them for your version).
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata: {name: maestro-1}
spec:
  strategy:
    canary:
      steps:
      - setWeight: 5          # 5% of conversations, design partners who opted in
      - pause: {duration: 24h}
      - analysis: {templates: [{templateName: maestro-slo-and-acceptance}]}
      - setWeight: 25
      - pause: {duration: 48h}
      - analysis: {templates: [{templateName: maestro-slo-and-acceptance}]}
      - setWeight: 100
      abortScaleDownDelaySeconds: 3600   # keep the old version warm for instant rollback

6. An incident, start to finish

Scenario
  • Month 16, 14:05: a dependency update in the sandbox image breaks test runs for Python 3.9 repositories; error rate jumps to 8%; the fast burn-rate rule pages at 14:11.
  • 14:20: the canary controller is not involved (no model change), so on-call rolls back the sandbox image; errors fall by 14:32.
  • Next day: post-incident review adds an image canary and a test against the oldest supported Python version. About 10% of the month's error budget was used.
  • These events are the story.

7. The weekly operating rhythm

DayMeeting or taskInputs
MondayOperations review (30 min)SLOs, budget burn, incidents, cost per solved task
TuesdayLoop cycle planningNew partner tasks, last gate decisions
WednesdayClient reviews (rotating)Acceptance, edits, reverts per client
ThursdaySafety reviewMonitoring samples, red-team updates (Part 13)
FridayRelease and gate sign-off (every other week)Gate report, canary results

8. Is the flywheel compounding?

flywheel.py ยท Python 3, no dependencies; scenario parameters
"""flywheel.py โ€” does the verified-skill flywheel compound or stall?

Each two-week cycle: tasks attempted -> verified skills -> (some) gate
approvals -> success rate rises a little, which brings more tasks.
Diminishing returns: each new skill helps less as the library grows.
All parameters are SCENARIO assumptions; the point is the shape.
"""
import math

def run(cycles=12, tasks=400, success=0.48, approve=0.5, gain_per_k=0.012, overlap=0.0004):
    lib = 0
    print(f"{'cycle':>5s} {'tasks':>6s} {'success':>8s} {'new skills':>10s} {'library':>8s}")
    for c in range(1, cycles + 1):
        solved = tasks * success
        new = solved * math.exp(-overlap * lib)          # more overlap with existing skills later
        lib += new
        if c % 2 == 0:                                    # gate approves an update every other cycle on average
            success = min(0.85, success + approve * gain_per_k * (new / 100))
        tasks = int(tasks * (1.04 + (success - 0.48)))   # better product -> more usage
        if c in (1, 3, 6, 9, 12):
            print(f"{c:5d} {tasks:6d} {success:8.1%} {new:10.0f} {lib:8.0f}")

if __name__ == "__main__":
    run()
$ python3 flywheel.py # scenario parameters cycle tasks success new skills library 1 416 48.0% 192 192 3 459 49.1% 185 561 6 554 51.3% 179 1103 9 696 52.4% 187 1655 12 922 54.7% 197 2235

Flywheel over 12 two-week cycles (scenario)

Cycle 1: tasks416
Cycle 6: tasks554
Cycle 12: tasks922
Cycle 1: library192 skills
Cycle 12: library2,235 skills

Real output of flywheel.py on scenario parameters.

9. Cost operations

10. One maestro, then a small team

Hats handed over by month 18 (scenario)

๐Ÿ”„ On-call and release pipelineFirst hire: site reliability engineer
๐Ÿค– Integrations and onboardingSecond hire: product engineer
๐Ÿšฆ The gate and sign-offGopala keeps it, with a second reviewer
๐Ÿ“œ Data agreements and complianceGopala, with counsel
๐Ÿ”ฌ Research agendaGopala

He gives away operations first and keeps what defines the company.

11. Watching for drift

12. Operating the data agreements

13. Learning from incidents

From incident to prevention

๐ŸšจIncidentDetected by alert or client
๐ŸงฏMitigateRollback first, diagnose second
๐Ÿ“ReviewBlameless, within a week
๐ŸงชNew checkAlert, test, canary step or rule

Each incident should make the next one less likely.

14. Capacity planning, monthly

15. Security in daily operations

16. Agents helping run the company

17. Seven operations mistakes

18. Delegation box

๐Ÿ™‹ Gopala himself
  • SLOs and their trade-offs with clients
  • The gate and sign-off
  • What to hand over and when
๐Ÿค– AI agents
  • Incident timelines and first-draft reviews
  • Weekly operations reports
  • Cost anomaly detection
๐Ÿค Bought, rented or borrowed
  • Open-source monitoring and rollout tools
  • Two hires: SRE and product engineer
  • The GPU provider's support

19. Budget and risk tracker

ItemStatus after Part 17Note
Spend to date~$11.4M (scenario)Serving, operations, first hires
OperationsSLOs met 2 of 3 months (scenario)One severity-1 incident
TimeMonth ~18Scorecard next
Reliability riskManagedBurn-rate alerts, canaries
Flywheel riskWatchGains per cycle are small (scenario)
Key-person riskReducedOn-call and pipeline handed over

20. The series

The Maestro: 18 parts in five phases

Verdict
  • Operations turn a launch into a business: objectives in numbers, alerts that fire on what matters, rollouts that can undo themselves, and a weekly rhythm that watches both the service and the flywheel.
    • The simulation was sobering: the loop compounds, but slowly.
    • Next, Part 18: the scorecard, and what Gopala would do differently.

21. Reading the numbers

Primary / standard
Error budgets and multi-window burn-rate alerting (14.4ร— over 1 h and 5 min, 6ร— over 6 h and 30 min, 1ร— over 3 days and 6 h) (Google SRE Workbook). DORA four key metrics (DORA research programme).
Secondary โš ๏ธ
None in this part.
Our assumptions
SLO targets, simulated error rates, flywheel parameters, incident details, hires and costs.