Multi-Agent Systems: End-to-End Architecture
Some tasks are too broad for one agent: they span several disciplines, benefit from parallel work, or need a second opinion. Multi-agent systems split them across specialized agents coordinated by a supervisor. This piece covers the architecture, roles, routing, coordination patterns, shared state, execution, failure handling, cost and termination — and measures the questions that decide whether a team of agents is worth it: when cross-checking actually reduces errors, how much parallelism saves, and what sharing everything costs.
Why Multiple Agents
Multiple agents help when a task is complex, multidisciplinary, or needs different skills, tools or perspectives. The claimed benefits: specialization (each agent has a role), parallel execution, separation of responsibilities, better quality through cross-checking, handling long-running tasks, modularity, freedom to use different models, tools and contexts per agent, and robustness. Typical tasks: market research and strategy, product development, data analysis and reporting, code generation and testing, compliance and risk analysis, end-to-end business workflows.
Each of those benefits has a condition attached, and the measured sections below show which ones hold — and when they don't.
End-to-End Architecture
Take the goal "Evaluate whether our company should enter a new market":
- A supervisor (manager) agent decomposes the goal into a task plan: research the market, analyze financials, check regulations, analyze competitors, assess risks, synthesize a report.
- It assigns each task to a specialist agent — research, financial, legal, competitor, risk — each with its own tools (web search, financial data and modelling, regulation databases, market data).
- Specialists write intermediate results (reports, data, analysis) to a shared state / workspace: task artifacts, files, structured data, decisions and memory.
- A reviewer / critic agent checks for consistency, accuracy, missing information and policy issues.
- If more work is needed, the supervisor reassigns, retries or refines the plan; if not, it produces the final synthesis: a report with recommendations, citations and next steps.
Roles and Responsibilities
| Role | Responsibilities |
|---|---|
| Supervisor / manager | Understands the overall goal, decomposes it, assigns tasks, tracks progress, combines results, handles failures and replanning |
| Specialists | Focus on one domain or skill, use domain-specific tools and data, produce structured outputs; can invoke subagents |
| Reviewer / critic | Check correctness, consistency and completeness; find conflicts or gaps; enforce policies and requirements |
| Tool-using agents | Interact with external systems — search, databases, APIs, code execution |
| Human in the loop | Oversight, approval for sensitive actions, final validation |
Task Decomposition and Routing
The supervisor breaks the goal into subtasks — identify key areas, determine the data each needs, assign agents — then routes each to the best agent using one of four strategies: rule-based (by task type), LLM-based (the model decides dynamically), capability matching (agent skills and tools), and load balancing (who's available). Rule-based routing is cheap and predictable; LLM routing handles tasks nobody anticipated. Many systems use rules for the known cases and an LLM for the rest.
Common Multi-Agent Patterns
| Pattern | How it works | Good for |
|---|---|---|
| Supervisor–worker (most common) | One manager delegates to specialists and combines results | Complex, structured tasks |
| Peer-to-peer | Agents communicate directly | Brainstorming, research, creative work |
| Pipeline (sequential) | Each agent's output is the next one's input | Step-by-step workflows |
| Debate / critic | Agents propose, critique and refine | Quality, reducing individual bias |
| Hierarchical | Supervisor → sub-supervisors → workers | Large, complex problems |
| Swarm / dynamic routing | Tasks passed between agents dynamically | Flexible, changing requirements |
Research frameworks have explored most of these: AutoGen (conversable agents), MetaGPT (software-company roles with standardized operating procedures), CAMEL (role-playing agents), and multi-agent debate, where Du et al. found that several model instances debating over rounds improved factuality and reasoning on their benchmarks.
Does Cross-Checking Actually Work?
"Better quality through cross-checking" is the most-cited benefit, so we measured when it holds. Each simulated agent answers yes/no questions and is wrong 20% of the time. We combine several agents by majority vote, optionally add a reviewer that catches half of the remaining errors, and vary one thing: how often all agents share the same mistake — as happens when they're the same model with the same prompt and the same blind spots.
With independent errors, voting works well: five agents cut the error rate from 20% to 6%, and a reviewer halves it again. But when agents mostly make the same mistakes, five agents barely beat one (19% vs 20%), at five times the cost. Cross-checking reduces errors only to the extent that the checkers are different — different models, different prompts, different information, different methods. A reviewer that looks at the problem another way (running the code, checking a source) is worth more than more copies of the same agent.
Agent Communication and Shared State
Agents share information through structured messages and a common workspace. A message typically records who it's from and to, its type, content, artifacts, and metadata such as task ID and timestamp:
{ "from": "research_agent", "to": "financial_agent", "type": "result",
"content": "Market size about $2.1B, growing 14% a year...",
"artifacts": ["report.pdf"], "metadata": { "task_id": "T1", "timestamp": "..." } }The workspace holds the task plan and status, intermediate results, files and documents, structured data, agent decisions and notes, memory, and access-control permissions. It can live in memory (short tasks), a database (long-running tasks), a vector store (semantic search) or object storage (files). Three context-sharing models: shared context (everyone sees everything), private context (each agent keeps its own notes), and selective sharing (each agent gets only what it needs and is permitted to see).
We measured the difference on the market-entry workspace:
| Agent | Broadcast everything (words) | Selective sharing (words) |
|---|---|---|
| research | 664 | 0 |
| financial | 483 | 25 |
| legal | 696 | 25 |
| competitor | 678 | 0 |
| risk | 707 | 46 |
| reviewer | 807 | 807 |
| total | 4,035 | 903 |
Selective sharing — only the results an agent depends on, as short summaries, filtered by permission — cut the total context by about three quarters; only the reviewer, which genuinely needs everything, still gets everything. Permissions matter too: the financial model was marked private to the supervisor, reviewer and financial agent, and when the risk agent asked for it, it was withheld. Sharing less also has a cost: an agent that doesn't see a decision can contradict it, which is the core of Cognition's warning below.
Sequential vs Parallel Execution
Agents work in sequence when tasks depend on each other (research, then financial analysis of that market, then a legal check) and in parallel when they're independent (research, competitor and risk analysis at once), combining results in the shared workspace. The practical approach is to treat the plan as a dependency graph and start every task as soon as its inputs are ready. In our run the seven-agent market-entry workflow took 1.40 s run one after another and 0.75 s run dependency-aware in parallel — close to its 0.75 s critical path, the longest chain of dependent tasks, which no amount of parallelism can beat.
Failure Handling and Recovery
Agents fail, and the supervisor should handle it gracefully: retry on network or timeout errors; use alternate tools or data sources; reassign to another agent if one can't complete the task; fall back to a simpler approach (a partial result, manual review); escalate to a human for critical failures; and log errors and learn from them. In our run the legal agent timed out twice, the supervisor reassigned "check regulations" to a backup agent, which completed it — while per-agent token budgets were tracked throughout.
The most systematic study of how multi-agent systems fail, Cemri et al.'s MAST taxonomy, analysed over 1,600 execution traces across seven frameworks and grouped 14 failure modes into three categories: system design issues (poor role or task specification), inter-agent misalignment (agents ignoring each other, withholding information, going off-task), and task verification failures (stopping too early, weak or missing checks). They also note that multi-agent gains on popular benchmarks are often minimal — the architecture doesn't help by itself.
Cost and Resource Management
Multiple agents multiply costs. Anthropic reported that its multi-agent research system outperformed a single agent by 90.2% on an internal evaluation, but used about 15 times the tokens of a chat interaction. Controls that keep it affordable: use the right model for each agent (smaller models for simple roles), set token and cost limits per agent, limit the number of tool calls and retries, run independent agents in parallel to cut latency, monitor usage against quotas, and terminate idle or stuck agents.
Observability and Termination
Track each agent's actions, tool calls and results; log messages, decisions and artifacts; measure latency, cost and success rate; and build a visual trace of the workflow — multi-agent bugs are nearly impossible to find without one. The system is done when all subtasks are complete, the reviewer approves, required outputs exist, a confidence threshold is reached, the user is satisfied — or a budget or time limit is hit, in which case it should stop and report what it has.
When Not to Use Multiple Agents
Multi-agent designs are not a free upgrade. Cognition, which builds a coding agent, argued in 2025 against multi-agent systems for most uses: when subagents work from partial context they make conflicting decisions, and a single agent with good context engineering is more reliable. A later follow-up reported patterns that do work for them — several agents contributing intelligence while writes (the actions that change things) stay with a single agent. Anthropic's research system suits the opposite case: broad, parallelizable research where subagents can explore independently. A reasonable rule: start with one agent; add others where work is genuinely independent, where different tools or context are needed, or where an independent check adds real diversity — and keep one agent responsible for anything that changes the world.
Run the Experiments
One script, standard library only, runs in about three seconds.
"""Multi-agent systems, measured: when does a team of agents beat one agent, and what does it cost? 1. Cross-checking: majority vote and a reviewer reduce errors - but only if agents' errors are independent. 2. Scheduling: the market-entry workflow as a dependency graph, run sequentially vs in parallel. 3. Shared state: broadcasting everything vs selective sharing with access control - context size per agent. 4. Failure handling: an agent fails, the supervisor retries, then reassigns; token budgets per agent. Agents here are simulated (fixed error rates, sleep-based work), so the coordination logic is what is real. Standard library only. """ import concurrent.futures as cf, json, random, time rng = random.Random(0) # ---------------- 1. cross-checking and correlated errors ---------------- def trial(n_agents, p_err, rho, reviewer_catch): """Each agent answers a yes/no question. With probability rho all agents share one 'common mistake' (same model, same blind spot); otherwise each errs independently with probability p_err.""" if rng.random() < rho: wrong = [rng.random() < p_err] * n_agents else: wrong = [rng.random() < p_err for _ in range(n_agents)] final_wrong = sum(wrong) > n_agents / 2 # majority vote if final_wrong and rng.random() < reviewer_catch: final_wrong = False return final_wrong print("1) final error rate (single agent errs 20% of the time); 40,000 simulated questions each") print(" agents review independent errors errors 50% shared errors 90% shared relative cost") for n, rev in [(1, 0.0), (3, 0.0), (5, 0.0), (5, 0.5)]: row = [sum(trial(n, 0.2, rho, rev) for _ in range(40000)) / 40000 for rho in (0.0, 0.5, 0.9)] print(f" {n:>6} {'yes' if rev else 'no ':<5} {row[0]:>18.1%} {row[1]:>17.1%} {row[2]:>17.1%} {n + (1 if rev else 0):>9}x") # ---------------- 2. sequential vs parallel execution of a task graph ---------------- TASKS = { # name: (seconds of work, dependencies) - the "enter the Indian market?" workflow "research": (0.30, []), "financial": (0.25, ["research"]), "legal": (0.20, ["research"]), "competitor": (0.25, []), "risk": (0.20, ["competitor"]), "review": (0.10, ["financial", "legal", "risk"]), "report": (0.10, ["review"])} def work(name): time.sleep(TASKS[name][0]); return name t = time.perf_counter() for name in ["research", "financial", "legal", "competitor", "risk", "review", "report"]: work(name) t_seq = time.perf_counter() - t t = time.perf_counter(); done, running = set(), {} with cf.ThreadPoolExecutor(8) as ex: while len(done) < len(TASKS): for name, (_, deps) in TASKS.items(): # start every task whose inputs are ready if name not in done and name not in running and all(d in done for d in deps): running[name] = ex.submit(work, name) finished, _ = cf.wait(running.values(), return_when=cf.FIRST_COMPLETED) for f in finished: n = f.result(); done.add(n); del running[n] t_par = time.perf_counter() - t def longest(n): return TASKS[n][0] + max([longest(d) for d in TASKS[n][1]] or [0]) print(f"\n2) market-entry workflow, 7 agents: sequential {t_seq:.2f}s, dependency-aware parallel {t_par:.2f}s " f"({t_seq / t_par:.1f}x); critical path {longest('report'):.2f}s") # ---------------- 3. shared workspace: broadcast vs selective sharing ---------------- def ntok(obj): return len(json.dumps(obj).split()) workspace = [] # messages: from, to, type, content, artifacts, acl def post(frm, to, content, artifacts=(), acl=("all",)): workspace.append({"from": frm, "to": to, "type": "result", "content": content, "artifacts": list(artifacts), "metadata": {"task_id": f"T{len(workspace) + 1}"}, "acl": list(acl)}) post("research", "supervisor", "Market size about $2.1B, growing 14% a year; " + "detail " * 120, ["market.pdf"]) post("financial", "supervisor", "Break-even in 26 months at base case; " + "model rows " * 150, ["model.xlsx"], acl=("supervisor", "reviewer", "financial")) post("legal", "supervisor", "Needs local entity and data-residency setup; " + "clause " * 90, ["legal.md"]) post("competitor", "supervisor", "Three incumbents hold 70%; " + "profile " * 110, ["competitors.md"]) post("risk", "supervisor", "Top risk is regulatory delay; " + "scenario " * 80, ["risks.md"]) agents = ["research", "financial", "legal", "competitor", "risk", "reviewer"] NEEDS = {"research": [], "financial": ["research"], "legal": ["research"], "competitor": [], "risk": ["competitor", "legal", "financial"], "reviewer": ["research", "financial", "legal", "competitor", "risk"]} def visible(agent, msg): return "all" in msg["acl"] or agent in msg["acl"] broadcast = {a: sum(ntok(m) for m in workspace if m["from"] != a) for a in agents} selective = {a: sum(ntok(m) if a == "reviewer" else ntok({**m, "content": m["content"][:60]}) for m in workspace if m["from"] in NEEDS[a] and visible(a, m)) for a in agents} print("\n3) context each agent receives from the shared workspace (words)") print(" agent broadcast everything selective (only what it needs, summaries, ACL)") for a in agents: print(f" {a:<12} {broadcast[a]:>20,} {selective[a]:>12,}") print(f" total {sum(broadcast.values()):>20,} {sum(selective.values()):>12,}") denied = [a for a in agents if "financial" in NEEDS[a] and not visible(a, workspace[1])] print(f" access control: the financial model is private to supervisor, reviewer and financial; " f"requested but withheld from: {denied or 'nobody'}") # ---------------- 4. failure handling and budgets ---------------- print("\n4) failure handling") BUDGET = {"research": 4000, "legal": 3000, "legal_backup": 3000} spent = {k: 0 for k in BUDGET} def run_agent(agent, task, attempt): spent[agent] += 1200 # tokens used per attempt (simulated) if spent[agent] > BUDGET[agent]: raise RuntimeError(f"{agent} over token budget") if agent == "legal": raise TimeoutError("regulation database did not respond") return f"{task}: done by {agent}" def supervise(task, agent, backup, retries=1): for attempt in range(retries + 1): try: return run_agent(agent, task, attempt) except TimeoutError as e: print(f" {agent} attempt {attempt + 1} failed: {e}") print(f" reassigning '{task}' to {backup}") return run_agent(backup, task, 0) print(" ", supervise("check regulations", "legal", "legal_backup")) print(f" tokens used: " + ", ".join(f"{k} {v:,}/{BUDGET[k]:,}" for k, v in spent.items()))
Output (fixed random seed; timings vary slightly by machine):
1) final error rate (single agent errs 20% of the time); 40,000 simulated questions each
agents review independent errors errors 50% shared errors 90% shared relative cost
1 no 20.2% 20.1% 20.1% 1x
3 no 10.2% 15.3% 18.6% 3x
5 no 6.0% 13.1% 18.9% 5x
5 yes 2.9% 6.4% 9.4% 6x
2) market-entry workflow, 7 agents: sequential 1.40s, dependency-aware parallel 0.75s (1.9x); critical path 0.75s
3) context each agent receives from the shared workspace (words)
agent broadcast everything selective (only what it needs, summaries, ACL)
research 664 0
financial 483 25
legal 696 25
competitor 678 0
risk 707 46
reviewer 807 807
total 4,035 903
access control: the financial model is private to supervisor, reviewer and financial; requested but withheld from: ['risk']
4) failure handling
legal attempt 1 failed: regulation database did not respond
legal attempt 2 failed: regulation database did not respond
reassigning 'check regulations' to legal_backup
check regulations: done by legal_backup
tokens used: research 0/4,000, legal 2,400/3,000, legal_backup 1,200/3,000⚠️ Notes
- "Better quality through cross-checking" holds only when agents' errors are independent; our simulation shows five same-minded agents barely improve on one.
- "Improves robustness and reliability" is not automatic: the MAST study found multi-agent gains on benchmarks often minimal, and catalogued 14 ways these systems fail.
- Cost is listed only as something to manage; in published systems it's the main trade-off (about 15× the tokens of a chat, in Anthropic's case).