Home โ€บ Blog โ€บ Agent runtimes
Inside the ML Layer ยท Part 8 ยท Agents

The Top of the Stack: Agent Runtimes, Tool Protocols and the Software That Lets Models Act

A model on its own only produces text. An agent runtime wraps it in a loop: decide, call a tool, read the result, decide again, until the task is done. Around that loop sit protocols that connect models to tools (MCP) and to other agents (A2A), frameworks that structure the work, sandboxes that keep code execution safe, and tracing that shows what happened. This final part of the series explains that top layer, and how agents feed back into every layer below.

FL
FrontierAGI Team
โš ๏ธ How this was researched (as of 27 Oct 2026) Sources are the ReAct and Toolformer papers, the MCP specification, Linux Foundation press releases (A2A, Agentic AI Foundation), documentation for the OpenAI Agents SDK, Claude Agent SDK, Google ADK and LangGraph, Firecracker's documentation and OpenTelemetry's GenAI conventions. Disclosure: Claude, made by Anthropic, helps write this site. MCP and the Claude Agent SDK are Anthropic projects; we describe them alongside their competitors from public documentation and try not to favour them. The cost calculator and sandbox chart are our own illustrations. See Reading the numbers.

1. Case file

๐Ÿค–
The agent layer, October 2026Layer 1 of 8 in the ML layer map: the top
Tool protocolMCP: 10,000+ servers; Agentic AI Foundation since Dec 2025
Agent-to-agent protocolA2A: Google โ†’ Linux Foundation, Jun 2025
Lab SDKsOpenAI Agents SDK ยท Claude Agent SDK ยท Google ADK
FrameworksLangGraph ยท Microsoft Agent Framework ยท CrewAI ยท LlamaIndex
SandboxesContainers, gVisor, Firecracker microVMs (~125 ms to guest init)
Tracing standardOpenTelemetry GenAI conventions, still in development

2. Timeline

From ReAct to foundations, 2022 โ†’ 2026

Tap a milestone. Colour: research, frameworks, lab SDKs and APIs, protocols and foundations.

Tap a milestone.

3. The agent loop

Almost every agent, however elaborate, is the loop the ReAct paper described in 2022: the model reasons, chooses an action, the runtime executes it and returns an observation, and the model continues. The runtime's job is everything the model cannot do itself: run tools, keep state, stop runaway loops, retry failures, ask a human when needed, and keep the context window from overflowing by summarising or dropping old material ("compaction").

Watch an agent fix a bug

A simplified coding task. Step through; the gauge shows the context window filling, then being compacted. Illustration.

Context used

4. Tool calling, from model to engine

Tool use starts in training: models are post-trained (Part 5) to emit structured calls, a tool name plus JSON arguments matching a schema. The serving engine (Part 7) can enforce the schema with constrained decoding, masking tokens that would break the JSON, which SGLang and vLLM support. Models can now request several calls at once ("parallel tool calls"), and the runtime runs them concurrently. Because every tool definition sits in the prompt, large tool sets cost tokens on every call; runtimes increasingly load tools on demand.

5. MCP and A2A

MCP: Model Context ProtocolReleased by Anthropic in November 2024. A client (in the AI app) talks to servers that expose tools, resources (data) and prompts, using JSON-RPC over stdio for local servers or streamable HTTP for remote ones. Adopted by Claude, ChatGPT, Cursor and others; over 10,000 servers by late 2025. Donated to the Linux Foundation's new Agentic AI Foundation on 9 Dec 2025, alongside Block's goose and OpenAI's AGENTS.md.
A2A: Agent2AgentAnnounced by Google in April 2025 for agents to discover each other (via "agent cards"), delegate tasks and exchange results, whoever built them. Moved to the Linux Foundation in June 2025 with backers including AWS, Cisco, Microsoft, Salesforce, SAP and ServiceNow. Complements MCP: MCP connects an agent to tools; A2A connects agents to agents.

Why a protocol: N ร— M versus N + M

Without a shared protocol, every AI app needs a custom integration for every tool. With one, each side implements the protocol once.

An MCP exchange

Simplified JSON-RPC messages between a client and a server. Step through.


      
โš ๏ธ The security catchProtocols make it easy to connect tools, including untrusted ones. Prompt injection (instructions hidden in a web page, email or file the agent reads) and tool poisoning (malicious tool descriptions) can steer an agent into leaking data or taking actions its user never asked for. Defences are partly in the runtime: least-privilege permissions, human approval for sensitive actions, isolating untrusted content, and allow-listing servers.

6. Runtime patterns

Four ways to structure an agent

Tap a pattern.

7. The frameworks

Two camps have formed. The lab SDKs (OpenAI's, Anthropic's, Google's) are thin and opinionated, built around their own models and hosted tools, though most accept other models. The independent frameworks (LangGraph, CrewAI, LlamaIndex, Microsoft's) are model-agnostic and add orchestration, state and integrations. All have converged on the same primitives: tools, handoffs or sub-agents, guardrails, memory, MCP support and tracing.

8. Feature matrix

Our reading of documentation, October 2026. โ— built in ยท โ— partial or via integration ยท โ€” not a focus. Tap a cell.

Tap a cell.

9. Sandboxes and isolation

Agents that write and run code, browse or control computers must be isolated from the systems they run on. Options trade isolation strength against start-up time and overhead. AWS built Firecracker (paper at NSDI 2020) for Lambda; its specification targets about 125 ms from start to the guest's init process with under 5 MiB of memory overhead, and it underlies several agent-sandbox services.

Isolation vs start-up

A qualitative illustration. Positions are our judgement, not measurements. Tap a point.

Tap a point.

10. Agents at AI-factory scale

Agents change the economics of everything below them. One task can mean dozens of model calls, each re-sending a growing context, so prefix caching (Part 7) becomes essential: most of each call is identical to the last. Long-running agents hold KV cache for minutes, shifting serving toward memory. And every agent run is potential training data: agentic RL (Part 5) trains models on exactly these loops, judged by agent evaluations (Part 6).

What does one agent task cost?

Our own model: context grows linearly each step; cached input is billed at a discount. Prices are inputs, not quotes.

11. Observability and safety

12. Choosing a runtime

Which runtime fits?

An editorial guide.

VerdictThe agent layer is the youngest and fastest-moving in the stack, but it has settled faster than expected: one loop, one tool protocol (MCP) now under neutral governance, one agent-to-agent protocol (A2A), and frameworks converging on the same primitives. The hard problems have moved to reliability, security and cost: keeping agents on task over hundreds of steps, defending against injected instructions, and making long contexts cheap. Those problems are solved as much in the layers below (training, evaluation, serving) as in the runtime itself.

13. The whole stack

Inside the ML Layer: the series map

Eight parts, from the top of the stack to the GPU. Tap to open a part.

An agent answering a question touches every layer in this series: its runtime calls a model served by an inference engine using kernels on GPUs, a model that was post-trained with RL, checked by evaluation harnesses, trained with parallelism libraries on a framework. Part 1's trace followed that path from the bottom; this series has now opened each box. The companion series, Inside the Data Layer, follows the data that feeds them.

14. Reading the numbers

Primary
Agentic AI Foundation formed 9 Dec 2025 with MCP, goose and AGENTS.md; 10,000+ MCP servers; platinum members include AWS, Anthropic, Block, Bloomberg, Cloudflare, Google, Microsoft, OpenAI (Linux Foundation). A2A Linux Foundation project announced 23 Jun 2025 (Linux Foundation). Claude Code SDK renamed Claude Agent SDK, 29 Sep 2025 (Anthropic). Firecracker ~125 ms to guest init, <5 MiB overhead (Firecracker specification).
Secondary โš ๏ธ
Microsoft Agent Framework (Oct 2025, unifying AutoGen and Semantic Kernel) per third-party coverage. OpenTelemetry GenAI conventions status per third-party write-ups.
Our illustrations
Agent loop and context gauge, integration counts, MCP messages (simplified), pattern diagrams, sandbox positions and the cost calculator.