The Harness Market Landscape: Who Builds the Infrastructure Around Frontier Models
Between the model API and the product sits a fast-growing market: agent frameworks, MCP servers and registries, retrieval and memory, browsers and sandboxes, durable workflows, observability, gateways, guardrails and managed agent platforms. This visual deep dive maps each layer as of September 2026 โ with verified 2026 releases, funding and acquisitions, an interactive stack builder, a consolidation timeline and a runnable availability model.
01 The Overall Harness Stack Map
A request from a user or app passes through an agent runtime, context and memory, tools and sandboxes, durable workflows and observability, reaching external systems โ with model calls going out to frontier and open model providers. Click a layer to see who competes there:
02 Agent Runtimes and Orchestration Frameworks
Frameworks give developers the agent loop, tool calling, handoffs, state and human-in-the-loop hooks. In 2025โ26 the field matured: major frameworks reached 1.0 with stable APIs, and most now ship durable state, MCP support and an opinionated "harness" on top.
| Framework | From | Key capabilities | 2025โ26 milestone |
|---|---|---|---|
| OpenAI Agents SDK | OpenAI | Agents, tools, handoffs, guardrails, tracing, sessions; Python and TypeScript | Sandbox agents and a model-native harness (April 2026) |
| LangGraph / LangChain | LangChain | Stateful graph-based orchestration, durable state, streaming, human-in-the-loop | LangChain and LangGraph 1.0 (October 2025) |
| Deep Agents | LangChain | "Batteries-included agent harness": planning, file-system memory, subagents, on the LangGraph runtime | Standalone harness library (2026) |
| CrewAI | CrewAI | Role-based multi-agent Crews plus event-driven Flows | Large developer community; enterprise platform |
| Microsoft Agent Framework | Microsoft | Successor to AutoGen and Semantic Kernel; native MCP and A2A; .NET and Python | 1.0 GA on 3 April 2026, MIT licence, long-term support |
| Google ADK | Agent Development Kit: build, evaluate and deploy agents; model-agnostic, Gemini-optimized | ADK 2.0; Kotlin 1.0 with Android support (September 2026) | |
| LlamaIndex Workflows | LlamaIndex | Event-driven workflows with strong retrieval and data integration | Agentic document workflows |
| Mastra | Mastra | TypeScript agents, workflows, memory and evals | Popular in the TypeScript ecosystem |
03 The MCP Ecosystem and Tool Connectivity
The Model Context Protocol has become the standard way to connect agents to tools and data: an agent's MCP client talks to MCP servers that expose tools, resources and prompts. The ecosystem now spans file systems, databases, SaaS apps, developer tools and internal systems โ and the 2026-07-28 specification made MCP stateless, cacheable and better secured (details in Part 2).
04 Context, RAG and Retrieval Infrastructure
Grounding agents in real data means ingesting, parsing, chunking, embedding, indexing, retrieving and re-ranking. Each step has its own vendors:
05 The Memory and State Layer
- Mem0 โ memory layer that extracts and retrieves user and agent memories
- Zep โ temporal knowledge-graph memory
- Letta โ from the MemGPT research; stateful agents with self-managed memory
- Built-in memory in model platforms (OpenAI, Anthropic's memory tool)
- PostgreSQL (with pgvector) โ the default for state and increasingly for vectors
- Redis โ fast working state and caches
- MongoDB โ documents plus vector search
- Object storage (S3, GCS, Azure Blob) โ artifacts and checkpoints
- Working (short-term) ยท episodic (past experiences)
- Semantic (knowledge, facts)
- Artifacts (files, code, outputs)
- Checkpoints (state snapshots)
06 Browser and Computer-Use Infrastructure
Agents that use the web need real browsers โ at scale, isolated, observable and hard to block. Two layers compete: automation libraries (Playwright, Puppeteer, and agent-oriented ones such as browser-use and Stagehand) and hosted browser infrastructure (Browserbase and others) that runs thousands of isolated sessions with recordings, logs and anti-bot handling. Computer-use models from the frontier labs drive full desktops through screenshots and actions.
07 Sandboxes and Code Execution
- Cloud sandboxes for AI agents: secure, isolated execution for code, shell and browser, built on Firecracker microVMs
- Serverless GPU and CPU compute with sandboxes for running agent-generated code
- Docker, Kubernetes, gVisor, Firecracker โ the do-it-yourself path
- Built into platforms: OpenAI sandbox agents, Anthropic's code execution tool
08 Durable Workflow and Long-Running Execution
Durable execution โ the ability to resume a long run after a crash without redoing or duplicating work (see Part 2) โ became one of the hottest infrastructure categories of 2026.
- Durable workflows, retries, state management; widely adopted
- September 2026: raised $550M at a $12.55B valuation, more than double its $5B valuation in February 2026
- Reported annualized revenue above $250M, growing over 200% a year; customers include OpenAI, NVIDIA, Netflix and Snap
- Developer-friendly long-running tasks, queues, retries, event-driven workflows and scheduled jobs
- Restate, DBOS, Airflow, Prefect, Dagster, BullMQ
- Built-in durability in LangGraph and the OpenAI Agents SDK
09 Observability, Tracing and Evaluation
Tracing, datasets, evals and production monitoring for agents is a crowded category โ and the first to consolidate:
| Tool | Focus | Status (Sep 2026) |
|---|---|---|
| LangSmith | Tracing, datasets, evals, agent deployment | LangChain's commercial platform |
| Braintrust | Evaluation datasets, experiments, production monitoring | Independent |
| Arize Phoenix / Arize AX | Open-source tracing and evals; enterprise platform | Independent |
| Langfuse | Open-source traces, metrics, prompt management | Acquired by ClickHouse (January 2026); stays open source |
| W&B Weave | Traces and evals from Weights & Biases | W&B is part of CoreWeave (acquired 2025) |
| Comet Opik | Open-source LLM and agent evaluation | Independent |
| MLflow | Experiment tracking with GenAI tracing and evals | Open source (Databricks) |
| Helicone | Proxy-based LLM observability | Acquired by Mintlify (March 2026); standalone product in maintenance mode |
10 Model Gateways, Routing and Cost Control
Gateways put one API in front of many model providers, adding routing, fallbacks, rate limits, budgets, caching and logging. Options range from open-source proxies (LiteLLM) and gateway products (Portkey) to aggregators (OpenRouter) and cloud gateways (Vercel AI Gateway, Azure AI Foundry, AWS Bedrock).
11 Guardrails, Security and Governance
12 Cloud and Managed Agent Platforms
- Agents API (managed) and Agents SDK (self-hosted), sandbox agents, tracing
- Amazon Bedrock and Bedrock AgentCore: runtime, memory, gateway, identity and observability for agents built with any framework
- Azure AI Foundry Agent Service and tooling; Microsoft Agent Framework
- Vertex AI Agent Engine and ADK; Gemini Enterprise
- Databricks, Snowflake, Salesforce (Agentforce) and others with agent and AI-app platforms next to customer data
13 Build vs Buy
At every layer you can self-host open source, use a managed service, or take your cloud's native offering. Build your stack and see what you've signed up for:
And what the composed stack means for uptime โ every independent service in the request path multiplies in:
"""Every vendor you add is another thing that can be down: availability of a composed agent stack.
An agent request touches several independent services in series (all must work), so availabilities
multiply. Redundancy (a gateway failing over across model providers, a second region) turns a series
link into a parallel one: it fails only if every copy fails. Availabilities below are illustrative
monthly figures you should replace with your vendors' SLAs and your own measurements.
Standard library only.
"""
from math import prod
MIN_PER_MONTH = 30 * 24 * 60
def downtime(a):
return (1 - a) * MIN_PER_MONTH
def parallel(*a):
return 1 - prod(1 - x for x in a)
layers = { # layer: availability (illustrative)
"API gateway / auth": 0.9995,
"agent runtime": 0.9990,
"model provider": 0.9950,
"vector DB (RAG)": 0.9990,
"memory service": 0.9990,
"MCP / tool servers": 0.9980,
"sandbox provider": 0.9990,
"durable workflow engine": 0.9995,
}
def report(name, parts):
a = prod(parts.values())
print(f" {name:<46}{a:>9.3%}{downtime(a):>12,.0f} min/month")
print("Availability of one agent request path (all layers in series)\n")
print(f" {'configuration':<46}{'avail.':>9}{'downtime':>22}")
report("single provider, every layer required", layers)
two_models = dict(layers, **{"model provider": parallel(0.995, 0.995)})
report("+ gateway failover to a 2nd model provider", two_models)
degrade = dict(two_models, **{"vector DB (RAG)": 1.0, "memory service": 1.0})
report("+ degrade gracefully if RAG or memory is down", degrade)
fewer = {k: v for k, v in layers.items() if k not in ("memory service", "sandbox provider")}
report("fewer vendors (no separate memory, no sandbox)", fewer)
weakest = min(layers, key=layers.get)
print(f"\nWeakest link: {weakest} ({layers[weakest]:.2%} alone = {downtime(layers[weakest]):,.0f} min/month).")
print(f"Eight layers at 99.9% each would give {0.999 ** 8:.2%} end to end.")Output:
Availability of one agent request path (all layers in series) configuration avail. downtime single provider, every layer required 98.806% 516 min/month + gateway failover to a 2nd model provider 99.300% 303 min/month + degrade gracefully if RAG or memory is down 99.498% 217 min/month fewer vendors (no separate memory, no sandbox) 99.003% 431 min/month Weakest link: model provider (99.50% alone = 216 min/month). Eight layers at 99.9% each would give 99.20% end to end.
Eight dependable-looking services add up to more than eight hours of monthly downtime in the request path. The biggest wins come from redundancy on the weakest link โ usually the model provider โ and from designing the agent to degrade gracefully rather than fail when an optional layer is down.
14 Market Structure
15 Consolidation, 2025โ2026
Money and acquisitions show where the market thinks value lies โ and which categories are becoming features of bigger platforms:
16 Commoditizing vs Differentiating, Buyer Personas and Key Shifts
- Builds AI features into products
- Needs speed, simplicity, good docs
- Often uses managed services
- Builds internal agent platforms
- Needs security, governance, integration
- Combines managed and self-hosted
- Moves fast, iterates on product
- Needs flexible, cost-effective stack
- Open source plus managed services
- Explores new ideas and capabilities
- Needs flexibility and latest tools
- Open source and experimental tools
End-to-end: selected examples at each layer
Next: Harness Case Studies โ OpenAI Codex, Claude Code auto mode and Gemini Deep Research taken apart.