Homeโ€บBlogโ€บAgentic AI System Designs
Agentic AI System Designs ยท Harness teardown

Codex, Taken Apart

OpenAI's Codex dismantled into eight layers: an open-source local agent and a cloud agent that runs every task in its own container and hands back a diff. One task traced through both, the design choices, failure modes, and what building one takes.

FL
FrontierAGI Team
โš ๏ธ How to read this (as of 7 Nov 2026)Facts come from the Codex GitHub repository (read 7 Nov 2026). OpenAI's documentation could not be reached from our build environment, so sandbox, approval and network-policy details are marked โš ๏ธ, and the cloud architecture is inferred from documented behaviour. Build times, team sizes and costs are our estimates. Disclosure: Claude, made by Anthropic, helps write this site, and this series covers Anthropic, Google and OpenAI products.

1. Case file

MakerOpenAI
CategoryAgent harness
FormsOpen-source CLI + cloud tasks
Licence (CLI)Apache-2.0
Sign-inChatGPT plan or API key
InstructionsAGENTS.md

2. The whole system in one picture

๐Ÿ›๏ธ The whole system, layer by layer, with one task traced

โ” control (who calls whom)โ”… data (what comes back)โ–ญ solid border: runs on your machineโ”… dashed border: runs in the vendor's cloud
Press play to follow one task down through the layers and back, or tap any component.

Same eight-layer reference stack in every article of the series. Components and calls are drawn from the project's documentation and code; where internals are not public the drawing is inferred from documented behaviour. The monospace box shows what actually moves between layers at each step (simplified).

3. Layer by layer

L1๐Ÿ–ฅ๏ธ Interfaces ยท Where people (or other agents) talk to it

What it does
Ways in: terminal, editor, and a web app for delegating tasks.
How it is built here
An open-source CLI (Apache-2.0) and Codex Web at chatgpt.com/codex, plus IDE integration โš ๏ธ; sign in with a ChatGPT plan or API key.
Design choice
Two products sharing a name and a model: a local agent and a cloud agent.
Without it
Local-only means one task at a time on your laptop; cloud-only means no access to your local setup.

L2๐Ÿงญ Sessions & orchestration ยท Who owns a task: sessions, background tasks, subagents

What it does
Ownership of work: interactive sessions locally, delegated tasks in the cloud.
How it is built here
Cloud tasks run independently and in parallel; results come back for review.
Design choice
Delegation is the main mode in the cloud: hand off, keep working, review later.
Without it
Long tasks block the developer; parallel work is impossible.

L3๐Ÿ” Agent loop ยท How it decides the next step

What it does
Plan, act, observe, repeat, per task.
How it is built here
An agent loop calling shell and edit tools; one loop per cloud task.
Design choice
Many small independent loops rather than one long conversation.
Without it
Tasks interfere with each other through shared state.

๐Ÿ” How this loop compares with the other four, dial by dial, and its RSI level: Inside the Agent Loop

L4๐Ÿง  Context engine ยท What the model sees on each turn

What it does
What the model knows about your project.
How it is built here
AGENTS.md instructions โš ๏ธ and the repository itself, read through tools.
Design choice
A plain markdown convention any harness can read, instead of a proprietary format.
Without it
The agent guesses how to install and test, and fails in the container.

L5๐Ÿ› ๏ธ Tools ยท What it can do

What it does
Actions: run commands and change files.
How it is built here
Shell commands and patch-based edits; MCP for outside tools โš ๏ธ.
Design choice
The shell as the universal tool.
Without it
Every build system needs a dedicated integration.

L6๐Ÿ›ก๏ธ Policy & safety ยท What it may do

What it does
What may happen without asking.
How it is built here
Approval modes from read-only to full access locally โš ๏ธ; per-task network policy in the cloud โš ๏ธ.
Design choice
Policy is tied to the environment: strict where the agent runs on your machine, isolated where it runs in the cloud.
Without it
A local agent with full access can damage your machine; a cloud agent with open network can leak code.

L7๐Ÿ“ฆ Execution ยท Where actions actually run

What it does
Where actions run.
How it is built here
Locally inside operating-system sandbox restrictions โš ๏ธ; in the cloud in a container per task.
Design choice
Isolation everywhere is the defining choice: the agent never shares an environment with another task.
Without it
One bad command affects everything else running.

L8๐Ÿงฌ Model & telemetry ยท Which model, and what gets logged

What it does
The model, and the record of what happened.
How it is built here
OpenAI models; logs and diffs for every cloud task.
Design choice
Every task ends in reviewable evidence (log + diff), not just a message.
Without it
You merge changes you cannot audit.

4. Connecting the dots

๐Ÿงฌ Design DNA: the three decisions that define it

Delegate, don't chathand off tasks and review results
Isolate every taskcontainer in the cloud, sandbox locally
Open-source local agentApache-2.0 CLI anyone can inspect

๐Ÿ’ธ Where the tokens go in the traced task (our estimate)

Setup and environment logs20% ยท ~60k
Instructions and repo context re-sent each turn30% ยท ~90k
Command output (tests, installs)25% ยท ~75k
Model output (plans, patches)15% ยท ~45k
Retries10% ยท ~30k

An illustrative split for a ~300k-token task: about $1.20 at an assumed blended $4 per million tokens. Real numbers vary widely with the repo, the model and caching; the shape (context re-sent every turn dominates) is the point.

5. How it evolved

Apr 2025 Codex CLI open-sourced โš ๏ธ.
May 2025 Codex cloud agent launched as a research preview โš ๏ธ.
2025 CLI rewritten and expanded; IDE integration โš ๏ธ.
Nov 2026 README: CLI runs locally; Codex Web at chatgpt.com/codex (read 7 Nov 2026).

6. Where it breaks, and what it does about it

FailureLayerWhat happensWhat this system does about it
Environment mismatchL7The container cannot install or build the project.AGENTS.md and setup scripts describe how to build โš ๏ธ.
Network neededL6A task needs a package or an API the policy blocks.Adjust the task's network policy deliberately โš ๏ธ.
Stale baseL2A long task finishes against an old branch.Review the diff; rebase before merging.
Over-broad changesL5Patches touch more than needed.Conventions in AGENTS.md; review before PR.
Prompt injectionL5Repository content tries to instruct the agent.Isolation limits what a hijacked task can reach.
Too much access locallyL6Full-access mode on a developer laptop.Prefer the default sandboxed modes โš ๏ธ.

7. Security and threat model

๐Ÿงฑ Trust boundaries

Leaves your machinecloud tasks: your repository is cloned into OpenAI's environment; CLI: the context sent to the model
Untrusted inputsrepository files, dependencies installed during setup, web content
Powerful actionsshell commands, package installs, network calls
Controlscontainer per task, network policy โš ๏ธ, approval modes and OS sandbox locally โš ๏ธ, diff review

8. Standards it shares, and benchmarks

9. What people use it for

IndividualBackground fixesHand off a bug and keep working.
IndividualLocal pair programmingThe CLI in your terminal with approvals.
TeamParallel backlogSeveral small tasks delegated at once, reviewed as diffs.
TeamTest writingAdd missing tests in an isolated environment.
TeamDependency upgradesUpgrade, build, run tests, return a diff.
EnterpriseIsolated executionUntrusted repos handled in containers, not laptops.
EnterpriseCustom harnessFork or study the open-source CLI.
Beyond codeScripting tasksData scripts and docs (typical uses).

10. How to build one from scratch

mini_codex.py ยท Python 3, standard library only; scripted stub model
"""mini_codex.py โ€” a toy harness in the style of Codex (illustrative).

Delegation and isolation: each task gets its own copy of the repository (standing
in for a cloud container), reads AGENTS.md, works under an approval mode, and
hands back a diff for review. Several tasks run independently. The "model" is a
scripted stub; standard library only.
"""
import copy, difflib
REPO = {"AGENTS.md": "test: python test_dates.py\nconvention: minimal diffs\n",
        "dates.py": "def parse(s):\n    d, m, y = s.split('/')\n    return (int(y), int(m), int(d))\n",
        "README.md": "Date utils\n"}
MODES = {"read-only": set(), "auto": {"edit", "test"}, "full": {"edit", "test", "network"}}
PLANS = {"fix-date-test": [("test", None), ("edit", ("dates.py", "    d, m, y = s.split('/')\n", "    y, m, d = s.split('-')\n")), ("test", None)],
         "update-readme": [("edit", ("README.md", "Date utils\n", "Date utils (ISO 8601 input)\n"))],
         "fetch-deps": [("network", "pip download requests")]}

def run_task(name, mode):
    box = copy.deepcopy(REPO)                       # isolated container per task
    log = [f"read AGENTS.md: {box['AGENTS.md'].splitlines()[0]}"]
    for act, arg in PLANS[name]:
        if act not in MODES[mode]: log.append(f"{act}: blocked by '{mode}' mode"); continue
        if act == "edit": f, old, new = arg; box[f] = box[f].replace(old, new); log.append(f"edit {f}")
        elif act == "test": ok = "y, m, d" in box["dates.py"]; log.append("test: " + ("pass" if ok else "fail"))
    diff = [l for f in REPO for l in difflib.unified_diff(REPO[f].splitlines(), box[f].splitlines(), f"a/{f}", f"b/{f}", lineterm="", n=0)]
    return log, diff

def main():
    for name, mode in (("fix-date-test", "auto"), ("update-readme", "auto"), ("fetch-deps", "auto")):
        log, diff = run_task(name, mode)
        print(f"task {name} [{mode}]: " + "; ".join(log))
        print("  diff for review: " + (" | ".join(l for l in diff if l[:1] in "+-" and l[:3] not in ("+++", "---")) or "(no changes)"))
    print(f"original repo untouched: {'d, m, y' in REPO['dates.py']}")

if __name__ == "__main__":
    main()
$ python3 mini_codex.py task fix-date-test [auto]: read AGENTS.md: test: python test_dates.py; test: fail; edit dates.py; test: pass diff for review: - d, m, y = s.split('/') | + y, m, d = s.split('-') task update-readme [auto]: read AGENTS.md: test: python test_dates.py; edit README.md diff for review: -Date utils | +Date utils (ISO 8601 input) task fetch-deps [auto]: read AGENTS.md: test: python test_dates.py; network: blocked by 'auto' mode diff for review: (no changes) original repo untouched: True
PhaseWhat you buildTime*Team*Cost*
0 ยท PrototypeLocal loop with shell and patch tools1โ€“2 weeks1$15โ€“40k
1 ยท Safe CLIApproval modes, OS sandboxing, AGENTS.md2โ€“3 months2โ€“4$200โ€“500k
2 ยท Cloud tasksContainer per task, repo cloning, setup scripts, logs3โ€“6 months5โ€“10$1โ€“2.5M
3 ยท Parallel + reviewTask queue, diff review UI, PR creation, network policies3โ€“4 months6โ€“12$1โ€“2M
4 ยท ProductAccounts, billing, IDE, scaling to many containers, security review6โ€“12 months20โ€“50+$5โ€“15M+ a year (incl. compute)

*Our estimates, not vendor figures: fully loaded engineer cost assumed at $250k a year (US-style salaries; lower elsewhere), plus model API and infrastructure. Treat as orders of magnitude.

๐Ÿงฎ What would running your own cost a month?

Our model: model cost = tasks ร— tokens ร— price ร— 30 days; people = engineers ร— $250k รท 12; infrastructure (sandboxes, storage, logs) assumed at 15% of model cost. A blended price mixes cached input, fresh input and output; check your provider's current prices.

11. When to choose it

โœ… Choose it when

  • Many well-defined tasks to delegate in parallel.
  • You want an open-source local agent.
  • You prefer reviewing diffs to chatting.

โš ๏ธ Think twice when

  • Tasks need your local environment and secrets: cloud containers will not have them.
  • You want an agent that learns across sessions.

12. How it compares, layer by layer

HarnessL1InterfacesL2Sessions & orchestrationL3Agent loopL4Context engineL5ToolsL6Policy & safetyL7ExecutionL8Model & telemetry
Claude CodeTerminal, IDE, desktop, web, Slack; Agent SDKSessions with resume; subagents; cloud sessions on the webOne general think โ†’ act โ†’ observe loopCLAUDE.md, auto memory, compactionBuilt-in file/shell/search tools, MCP, skillsPermission rules and modes, hooksYour machine (optional sandboxing โš ๏ธ); cloud for web sessionsClaude models; transcripts, cost and usage telemetry
CodexTerminal, IDE, web (chatgpt.com/codex)Parallel cloud tasks; local sessionsAgent loop per taskAGENTS.md, repo contextShell, file edits, MCPApproval modes โš ๏ธOS sandbox locally โš ๏ธ; container per cloud taskOpenAI models; task logs and diffs
Gemini CLI / JulesTerminal (CLI); browser + GitHub (Jules)Interactive session vs asynchronous taskAgent loop (CLI); plan-then-execute (Jules โš ๏ธ)GEMINI.md context filesBuilt-in tools, MCPConfirmations in the CLI; plan approval in Jules โš ๏ธYour machine (CLI); cloud VM (Jules โš ๏ธ)Gemini models (CLI "not optimized for Gemini 4+")
Deep AgentsYour own app (library)LangGraph threads; sub-agentsPlanning loop on LangGraphFilesystem as memory; offloadingYour tools, MCP, skillsHuman-in-the-loop interruptsPluggable backends: local, sandboxed, remoteAny chat model; LangGraph state and tracing
Hermes AgentTerminal + Telegram, Discord, Slack, WhatsApp, Signal, emailGateway, subagents, cron schedulerAgent loop (details on its docs site โš ๏ธ)Curated memory, FTS5 session search, user modelTools, self-written skills (agentskills.io), Python RPC scriptsDepends on backend and configuration โš ๏ธSix backends: local, Docker, SSH, Modal, Daytona, Vercel SandboxModel you configure โš ๏ธ

One line per layer; our reading of each project's documentation and code as of 7 Nov 2026.

HarnessRuns whereOpen sourceSubagentsPersistent memorySkillsMCPSandboxHuman approvalAsync / cloud tasks
Claude CodeTerminal, IDE, desktop, webโ€”โœ“โœ“โœ“โœ“โ—โœ“โœ“
CodexTerminal, IDE, cloudโœ“ (CLI)โ—โ—โ—โœ“โœ“โœ“โœ“
Gemini CLI / JulesTerminal (CLI), cloud VM (Jules)โœ“ (CLI)โ—โ—โ—โœ“โ—โœ“โœ“
Deep AgentsYour Python / JS appโœ“โœ“โ—โœ“โœ“โ—โœ“โ€”
Hermes AgentTerminal + chat appsโœ“โœ“โœ“โœ“โ—โœ“โ—โœ“

โœ“ built in ยท โ— partial or via add-ons ยท โ€” not a feature. Our reading of each project's documentation as of 7 Nov 2026; check before relying on it.

13. The series

Agentic AI System Designs: four kinds of agent system, one teardown each

Start here๐Ÿงญ Four ways to build an agentHarness๐Ÿงฐ Claude CodeHarness๐Ÿงฐ Codex (this article)Harness๐Ÿงฐ Gemini CLI, Antigravity & JulesHarness๐Ÿงฐ Deep AgentsHarness๐Ÿงฐ Hermes AgentFramework๐Ÿงฑ OpenAI Agents SDK ยท comingFramework๐Ÿงฑ Google ADK ยท comingFramework๐Ÿงฑ LangChain / LangGraph ยท comingFramework๐Ÿงฑ CrewAI ยท comingFramework๐Ÿงฑ Claude Agent SDK & Microsoft Agent Framework ยท comingRuntimeโš™๏ธ Amazon Bedrock AgentCore ยท comingRuntimeโš™๏ธ LangSmith Deployment ยท comingRuntimeโš™๏ธ Temporal ยท comingRuntimeโš™๏ธ Vertex AI Agent Engine & Azure AI Foundry ยท comingBuilder๐Ÿงฉ n8n ยท comingBuilder๐Ÿงฉ Zapier Agents ยท comingBuilder๐Ÿงฉ Microsoft Copilot Studio ยท comingBuilder๐Ÿงฉ Make ยท coming

All of them on one page: Agentic Systems at a Glance.

๐Ÿ“– Glossary: 15 terms used across the series
Agent loop
The repeat cycle: the model picks an action, the harness runs it, the result goes back to the model.
Harness
Everything around the model that turns it into a working agent: loop, tools, context, policy, execution.
Context window
The text the model can see at once. Everything the agent knows on a turn has to fit in it.
Context compaction
Summarising older conversation so a long task keeps fitting in the context window.
Tool schema
A machine-readable description of a tool (name, inputs) that tells the model how to call it.
MCP
Model Context Protocol: an open standard for connecting agents to outside tools and data.
Subagent
A second agent with its own context, used for a side task; only its summary comes back.
Skill
A packaged set of instructions (and sometimes scripts) the agent loads when relevant.
Hook
Your own command that runs automatically before or after an agent action.
Sandbox
A restricted place to run commands so mistakes or attacks cannot reach the rest of the system.
Approval mode
The rule for what the agent may do without asking you first.
Prompt injection
Instructions hidden in data the agent reads (a file, a web page) that try to hijack it.
Trace
The full record of a run: every model call, tool call and result.
AGENTS.md / CLAUDE.md / GEMINI.md
Plain markdown files of project instructions that harnesses read at the start of a session.
Asynchronous agent
An agent you hand a task to and leave; it returns a result (often a pull request) later.

14. Sources