Harness Case Studies: OpenAI Codex, Claude Code Auto Mode and Gemini Deep Research
Three real agent products, three harness designs. Codex runs many coding agents in parallel in isolated worktrees and cloud sandboxes. Claude Code's auto mode replaces most permission prompts with a sandbox and a two-stage safety classifier โ and publishes how often that classifier misses. Gemini Deep Research turns one question into a long, iterative, cited investigation. This visual deep dive takes each apart from public documentation, with interactive diagrams and a runnable simulation of approval fatigue using the published numbers.
01 Case Study 1: OpenAI Codex โ a Coding-Agent Harness
Theme: end-to-end software engineering with multiple agents, worktrees and background tasks.
Codex is OpenAI's AI coding agent for end-to-end engineering work โ features, refactors, migrations, tests, docs. It runs in the CLI, IDE extensions and cloud, and since February 2026 in a desktop Codex app built to manage many agents over long periods (Windows followed in March 2026). The same month OpenAI shipped GPT-5.3-Codex and GPT-5.3-Codex-Spark as its underlying models.
Each agent gets its own git worktree โ a separate working copy of the same repository โ so parallel agents never edit the same files. Inside each, the agent runs a loop:
What keeps it safe is a two-dimensional control: a sandbox mode (what the agent can touch) and an approval policy (when it must ask). Try combinations:
AGENTS.md file; background automations run scheduled jobs โ issue triage, alert monitoring, CI/CD tasks โ in their own worktrees.- Worktrees and cloud environments
- Issue triage, alerts, CI/CD tasks
- AGENTS.md, repo conventions
- Diffs, logs, test results
- Progress, status, run history
02 Case Study 2: Claude Code Auto Mode โ Safer Autonomous Tool Use
Theme: reduce approval fatigue while preserving safety through layered safeguards.
Anthropic found that Claude Code users approved 93% of permission prompts โ and that the more prompts people see, the less attention each gets. Two harness features attack this. Sandboxing lets Claude work freely inside OS-level file-system and network boundaries (Seatbelt on macOS, bubblewrap on Linux), which cut permission prompts by 84% in internal use. Auto mode goes further: model-based classifiers approve or block actions instead of a human, with two layers of defence โ one for what Claude reads, one for what Claude does. Pick an action:
The transcript classifier (running on Claude Sonnet 4.6) works in two stages: a fast stage 1 told to "err on the side of blocking" that outputs a single yes/no, and a stage 2 that reasons carefully about anything stage 1 flagged. Crucially it sees only the user's messages and the executable action โ assistant prose and tool outputs are stripped, so neither the agent nor injected content can argue with it. Anthropic published how well it works:
03 Approval Fatigue, Simulated
How do manual prompts, a sandbox, auto mode and their combination compare over a week of agent work? This simulation uses Anthropic's published rates plus stated assumptions about how often actions are dangerous and how human attention decays:
"""Approval fatigue: manual prompts vs a sandbox vs a classifier (auto mode), on the same agent session.
Published inputs (Anthropic engineering posts):
- users approve 93% of permission prompts
- sandboxing cut permission prompts by 84% in internal use
- auto-mode classifier, full two-stage pipeline: 0.4% false positives, 17% false negatives on
real overeager actions
Assumptions (ours, adjustable): 2,000 actions per week of sessions; 1% of actions are genuinely
dangerous/overeager; a human reviewer's chance of catching a dangerous prompt starts at 90% and
decays as prompts pile up in a session (fatigue); a sandbox boundary contains 80% of dangerous actions
(the rest - e.g. deleting files inside the project - happen within it). Standard library only.
"""
import random
ACTIONS, DANGER_P, CONTAIN = 2_000, 0.01, 0.8
AUTO_FPR, AUTO_FNR, SANDBOX_CUT = 0.004, 0.17, 0.84
def human_catch(prompts_so_far):
return max(0.35, 0.90 - 0.004 * prompts_so_far) # fatigue: attention decays with volume
def simulate(mode, rng):
prompts = executed_danger = blocked_safe = caught = 0
session_prompts = 0
for i in range(ACTIONS):
if i % 100 == 0: session_prompts = 0 # a new session every 100 actions
danger = rng.random() < DANGER_P
if mode == "manual prompts":
ask = rng.random() < 0.6 # ~60% of actions need a prompt
elif mode in ("sandbox", "sandbox + auto"):
ask = rng.random() < 0.6 * (1 - SANDBOX_CUT) # most actions run freely inside the boundary
if mode == "sandbox + auto": ask = False # the classifier handles what the boundary doesn't
if danger and rng.random() < CONTAIN: caught += 1; continue # the boundary contains it
if mode == "sandbox + auto":
if danger:
if rng.random() < AUTO_FNR: executed_danger += 1
else: caught += 1
elif rng.random() < AUTO_FPR: blocked_safe += 1
continue
else: # auto mode: a classifier replaces the human
if danger:
if rng.random() < AUTO_FNR: executed_danger += 1
else: caught += 1
elif rng.random() < AUTO_FPR: blocked_safe += 1
continue
if not ask:
if danger: executed_danger += 1
continue
prompts += 1; session_prompts += 1
if danger:
if rng.random() < human_catch(session_prompts): caught += 1
else: executed_danger += 1
return prompts, executed_danger, caught, blocked_safe
print(f"{ACTIONS:,} agent actions, {DANGER_P:.0%} dangerous; 200 simulated weeks per mode\n")
print(f" {'mode':<16}{'prompts shown':>15}{'dangerous executed':>21}{'dangerous stopped':>20}{'safe actions blocked':>23}")
for mode in ["manual prompts", "sandbox", "auto mode", "sandbox + auto"]:
rng = random.Random(4)
r = [simulate(mode, rng) for _ in range(200)]
avg = [sum(x[k] for x in r) / len(r) for k in range(4)]
print(f" {mode:<16}{avg[0]:>15,.0f}{avg[1]:>21.1f}{avg[2]:>20.1f}{avg[3]:>23.1f}")
print("\nManual prompts miss dangerous actions that never asked, and fatigue erodes the rest; layers compound.")Output:
2,000 agent actions, 1% dangerous; 200 simulated weeks per mode mode prompts shown dangerous executed dangerous stopped safe actions blocked manual prompts 1,200 11.1 9.2 0.0 sandbox 191 3.9 16.4 0.0 auto mode 0 3.3 16.8 8.1 sandbox + auto 0 0.8 18.9 7.7 Manual prompts miss dangerous actions that never asked, and fatigue erodes the rest; layers compound.
Showing people 1,200 prompts a week does not make them safe: most dangerous actions never asked, and fatigue erodes the rest. A sandbox removes most prompts and contains most damage; the classifier catches much of what remains at the cost of a few wrongly blocked actions; stacked, fewer than one of the week's 20 dangerous actions gets through (0.8 on average). The exact numbers depend on the assumptions โ the ordering is the lesson, and it is why Anthropic ships the layers together.
04 Case Study 3: Google Gemini Deep Research โ a Long-Horizon Research Harness
Theme: autonomous, iterative research with citations, tools and asynchronous execution.
Gemini Deep Research is an autonomous research agent: it plans, searches, reads, notices gaps in what it knows, searches again, and writes a cited report. In April 2026 Google launched Deep Research and Deep Research Max on Gemini 3.1 Pro, available through the Gemini app and โ for developers โ exclusively through the Interactions API. Max spends more test-time compute for long, accuracy-critical investigations across hundreds of sources.
05 Cross-Case Comparison
06 What These Cases Teach Us
07 Model vs Harness: Same Model, Different Outcomes
The same frontier model โ its reasoning, tool use, multimodal understanding and long-context processing โ performs very differently depending on the harness around it. Toggle harness components on and off to see which failure each one prevents (illustrative):
08 A Reusable End-to-End Harness Pattern
Distilled from the three cases โ the same twelve steps appear, in different forms, in each:
09 What Remains Proprietary or Unknown
These products are unusually well documented โ Anthropic even published its classifier's error rates โ but important details stay private:
Next: Product case study #1 โ the OpenAI Agents API: the Codex harness as a managed production runtime.