Home โ€บ Blog โ€บ Memory and state
Inside the Agent Layer ยท Part 6 ยท Memory and state

What Agents Remember: Memory, State and the Context Window

A model remembers nothing between requests. Everything an agent "knows" during a task is text in its context window, re-sent on every call, and everything it should know next week must be stored somewhere outside the model and fetched back. On long tasks the context fills with tool results and old reasoning until the agent slows, costs more and gets confused. This part explains the kinds of agent memory, how agents trim and summarise their context, the memory products competing to be the agent's long-term store, and why prompt caching has quietly become one of the most important pieces of agent infrastructure.

FL
FrontierAGI Team
โš ๏ธ How to read this (as of 28 Oct 2026) Anthropic's context-editing and memory-tool figures are its own internal evaluations. Memory-product benchmark scores are vendor-reported and openly disputed between vendors (โš ๏ธ). Funding is as reported. The context simulator and cache calculator are our models. Disclosure: Claude, made by Anthropic, helps write this site; Anthropic's features are one of several examples.

1. Case file

๐Ÿงฉ
Agent memory, October 2026Part 6 of the agent layer
Anthropic context editing + memory tool (Sep 2025)+39% on an internal agentic-search eval; 84% fewer tokens in a 100-turn test (company figures)
Mem0$24M raised (seed + Series A), Oct 2025 (reported)
Letta (formerly MemGPT, UC Berkeley)Agent manages its own core, recall and archival memory
ZepTemporal knowledge graph (Graphiti)
BenchmarksLoCoMo, LongMemEval; vendors dispute each other's scores โš ๏ธ

2. Four kinds of memory

๐ŸชŸ WorkingThe context window: the task, recent steps, tool results.Lives for one call; re-sent every time.
๐Ÿ“’ Session stateThe full history of this run, saved by the runtime (Part 3).Lives for the task.
๐Ÿ—‚๏ธ Long-termFacts about the user, project or organisation: preferences, decisions, past outcomes.Lives across sessions; must be retrieved.
๐Ÿง  ProceduralHow to do things: instructions files, skills, saved playbooks, and ultimately the model's own weights.Changes slowly; loaded when relevant.

Our explainer Context Engineering and Memory covers the prompting side; Agent Memory and Planning Patterns covers design patterns. This part is about the infrastructure.

3. The context fills up

Tokens in context over a long run

Each turn adds the model's output and a tool result. Pick a strategy. Our simulation with typical sizes.

4. Editing, compaction and notes

Clear old tool resultsMost tool output (a file listing, a page of search results) matters for one step. Context editing removes stale results once a threshold is passed, keeping recent ones. Anthropic reported this alone improved an internal agentic-search eval by 29%.
CompactionWhen the context nears its limit, the model summarises the run so far and continues from the summary. Coding agents such as Claude Code and Codex do this automatically. Cheap, but details can be lost.
Notes to selfThe agent writes key facts and progress to files (a memory tool, a to-do file, a notes directory) and reads them back when needed. Anthropic's memory tool works this way, with the developer storing the files.
Sub-agentsHand a messy sub-task to a fresh agent with a clean context and get back only its conclusion.

5. Long-term memory stores

ApproachExampleHow it works
Extract factsMem0After a conversation, extract durable facts, de-duplicate them, and retrieve them later with combined semantic, keyword and entity scoring
Self-managed tiersLetta (MemGPT)The agent itself decides what stays in its core memory and what is moved to recall or archival storage, like an operating system paging memory
Knowledge graphZep (Graphiti)Facts stored as a graph with time attached, so the agent knows what was true when
Plain files + searchMemory tools, instruction filesMarkdown files the agent reads, writes and greps; transparent and easy to edit
Search past chatsConsumer assistantsLook up relevant earlier conversations on demand instead of keeping a profile

Which works best is contested. Mem0 reports high LoCoMo and LongMemEval scores; Zep published a rebuttal arguing its own corrected LoCoMo result is higher; Letta reported that an agent with just file-search tools scored 74.0% on LoCoMo, above Mem0's best reported graph variant โš ๏ธ. Different harnesses and judges make the numbers hard to compare (Evaluation Layer Part 3 explains why judge choice matters). The safe reading: simple, inspectable memory is a strong baseline.

6. Prompt caching: memory for the hardware

Every agent step re-sends the whole context: system prompt, tool definitions, history. Without help, the inference server recomputes the attention keys and values (the KV cache, see ML Layer Part 7) for all of it, every time. Prompt caching keeps the computed prefix on the server so the next call only processes the new tokens. Providers bill cached reads at a large discount (Anthropic charges cache reads at 10% of the base input price; OpenAI applies caching automatically at a discount). For agents, whose prompts are mostly repeated prefix, this changes the economics.

What caching saves on one agent run

Our model: each turn re-sends the prefix; cached tokens billed at the chosen fraction.

This is why agent builders keep the start of the prompt stable (system prompt, tools, then history appended at the end) and why context editing has a cost: deleting something early in the context invalidates the cache after that point.

7. Memory risks

Memory poisoningAn injected instruction saved to memory comes back in every future session. Memory writes need the same suspicion as tool results.
Stale factsThe user moved team; the project changed its API. Memories need dates, sources and a way to expire.
Privacy and deletionStored memories are personal data: users must be able to see, edit and delete them, and data-protection rules apply.
Leaking across usersA shared memory store must be partitioned per user and tenant, with the same permission checks as any database (Part 5).

8. The series

Inside the Agent Layer: eight parts

VerdictAgent memory is mostly context management: keep the window small and relevant, save what matters to inspectable storage, and keep the prompt prefix stable so caching does the heavy lifting. Long-term memory products differ in approach, and their benchmark claims are disputed; simple files with search are a strong baseline. To know whether any of this is working, and what it costs, you need to see inside each run: Part 7, observability and cost.

9. Reading the numbers

Primary (company-reported)
Context editing and memory tool, +29% / +39% / โˆ’84% tokens (Anthropic, Sep 2025, internal evals). Cache-read pricing (provider pricing pages).
Reported / disputed โš ๏ธ
Mem0 funding (TechCrunch via secondary). LoCoMo and LongMemEval scores from Mem0, Zep and Letta, each vendor-run.
Our illustrations
Context simulator, caching calculator.