NOTES / PROMPTING AND CONTEXT ENGINEERING
Memory
Giving agents recall beyond one window: working notes, files the agent can read and write, and the OS-style idea of paging information in and out of context.
Why a model needs memory
A model has no memory between calls beyond what is in its prompt. Everything it “remembers” is text you supplied. For a single task the context window is enough. For work that spans sessions or exceeds the window, you need storage outside it.
The operating-system analogy
The MemGPT paper proposed treating the context window like main memory in an operating system, with external storage like disk. Borrowing the idea of virtual memory paging, it lets the LLM manage what sits in its own context, paging information in and out via function calls, to give the illusion of unbounded context with a fixed-window model1. The system alerts the model when context space is running low, so it can write important data to persistent memory, and it can search out-of-context data to bring it back in1.
Diagram: my own, illustrating the idea in 1.
Memory as files the agent manages
A simple, robust pattern is a directory of files the agent reads and writes. Anthropic’s memory tool follows this shape: Claude can create, read, update and delete files in a memory directory that persist across conversations, building knowledge over time without keeping everything in the window2. It is a client-side tool: Claude requests the operation and your application executes it against storage you control, returning the result as a tool result. Your handler must reject paths outside the memory directory to prevent path traversal2.
Diagram: my own, following the flow in 2.
Anthropic’s engineering post describes the same idea as structured note-taking, such as a to-do list or a notes file the agent keeps outside the window and pulls back in later3.
Kinds of memory
This split is my own working taxonomy rather than one from the sources:
| Kind | What it holds | Typical home |
|---|---|---|
| Working | Current plan and progress | A notes or to-do file |
| Episodic | What happened in past sessions | Logs, summaries |
| Semantic | Stable facts and preferences | A curated file or database |
| Procedural | How to do tasks | Instructions and skills |
Design questions
- What to store. Save decisions, constraints and unresolved items, not transcripts.
- When to read. Load a short index at start and fetch detail on demand, to avoid refilling the window.
- Staleness and conflict. Memories go out of date. Record dates and sources, and prefer updating over appending.
- Privacy. Stored memory can hold personal data. Apply retention, deletion and access rules; see Privacy and regulation for a high-stakes example.
- Security. Memory is an input like any other, so poisoned content can persist across sessions. See Prompt injection.
- Retrieval at scale. Large memories need search, not just reading files. See Embeddings and vector search.
Memory pairs with compaction: write what must survive before summarising what need not.
Sources
- [1]MemGPT: Towards LLMs as Operating Systems — Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez · arXiv · 2023PARAPHRASED
- [2]Memory tool — Anthropic, Claude Platform docsPARAPHRASED
- [3]Effective context engineering for AI agents — Anthropic Engineering · 2025PARAPHRASED