Agent memory architecture: short-term, long-term, and retrieval patterns

1. Memory is not the context window

Context engineering manages what fills the window during one session: tool results, retrieved documents, conversation history. All of it is gone when the session ends, however carefully it was chosen. Memory is the separate question of what should still be known the next time an agent starts with an empty window. Mixing the two produces opposite mistakes. Treat everything in a long session as worth persisting, and memory becomes an uncurated transcript dump. Treat nothing as worth persisting, and the agent relearns the same facts about a user or task from scratch every session.

2. Short-term memory: working state within a session

Within a session, an agent's working memory is mostly its context window. More sophisticated designs add an explicit scratchpad the agent writes intermediate state to on purpose. A scratchpad has a narrower job than the full context: a running plan, a list of completed subtasks, facts gathered mid-task that the agent wants to reference reliably later in the session without depending on the model to recall them from a long, cluttered context. This is still session-scoped. It structures one session's working state and decides nothing about what survives past it.

3. Long-term memory: deciding what needs to persist

The hard design question is what belongs in persistent memory, not how to store it. Good candidates are facts whose value comes from being available before the current session could surface them again:

• a user's stated preference that's likely to hold across many future sessions;
• a coding agent's discovery that a particular test in a particular repository is known to be flaky and unrelated to what it's debugging;
• a customer's account-specific context a support agent shouldn't re-derive on every new ticket.

A one-off detail relevant only to the task running when it came up is not a candidate. It's ordinary session content that should disappear when the session ends. Storing it because it was true once inflates the store with content that will never be useful again.

4. Storage shapes: structured facts, embeddings, or raw transcripts

A structured store holds explicit key-value facts or records with defined fields: a user's timezone, a project's preferred code style. It is the easiest shape to retrieve from reliably and to keep consistent, because a later write to the same field overwrites the old value. No two versions of a fact sit side by side with nothing to resolve them. It is also the most limited: it only holds what someone designed a field for, and a fact that fits no field gets forced into the wrong one or dropped.

An embedding-based store indexes memory for semantic retrieval the way a RAG system indexes documents. It handles open-ended memories, such as a past conversation's gist or a preference mentioned in passing that maps to no predefined field. It also inherits the retrieval failures RAG failure modes documents: a relevant memory can fail to surface if its phrasing doesn't match the current query well in embedding space.

A raw transcript store keeps full past session logs. It is the least filtered and the hardest to use well, since finding one relevant detail in a long transcript is itself a retrieval problem. Most systems that store raw transcripts as memory depend on a summarization or extraction step to turn them into one of the other two shapes before they're useful.

5. Write policy: the harder problem than storage

What gets written to long-term memory, and when, decides more of a memory system's quality than the storage shape does. Write everything a session touches and the store grows without bound, dominated by the one-off details from Section 3. That dilutes retrieval the way an over-broad RAG corpus does. Write nothing automatically, requiring an explicit user action to save, and you avoid noise but miss the common case: a preference stated in passing that the user expects to be remembered without ever saying "remember this."

A middle design tends to work better: a deliberate extraction step at the end of a session, or at task boundaries within one, that asks what from this session meets Section 3's bar. That step is its own design surface. Too aggressive and it writes noise anyway. Too conservative and it misses things a user reasonably expected to be remembered without asking.

6. Staleness and contradiction

A memory doesn't stay true on its own. Without a way for a new fact to override or retire an old one, the store accumulates contradictions, like an event-sourced system accumulating history with no projection step to resolve it into current state. A preference from six months ago that has since changed, or a fact about a project's structure that a refactor made false, sits in the store as confidently as anything written yesterday. Nothing marks "still true" apart from "was true once."

The mitigation is an explicit resolution step. Check a new fact that may contradict or update an existing memory against what's stored. Either the newer one replaces the older one outright (for a single-valued fact like a preference), or both are kept with enough context, at minimum a timestamp, for a later retrieval to prefer the more recent one when they disagree. A store with no timestamps and no update mechanism loses reliability over time, invisibly, until an agent confidently acts on something that stopped being true a while ago.

7. Worked example: a coding agent's stale preference

A coding agent stores a memory that a user prefers verbose logging in a service it works on often. It learned this in an early session where the user asked for more logging while debugging. Months later, in a different session, the user finds the logging too noisy and asks the agent to trim it. That session never revisits the stored preference. Later still, working on an unrelated task in the same service, the agent follows the "prefers verbose logging" memory and adds a new log statement, reintroducing the noise the user had asked to remove.

Memory for preferences like this is fine. The later request to reduce logging should have triggered a write that updated or retired the original memory. A system that only writes new facts and never revisits old ones keeps resurfacing decisions the user has reversed. The defect is a missing update path; the original memory was a reasonable thing to store.

This pairs with context engineering for the session-scoped budget retrieved memory competes for once it's pulled back into a live window, and with multi-agent orchestration patterns with Claude for using a persistent memory store as shared state across agents that otherwise can't see each other's sessions.