Agent Memory Architectures

The first article said an agent is a model in a loop with tools. The second said the model is a brain in a jar — it can't reach your systems, only ask. This one adds the part both of those quietly assumed away: that brain has no memory between calls. Every turn starts from a blank slate. What you carry forward, what you persist, and what you leave out is the architecture.

1The model is a brain in a jar — and the jar is empty every turn

The tool article landed on a single uncomfortable fact: the model runs in a sealed box with no hands. Here is its twin, just as load-bearing: that box is also wiped clean between every call. The provider is stateless. When you POST a request, the model reads what you sent, produces text, and forgets all of it the instant the response leaves. There is no session on their side. There is no "it." The next turn is a brand-new brain handed a transcript.

This is why memory in agents is not a feature you bolt on — it is the thing that makes a conversation possible at all. If the model remembers nothing, then the only reason it "knows" your name on turn five is that you re-sent your name on turn five. The transcript you keep re-transmitting is the memory. There is no other.

The context window is not where the model looks things up. It is the only thing the model is.

So the real question of this primer is never "does the agent have memory?" It's "what do you choose to put back in front of a stateless model each time it wakes up?" Too little and it forgets the order number it was just given. Too much and it drowns — slower, costlier, and worse at finding the one fact that mattered. Memory engineering is the discipline of answering that question well, and it splits cleanly into two layers that beginners almost always merge.

2The one distinction that fixes most memory bugs

Section 1 ended on a claim it didn't yet justify: that the "what do I put back in the window?" question splits into two layers. This section is where that split earns its place — because getting it wrong is the single most common way production memory breaks, and getting it right is the frame the rest of the article is built on.

Almost every "the agent forgot" bug traces back to collapsing two different things into one. They live in different places, are scoped by different keys, survive for different lengths of time, and answer different questions. Get them separate in your head before you write a line of code.

The classic failure is the returning customer. They told your support agent last week that they're on the Enterprise plan with an SSO setup. They open a new chat today — new thread_id — and the agent asks for all of it again. Nothing is broken in the agent loop. The agent simply never had long-term memory, so a new thread starts cold. Short-term memory would have carried it within last week's chat and lost it at the door. Only long-term memory, keyed to the user and not the thread, survives that door.

3Short-term memory: the checkpointer, in plain mechanism

With the two layers named, the next four sections take them one at a time. We start with the easier layer — short-term memory — because it's the one the framework largely handles for you, and seeing how little work it takes makes the long-term half (which you build yourself) easier to appreciate by contrast. The goal here is narrow: understand the exact mechanism that carries a conversation from one turn to the next.

Short-term memory is the easy half — so easy it's almost invisible, which is exactly why people skip it and then wonder why their bot re-asks for the order number. The mechanism is a checkpointer: a storage backend that snapshots the graph's state after every step and keys those snapshots by thread_id. Pass the same thread_id next turn and the prior state is restored before your nodes run. Pass a new one and you get a clean conversation. That's the whole idea.

There's nothing magic underneath — the messages accumulate in a typed state object, and the checkpointer just serializes that object to a backend you choose. The only real decision is which backend, and it maps directly to how durable you need to be:

The same interface across all three means you prototype on MemorySaver and swap to PostgresSaver for production with no change to your node logic. Here is the entire short-term-memory mechanism — the "amnesia bug" and its one-line fix:

# No checkpointer: every invoke() starts from an EMPTY state.
# The model isn't forgetting — it was never told.
graph = builder.compile()  # <- no memory wired in

graph.invoke({"messages": [{"role":"user",
              "content":"My order number is 9847."}]})

graph.invoke({"messages": [{"role":"user",
              "content":"What's my order number?"}]})
# -> "Could you provide your order number?"  ← product-killing bug

from langgraph.checkpoint.memory import MemorySaver

checkpointer = MemorySaver()
graph = builder.compile(checkpointer=checkpointer)  # <- the whole fix

# thread_id is the routing key: same id = same conversation.
cfg = {"configurable": {"thread_id": "conv-001"}}

graph.invoke({"messages":[{"role":"user",
             "content":"My order number is 9847."}]}, config=cfg)

graph.invoke({"messages":[{"role":"user",
             "content":"What's my order number?"}]}, config=cfg)
# -> "Your order number is 9847."  ← state was restored from the checkpoint

# Same interface, durable backend. Multi-worker + crash recovery.
from langgraph.checkpoint.postgres import PostgresSaver

DB_URI = "postgresql://user:pass@localhost:5432/db"

with PostgresSaver.from_conn_string(DB_URI) as checkpointer:
    checkpointer.setup()        # run ONCE — creates the tables (CI/migration step)
    graph = builder.compile(checkpointer=checkpointer)

    cfg = {"configurable": {"thread_id": "conv-001"}}
    graph.invoke(..., config=cfg)
# Every node writes a checkpoint row, so a crashed run resumes from
# its exact last state — the durable-execution thread picks this up later.

Where that state physically lives is a deployment decision

One caveat before we leave short-term memory, because it's where the "easy half" quietly bites. Everything above treated the checkpointer as a single thing, but where it persists is a separate choice that depends on how your agent is deployed — and getting it wrong produces a subtle amnesia bug that passes every local test.

The checkpointer is an interface; the backend you point it at must match your runtime. If your agent runs as a long-lived stateful container, you could hold conversation state in process memory — fast, but it dies with the container and can't be shared. The moment you move to a serverless or multi-worker deployment, that breaks: turn 1 lands on worker A, turn 2 on worker B, and worker B has never seen the conversation. Stateless compute forces state into shared storage — a database-backed checkpointer (or a shared cache) every worker can reach — which is exactly why PostgresSaver is the production default and in-process state is a dev-only convenience.

4The window is a budget, so short-term memory needs pruning

Section 3 got the conversation to persist. This section makes sure that persistence doesn't quietly destroy the agent — because the same accumulation that gives you memory, left unmanaged, is what makes the agent slow, expensive, and forgetful. Keeping the window honest is the price of the short-term layer working at all.

Here's the trap hiding inside the easy half. The checkpointer faithfully accumulates every message — and §1 told you that the whole accumulated transcript is re-sent on every turn. Left alone, a long conversation walks straight into the context window's ceiling: the model slows down, costs climb, and — the part people miss — recall actually gets worse, because attention spreads thin across a wall of mostly-irrelevant history. More context is not more memory. Past a point, it's less.

A bigger window tempts you to remember everything. The skill is choosing what to forget.

So short-term memory isn't just "keep the messages." It's "keep the messages, and have a plan for when there are too many." Three plans, in rough order of fidelity-vs-savings:

1. Trim. Keep the last N turns (and always the system message); drop the oldest. Cheapest, dumbest, and fine for shallow chat where ancient turns don't matter. trim_messages does this out of the box.
2. Summarize / compact. When the transcript crosses a threshold, replace the old turns with an LLM-written summary and keep recent turns verbatim. You trade exact wording for a far smaller footprint that still carries the gist. This is the workhorse for long sessions.
3. Offload. Move detail out of the window entirely into long-term memory or external storage, and pull it back only when a turn needs it. The window holds a pointer; retrieval rehydrates on demand. This is the bridge into §5.

The fattest thing in the window is usually a tool result

Trimming and summarizing turns is half the budget story. The other half is what each turn drops into the window — and the worst offender is a raw tool payload. A query tool that returns five hundred rows of JSON will sit in context, verbatim, on every subsequent turn, crowding out everything that matters. The fix is the same principle the tool article's "minimal return shape" pointed at: transform the result before it enters memory. Run the raw payload through a cheap function — plain Python, a parser, or a small model pass — that keeps the counts, the schema, the handful of rows the model actually needs, and discards the rest. The model sees a dense summary; the raw blob never touches the budget.

Write durable state off the hot path

One more budget-adjacent rule, and it's about latency rather than tokens. When a turn produces something worth persisting to long-term memory, don't make the loop wait for the database write. Embedding a transcript and writing it to a store is slow; block on it and every turn inherits that latency. Instead, update the fast in-process or thread state synchronously so the loop continues, and push the durable write onto a background path — a queue, a worker, a deferred task. The agent stays responsive; persistence catches up behind it. The one thing the background write must guarantee is that a mid-task crash doesn't lose committed work, which is the durability concern the dedicated durable-execution article takes up.

5Long-term memory has three different jobs

Sections 3 and 4 covered the short-term layer end to end: how it persists, and how to keep it from overflowing. Now we cross to the other layer named in §2 — the long-term store — and the contrast is the whole point. Short-term memory was structural and mostly automatic; long-term memory is the half you design, and this section builds it up from what to store, to how the store works, to the exact nodes that read and write it.

Now the half you own. Long-term memory is not one bucket — it's three, and they come straight from how human memory is usually carved up. The reason to keep them separate is practical: each is written at a different moment and retrieved in a different way. Build one undifferentiated "memory" table and you'll retrieve a user's coffee preference when you needed last month's incident summary.

So much for what the three jobs are. Now for how you actually hold them. The mechanism for all three is the store: a backend that holds memories as JSON documents organized under namespaces — tuples that scope and segregate, like directories. ("memories", "user-123") walls one user's memories off from everyone else's, which is also your first line of defense against the cross-user memory leak. Within a namespace you put, get, and — crucially — search by semantic similarity when you configured an embedding index.

One clarification that trips people up: the three types (semantic / episodic / procedural) are a different axis from the two layers (short-term thread vs. long-term store) in §2. The types describe what kind of thing you're remembering; the layers describe where it lives and how long. The same type can appear in both layers — and seeing how they cross is what makes the architecture click:

Unlike the checkpointer, nothing in the long-term column is automatic. The framework gives you the store; you write the node that reads from it before the model reasons, and the node that writes back what's worth keeping. That read and that write are the entire long-term-memory pattern — two extra nodes wrapped around the model, with both memory layers wired in. The graph below is the whole thing, with the two entry paths coloured by where the session is:

Where the agent loop from article 1 fits

If you read the first article, one box in that graph should look suspicious: call_model is drawn as a single step, but article 1 said the model runs in a loop — think, call a tool, observe the result, think again, until it's done. Both are true. The loop lives inside call_model. What's really there is the model and a tool node ping-ponging: the model emits a tool call, the tool node runs it, the observation comes back, the model reasons again — that's the think-act-observe cycle from article 1, possibly many times per turn. The two memory nodes simply bracket that loop.

This is how the two articles compose. Memory isn't part of the loop — it wraps it. recall is the loop's pre-step (load the user's facts before the model starts), save is its post-step (write back what the turn produced). And the loop itself is quietly feeding short-term memory the whole time: every tool observation gets appended to the running message list — the working memory from §2 — which is exactly why a long, tool-heavy loop is the thing that blows the context budget §4 warns about. The loop generates the history; compaction keeps it from drowning the window.

The code for the two bracket nodes is below — recall_memories reads, save_memories writes — plus the wiring that hands both layers to the graph:

# BEFORE the model reasons: pull this user's relevant memories
# and put them in the system prompt. This is the "recall" half.
def call_model(state, runtime):
    user_id   = runtime.context.user_id
    namespace = ("memories", user_id)          # scope to THIS user

    # semantic search: most relevant facts for the latest message
    hits = runtime.store.search(
        namespace,
        query=state["messages"][-1].content,
        limit=5,
    )
    facts = "\n".join(h.value["data"] for h in hits)

    system = f"You are a support agent.\n\nKnown about this user:\n<memories>\n{facts}\n</memories>"
    return {"messages": model.invoke([{"role":"system","content":system}, *state["messages"]])}

# AFTER a turn resolves: write back the durable bits. This is "retain".
# Key choice matters: stable key for facts that OVERWRITE,
# UUID for events that ACCUMULATE.
import uuid

def save_memory(state, runtime):
    ns = ("memories", runtime.context.user_id)

    # semantic fact — deterministic key, so it updates in place
    runtime.store.put(ns, "plan", {"data": "Plan = Enterprise"})

    # episodic event — UUID key, so each incident is its own record
    runtime.store.put(ns, str(uuid.uuid4()),
        {"data": "Resolved SSO 502 on 2026-06-03 by rotating the cert."})
    return {}
# DON'T blindly append every turn — dedupe, or memory fills with noise
# and search quality collapses. Extraction is a design problem, not a dump.

# Dev: in-memory, vector-indexed, lost on restart.
from langgraph.store.memory import InMemoryStore
store = InMemoryStore(index={"dims": 1536,
                             "embed": "openai:text-embedding-3-small"})
# NOTE: without the index config, store.search() returns nothing.

# Production: durable, same interface.
from langgraph.store.postgres import PostgresStore
store = PostgresStore.from_conn_string("postgresql://user:pass@host:5432/db")
store.setup()                       # once, as migration

# Wire BOTH layers into the graph — they are not the same object:
graph = builder.compile(checkpointer=checkpointer,  # short-term, per thread
                        store=store)                # long-term, per user

6Two ways a conversation comes back to life

Section 5 built both nodes that touch long-term memory — but it left open a question those nodes can't answer on their own: when a turn arrives, how much history should you actually load? That depends entirely on whether the user is continuing an old conversation or starting a new one, and this section is about telling those two cases apart. It's where the §2 distinction and the §5 store finally meet a live request.

Persisted memory is inert until something pulls it back into the window — and there are two genuinely different ways that happens, depending on what triggered the turn. Confusing them is a quiet but expensive bug, because the right move for one is exactly the wrong move for the other. The next two subsections are those two cases, side by side.

Case A: resuming a paused thread

Take the first case — the user returns to the same conversation from yesterday. The thread's short-term state may have aged out of any fast cache, but the checkpointer persisted it. The right move is to restore that thread's actual history — load the prior turns back so the model picks up mid-conversation, exactly where it paused. Same thread_id, same story continuing. This is the checkpointer doing its job: the transcript is the memory, and you're rehydrating it.

Case B: starting a fresh session

Now the opposite case — the same user opens a brand-new conversation. Here the instinct to "load their history" is a trap: dumping yesterday's full transcript into a new thread burns the token budget and confuses the model with stale context it didn't ask for. Instead you load the long-term layer — the user's durable facts and preferences into the system prompt — and leave the old transcript on disk. The model starts with a clean thread but already knows who it's talking to. If a specific past episode becomes relevant mid-task, you retrieve that one episode by similarity, on demand — not the whole history.

The difference is visible in the code, and the clearest way to see it is to look at the window each path assembles. Same two LangGraph objects from §5 — the checkpointer and the store — used in opposite proportions:

# SAME thread_id as yesterday. The checkpointer already holds the
# transcript — you don't rebuild it, you just point at the thread and
# LangGraph restores the saved state before your nodes run.
cfg = {"configurable": {"thread_id": "conv-001"}}   # the SAME id as before

graph.invoke({"messages": [{"role":"user",
              "content":"Let's pick that back up."}]}, config=cfg)

# The window the model actually sees:
# [SYSTEM]  ... + this user's durable facts (still injected)
# [HISTORY] full restored transcript — "My order is 9847", the tool
#           lookups, the prior replies — ALL of it, from the checkpoint
# [USER]    "Let's pick that back up."
# -> The model continues mid-conversation. Short-term memory did the work.

# NEW thread_id. The checkpointer has nothing for it — and that's correct.
# DON'T reach back for yesterday's transcript; load the long-term FACTS only.
cfg = {"configurable": {"thread_id": "conv-002"}}   # a FRESH id

# recall the user's durable facts from the store (not their old turns)
ns    = ("memories", user_id)
facts = "\n".join(h.value["data"] for h in store.search(ns, query="profile", limit=5))

graph.invoke({"messages": [
    {"role":"system", "content": f"Known about this user:\n{facts}"},
    {"role":"user",   "content":"Start a new sync for the Q2 metrics."},
]}, config=cfg)

# The window the model actually sees:
# [SYSTEM]  ... + plan=Enterprise, prefers terse replies, tz=ET
# [HISTORY] EMPTY — 0 tokens of old transcript
# [USER]    "Start a new sync for the Q2 metrics."
# -> Clean thread, but it already knows WHO it's talking to.

Notice what stays constant across both: the durable facts are injected either way — knowing the user is always cheap and always useful. What differs is the transcript. Resuming restores it in full; a fresh start leaves it at zero and pulls a specific past episode only if a turn actually needs it.

7Where memory ends and retrieval begins

Section 6 used the store to recall a user's facts on demand — which raises an objection worth settling before we assemble the full picture, because if you don't draw this line your memory and your knowledge base will quietly bleed into each other. A fair question by now: if long-term memory is "search a store for relevant text and inject it," how is that different from RAG? The honest answer is that the mechanism overlaps — both embed, both do similarity search, both inject — but the intent draws a clean line, and confusing them produces muddled systems.

Keep them as separate stores even when the vector tech is identical. Memory is per-user and write-heavy from the agent's own experience; retrieval is shared and authored elsewhere. The store from §5 holds memory; your RAG knowledge base holds retrieval. The deep treatment of retrieval lives in its own articles — this primer only needs you to not collapse the two, because a "memory" that's really your product docs leaks across users and a "knowledge base" that's really per-user facts can't be curated. The full picture is the subject of the agentic-RAG primer.

Episodic memory leans on the very mechanics this section warns about — embed, similarity-search, inject — but it searches the agent's own past, not an authored corpus. Here it pays off as in-loop self-correction:

8The architecture on one page

Every prior section added one piece in isolation — the two layers, the budget, the three jobs, the bracket nodes, the two hydration paths, the memory/retrieval line. This section does only one thing: assemble them into a single picture so you can see how a turn flows through all of them at once. Nothing new is introduced here; it's the synthesis.

Put the two layers and the budget together and the whole memory system fits in a single diagram. A turn arrives; you load both layers; the model reasons over a window you deliberately assembled; you write back what's durable; and you compact before the window overflows. Every box here is a decision you made, not a default that happened to you.

9The memory checklist

The diagram in §8 showed the system whole; this section turns it into something you can act on. Each row distills one decision from the sections above into a single rule, so you can run a finished agent against it before shipping. Everything above, in one place — the questions to answer before you ship an agent that's meant to remember.

Conclusion: memory is what you choose to re-send

The loop article gave you a model in a loop with tools. The tool article showed that model is a brain in a jar that can only ask, never act. This one closes the gap both of them stepped over: that brain also forgets everything between calls, so the entire feeling of an agent that "knows you" is an illusion you construct, one turn at a time, by deciding what text to put back in front of it.

That decision splits cleanly. Short-term memory is structural — wire the checkpointer, key it by thread, prune before the window overflows, and put state somewhere every worker can reach when you scale out. Long-term memory is yours to design — write the recall and the retain, scope by user, keep semantic, episodic, and procedural distinct, persist on a background path so the loop stays fast, and don't let it drift into your knowledge base. Resume a paused thread by its transcript; greet a fresh session with the user's facts, not their history. Get both right and you've turned a stateless model into something that holds a conversation today and remembers it next week. Get the budget right on top, and it does that without going broke or slow.

This is the same spine the first two articles ended on: reasoning fluid, control flow rigid, state deterministic. Memory is the deterministic-state half made concrete — the part you persist, scope, and re-send on purpose. The model brings the reasoning. You bring the memory.