Updated
The taxonomy, write and retrieval architecture, evaluation guidance and product examples were checked against peer-reviewed research and current Mem0 and LangGraph documentation on October 1, 2026.
Tested with
CoALA · LoCoMo · LongMemEval · LongMemEval-V2 preprint · Mem0 · LangGraph
An AI agent does not remember in the human sense. A language model call receives an input context and produces an output; continuity across calls is created by the surrounding application. That application may retain conversation state, checkpoints, facts, events, instructions or external records, then select some of them for a later model call.
A useful engineering definition is therefore:
Agent memory is a selectively managed state subsystem that writes, maintains, retrieves and governs information beyond a single model call.
This definition is deliberately broader than a vector database and narrower than “everything the agent can access.” It includes the policies that determine what becomes a memory, how a newer observation affects an older one, which records may cross a user or tenant boundary, and how retrieved evidence enters a finite context window.
Key takeaway
Memory is not a transcript archive and it is not the model's context window. It is a governed process for preserving and reintroducing state when that state is relevant, authorized and sufficiently reliable.
Several mechanisms are routinely called “memory,” although they have different failure modes:
| Mechanism | What it contains | Persistence | Primary failure mode |
|---|---|---|---|
| Model context | Instructions, messages, tool results and retrieved evidence supplied to one model call | One call unless the application resends it | Overflow, distraction or missing evidence |
| Thread state | Messages, task variables, tool outputs and checkpoints associated with a conversation or workflow | Usually durable for the thread | Stale or inconsistent state after retries and concurrent writes |
| Long-term memory | Selected facts, events, preferences, examples or procedures intended for reuse | Across sessions | Incorrect extraction, conflict, leakage or irrelevant retrieval |
| External knowledge base | Documents and records maintained independently of the agent | Defined by the source system | Staleness, access-control mismatch or weak retrieval |
| Parametric knowledge | Patterns encoded in model weights during training or adaptation | Model-version lifetime | Unknown provenance, outdated knowledge or hallucination |
Only the first four are directly manageable by an application at runtime. Loading a stored fact into a prompt does not update the model's weights, and saying that an agent “learned” a preference usually means that an external record was written and later retrieved.
LangGraph's current documentation illustrates another important distinction: short-term memory can be durable thread-scoped state maintained through a checkpointer, even though only a selected portion is loaded into the model context. “Short term” describes scope and use, not necessarily volatile storage.
Human-memory terms—working, semantic, episodic and procedural—are useful analogies, and the CoALA framework uses them to organize language-agent architectures. They are not a universal storage schema. A production design should classify memory along independent axes rather than place every concept in one flat list.

Scope answers who or what the record belongs to, while lifetime answers how long it remains valid.
- A turn-level observation may expire after the next action.
- Thread state may last for one workflow and survive process restarts.
- A user preference may persist across sessions but must not leak to another user.
- Organization knowledge may be shared only with authorized members.
- A task checkpoint may be durable even though it is irrelevant after task completion.
Scope must be enforced before similarity ranking. A highly similar record from another tenant is still an unauthorized record.
Representation describes what the record means:
- Semantic facts represent propositions such as a preference, identity attribute or project decision.
- Episodic records represent an event or interaction situated in time, often with participants and outcome.
- Procedural knowledge represents how to perform a task: instructions, examples, workflows or policies.
- Task state represents the current status of a process, including completed steps, pending actions and tool outputs.
These categories can overlap. A support interaction is an episode; its resolved account preference may become a semantic fact; the successful resolution path may later be curated into procedural guidance. The transformation between them requires an explicit policy.
Retrieval is not synonymous with vector search. Useful systems combine mechanisms according to the query:
| Mechanism | Strongest use | Typical weakness |
|---|---|---|
| Exact key or lookup | IDs, account facts, canonical settings | Requires a known schema or key |
| Metadata filter | Tenant, user, project, source, status | Does not rank relevance by itself |
| Keyword search | Names, error codes, quoted phrases | Misses paraphrases |
| Semantic similarity | Conceptual matches and paraphrases | Can return plausible but contextually wrong items |
| Temporal retrieval | Current state, recency and “when” questions | Requires reliable time metadata and update semantics |
| Entity or graph traversal | Relationships among people, projects and resources | Adds extraction and consistency complexity |
| Task/dependency lookup | Workflow state and prerequisite steps | Usually application-specific |
A retrieval policy can use strict authorization and metadata filters first, generate candidates through several signals, rerank them, and then assemble a context under a token budget. Mem0's current managed documentation, for example, describes semantic, keyword, entity and temporal signals; its open-source behavior depends on the configured vector store and optional reranker, and does not provide the same managed graph-memory behavior.
The common diagram “retrieve, prompt, answer, write a summary” is too universal. Some turns need no long-term read. Many should produce no memory write. Writes may occur synchronously, in a background job, after user confirmation or only at a workflow checkpoint.
A defensible write path separates observation from commitment:

- Observe. Receive messages, tool results, user edits or environment events without assuming they should be retained.
- Propose. Extract candidate facts, events, task transitions or reusable procedures.
- Validate. Check authorization, sensitivity, usefulness, source reliability, confidence and retention policy.
- Resolve. Compare candidates with existing records; deduplicate, supersede, merge, preserve as history or reject.
- Commit. Store a versioned record with provenance, temporal fields and an audit trail.
The timing is a product decision. LangGraph documents both hot-path writes, which make a new memory immediately available but add latency and task complexity, and background writes, which separate memory formation from the response path but introduce freshness and scheduling trade-offs.
A useful record schema makes those decisions inspectable:
type MemoryRecord = {
id: string;
namespace: { tenantId: string; userId?: string; agentId?: string };
kind: "fact" | "event" | "procedure" | "task-state";
content: string;
source: { type: "user" | "tool" | "system"; reference: string };
observedAt: string;
effectiveFrom?: string;
expiresAt?: string;
supersedes?: string;
confidence?: number;
sensitivity: "public" | "internal" | "personal" | "restricted";
};This is an illustrative contract, not a universal standard. The important properties are explicit ownership, provenance, time, versioning and sensitivity—not these exact field names.
“The user lives in Austin” and “the user moved to Seattle” need more than similarity search. The system must distinguish:
- observation time: when the application learned the statement;
- effective time: when the statement became true;
- expiration: when the record should no longer be used;
- historical validity: whether an old fact remains useful for past-tense questions;
- supersession: which record replaces another for current-state queries.
Deleting the older record loses history; keeping both without temporal semantics can make the current answer ambiguous. The correct policy depends on the product: an itinerary, a medical record and a music preference should not share the same update rules.
Mem0's current default extraction path is additive: new facts do not silently rewrite older ones. Its documentation directs applications to use explicit update or delete operations when correction or removal is required. That behavior is one implementation choice, not a substitute for application-level temporal rules.
The read path should first establish authorization and scope, then gather and rank evidence:

- Derive the query, task state and temporal intent.
- Apply tenant, user, agent, consent and purpose filters.
- Generate candidates through the mechanisms appropriate to the question.
- Rerank using relevance, recency, reliability, diversity and conflict signals.
- Assemble a bounded context that preserves source and time metadata.
- Require the model to distinguish evidence, uncertainty and absence.
The context assembler must not convert a low-confidence extracted memory into an unquestioned system instruction. Records should retain their status and provenance so that the model—or application logic—can prefer authoritative sources, surface a conflict or abstain.
Retrieved does not mean true
A memory may faithfully preserve a user's earlier statement and still be outdated, malicious, incorrectly extracted or irrelevant to the present purpose. Retrieval establishes availability, not truth or authorization.
Different access patterns justify different stores. A relational database may be the source of truth for facts and metadata; a vector index may accelerate semantic candidate generation; an event log may preserve history; a graph may support relationship queries; object storage may retain larger artifacts.
Mem0's documented architecture illustrates this separation. Its managed system uses SQL for facts and metadata, a vector database for embeddings and an entity store for entity-oriented retrieval. The application still chooses which results to include in the model prompt. Managed Mem0 and Mem0 OSS also differ in available retrieval behavior, so claims about one should not be generalized to the other.
mem0
mem0ai/mem0
A memory layer that extracts, stores and retrieves user-specific context for agents
The Memory Layer for AI Agents - Drop-in memory infrastructure for AI agents and apps. Context that persists. Built for production.
- Python
- agentic-memory
- agentic-memory-system
69,188 stars8,230 forksSnapshot Nov 18, 2026
The correct question is not “Which memory database should we use?” It is “Which record types, consistency rules and queries must the system support?” The store follows from that contract.
An end-to-end answer can fail because the right event was never written, because retrieval missed it, because stale evidence outranked the current fact, or because the model ignored correct evidence. One aggregate accuracy score cannot identify those causes.
Measure whether the memory formation process creates the right records:
- extraction precision and recall against annotated candidates;
- duplicate and contradiction rate;
- proportion of sensitive or disallowed content incorrectly retained;
- provenance completeness;
- latency and cost of synchronous or background formation.
Evaluate the evidence set before generation:
- recall at a fixed context budget;
- precision or nDCG for ranked candidates;
- current-fact accuracy and stale-record rate;
- scope-violation rate, which should be zero for protected boundaries;
- evidence diversity, conflict detection and retrieval latency.
Measure what the agent does with memory:
- task or answer accuracy;
- citation or evidence attribution correctness;
- temporal and multi-session reasoning;
- correct handling of knowledge updates;
- abstention when the required information is absent;
- user-visible correction and deletion behavior.
LongMemEval, published at ICLR 2025, contains 500 questions testing information extraction, multi-session reasoning, temporal reasoning, knowledge updates and abstention. Its authors report a 30% accuracy drop for evaluated commercial assistants and long-context models in sustained-interaction settings, and analyze memory as indexing, retrieval and reading stages. The result demonstrates difficulty under that benchmark; it is not a universal estimate for every production agent.
LoCoMo, published at ACL 2024, evaluates long-term conversations averaging 600 turns and 16,000 tokens over as many as 32 sessions. Its experiments found that long-context and retrieval-augmented approaches improved some results but still struggled with long-range temporal and causal relationships relative to human performance.
As of this revision, LongMemEval-V2 is an arXiv preprint, not a peer-reviewed conference result. It extends evaluation from conversational history to multimodal web-agent trajectories and explicitly measures both answer accuracy and query latency. Its framing is useful because operational memory must retrieve adequate evidence within a bounded time, but its reported leaderboard numbers should be treated as preliminary research results.
Persistent memory creates risks that a stateless prompt does not:
- Cross-user or cross-tenant leakage. Missing namespace filters can expose another user's data.
- Memory poisoning. Untrusted text can plant instructions or false facts that influence later sessions.
- Sensitive-data retention. Raw transcripts may contain credentials, health information, identifiers or third-party data.
- Purpose drift. Information collected for one workflow may be reused in another without a valid product or legal basis.
- Deletion failure. Removing the source record may leave embeddings, caches, summaries, backups or derived memories intact.
- Invisible inference. A model may extract a sensitive or incorrect attribute that the user never intended to persist.
Minimum controls include server-side authorization on every read and write, tenant-scoped indexes, field-level sensitivity labels, encryption, retention limits, audit logs, poisoning-resistant separation between data and instructions, and a user-visible correction and deletion path. Deletion should propagate to derived indexes and caches, not merely hide a row in a dashboard.
Mem0's current guidance explicitly recommends scoping searches by identifiers such as user_id, agent_id or run_id and avoiding secrets, raw credentials and unredacted sensitive data. These are necessary controls, not a complete privacy or security program.
Before selecting a memory framework, write down the contract:
- Enumerate decisions. Which future decisions improve when past information is available?
- Define record types. Separate facts, events, procedures and task state.
- Set authority. Identify who may create, confirm, update and delete each type.
- Model time. Define observation, effective, expiration and supersession semantics.
- Enforce scope. Make tenant and user boundaries structural, not optional query hints.
- Choose retrieval. Match exact, filtered, semantic, temporal or graph retrieval to each query class.
- Budget context. Specify how much evidence can enter a model call and how conflicts are represented.
- Evaluate stages. Test write, retrieval and end-to-end behavior independently.
- Design recovery. Support correction, deletion, re-indexing and audit before launch.
The engineering test
A memory system is trustworthy only when it can explain what was stored, why it was stored, who may retrieve it, which newer record changed its meaning, and how its contribution to an answer was evaluated.
Agent memory is best understood as governed state management around a model, not as a human-like faculty and not as a vector-search feature. Its design spans independent choices about scope, representation, write policy, temporal consistency, retrieval, context assembly, security and evaluation.
The most important architectural move is separation: stored memory from model context, current state from history, semantic content from retrieval technique, and retrieval success from downstream answer quality. Once those boundaries are explicit, storage technology becomes a replaceable implementation choice rather than the definition of memory itself.
- Sumers et al., “Cognitive Architectures for Language Agents,” Transactions on Machine Learning Research, 2024 — a modular framework for organizing language-agent memory, action and decision-making.
- Maharana et al., “Evaluating Very Long-Term Conversational Memory of LLM Agents,” ACL 2024 — the LoCoMo dataset and evaluation of long-range conversational memory.
- Wu et al., “LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory,” ICLR 2025 — a benchmark for extraction, multi-session and temporal reasoning, updates and abstention.
- Wu et al., “LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues,” arXiv preprint, 2026 — a current, non-peer-reviewed benchmark for memory over long web-agent trajectories.
- Mem0, “How Mem0 Works” and “Add Memory” — current implementation behavior, retrieval signals, store roles and managed-versus-OSS distinctions.
- LangGraph, “Memory overview” — current documentation for thread-scoped state, long-term stores and hot-path versus background writes.
