arXiv 2504.19413 · April 2025

Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory

Prateek Chhikara · Dev Khant · Saket Aryan · Taranjeet Singh · Deshraj Yadav · mem0.ai

TL;DR

LLMs forget everything once it falls outside their context window. Mem0 fixes this by dynamically extracting compact memories from conversations, storing them in a vector database, and retrieving only what's relevant at query time. A graph-memory variant (Mem0^g) adds structured entity–relationship storage for temporal and relational queries. Both beat every baseline on the LOCOMO benchmark while using 91% less latency and 90%+ fewer tokens than full-context approaches.

26% better than OpenAI (LLM-as-Judge) 91% lower p95 latency vs full-context 7k tokens vs 26k (full-context) Graph memory excels at temporal reasoning SOTA on LOCOMO across 4 question types

Why This Matters

Every time you start a new chat with an AI assistant, it has no memory of you. Tell it you're vegetarian today; ask for dinner recommendations tomorrow — it might suggest chicken. This isn't a quirk; it's a fundamental architectural constraint. LLMsLarge Language Models — neural networks trained to predict and generate text, like GPT-4. process a fixed-length "context window" of tokens, and anything outside that window is simply gone.

Even as context windows grow (GPT-4: 128K tokens, Claude 3.7: 200K, Gemini: 10M+), the problem isn't fully solved. Real conversations span weeks or months. A user's dietary preference from three weeks ago sits buried among thousands of tokens of unrelated coding discussions. The model either can't fit it all, or drowns in irrelevant context — attention degrades over distant tokens.

For high-stakes domains — healthcare, education, enterprise support — this forgetfulness actively erodes trust. What we need isn't a bigger window; we need a smarter memory system that selectively stores, consolidates, and retrieves only what's relevant.

Interactive Demo: Memory vs. No Memory

The Big Idea

Mem0 treats memory as a dynamic knowledge base rather than a static log. As a conversation unfolds, an LLM continuously extracts salient facts from each new message pair, compares them against existing memories, and decides whether to add, update, delete, or ignore them. At query time, only the top-s semantically similar memories are retrieved and injected into the prompt — keeping token counts tiny while preserving recall accuracy.

A second variant, Mem0^g, augments this with a knowledge graphA structured representation where entities (people, places, events) are nodes and their relationships are labeled edges. — storing memories as entity–relationship triplets in Neo4j. This enables richer temporal reasoning ("when did Alice visit Paris?") and multi-hop inference ("what does Alice's employer think about her project?") that flat text memories struggle with.

How It Works

1. Extraction Phase

When a new message pair $(m_{t-1}, m_t)$ arrives, Mem0 builds a rich prompt $P$ combining three sources:

A conversation summary $S$ — a compressed semantic snapshot of the entire history, refreshed asynchronously.
The last $m$ messages — fine-grained recent context (default $m=10$).
The new message pair itself.

An LLM extraction function $\phi(P)$ then produces a candidate set $\Omega = \{\omega_1, \omega_2, \ldots, \omega_n\}$ of salient facts from the new exchange.

Extraction Pipeline — hover each block

👆 Hover any block to learn its role in the pipeline.

2. Update Phase — The Four Operations

Each candidate fact $\omega_i$ is compared against the top-$s$ semantically similar existing memories (default $s=10$). An LLM "tool call" then decides which of four operations to apply. Rather than a separate classifier, the LLM's own reasoning determines the right action based on semantic relationship.

Click an operation above to see when it's triggered and an example.

3. Mem0^g — Graph-Based Memory

Mem0^g represents memories as a directed labeled graph $G = (V, E, L)$:

Nodes $V$: entities (Alice, San Francisco, …)
Edges $E$: relationships (lives_in, prefers, owns, …)
Labels $L$: semantic types (Person, City, Event, …)

A two-stage pipeline first runs an entity extractor then a relationship generator, both LLM-based. Conflict detection marks outdated relationships as invalid (rather than deleting them), enabling temporal reasoning. Retrieval uses a dual strategy: entity-centric graph traversal + semantic triplet matching.

Live Knowledge Graph — click nodes to explore

👆 Click any node to see its entity type and relationships.

Results

Evaluated on LOCOMOA benchmark of 10 long conversations (~600 turns, ~26K tokens each) with ~200 questions per conversation across 4 types: single-hop, multi-hop, temporal, open-domain. — 10 extended conversations (~26,000 tokens each) with ~200 questions per conversation. The primary metric is LLM-as-a-Judge (J)An LLM evaluates each response as CORRECT/WRONG by comparing to the ground truth, measuring factual accuracy, relevance, and completeness. Reported as mean ± 1 std dev over 10 runs., which better captures semantic correctness than lexical metrics like F1 or BLEU.

67.1%

Mem0 Overall J Score

vs 52.9% OpenAI, 58.1% LangMem

1.44s

Mem0 p95 Total Latency

vs 17.1s full-context (91% reduction)

Mem0 avg tokens per conversation

vs 26K full-context, 600K+ Zep

58.1%

Mem0^g Temporal J Score

Best on temporal questions; graph helps here

Limitations & Open Questions

Full-context still wins on accuracy

Despite Mem0's efficiency gains, the full-context approach (ingesting all ~26K tokens) still achieves the highest J score (~73%) compared to Mem0's ~67%. Memory compression inevitably loses some information. The gap narrows as conversations grow longer — where full-context becomes impractical — but for short conversations the trade-off favors full-context.

Graph memory doesn't help multi-hop queries

Surprisingly, Mem0^g underperforms plain Mem0 on multi-hop questions (J: 47.19 vs 51.15). The paper suggests "potential overhead or redundancy when navigating more intricate graph structures in multi-step reasoning scenarios." The added complexity of graph traversal may introduce noise rather than signal for these queries.

Small benchmark scale (10 conversations)

LOCOMO consists of only 10 extended conversations. While each is rich (~600 turns, ~26K tokens), generalization claims are limited. Results may not transfer to other domains (technical support, medical consultations) or other languages.

Latency overhead of Mem0^g graph construction

Mem0^g's p95 search latency (0.657s) and total latency (2.59s) are higher than plain Mem0 (0.200s, 1.44s). The paper identifies optimizing graph operations as a key future direction. For real-time interactive applications, this overhead may be prohibitive.

LLM-dependency for memory operations

Both extraction and update decisions rely on GPT-4o-mini. This creates a dependency on a proprietary model, adds cost per memory operation, and raises questions about consistency — the LLM's stochastic nature means identical inputs may produce different memory decisions across runs.

Glossary

Context Window

The maximum number of tokens an LLM can process in a single forward pass. Information outside this window is inaccessible. GPT-4: 128K tokens; Claude 3.7: 200K; Gemini 1.5: up to 10M.

RAG (Retrieval-Augmented Generation)

A technique that splits documents into chunks, embeds them as vectors, and retrieves the top-k most relevant chunks at query time to augment the LLM's prompt. Mem0 outperforms all RAG configurations tested.

LLM-as-a-Judge (J)

A separate, capable LLM evaluates each generated answer as CORRECT or WRONG by comparing it to the ground truth. More nuanced than F1/BLEU because it captures semantic equivalence (e.g., "May 7th" = "7 May"). Reported as mean ± 1 std dev over 10 independent runs.

LOCOMO Benchmark

Long-Context Conversational Memory benchmark (Maharana et al., 2024). 10 extended conversations, ~600 dialogues and ~26K tokens each, with ~200 questions per conversation across single-hop, multi-hop, temporal, and open-domain categories.

Vector Embeddings

Dense numerical representations of text in high-dimensional space, where semantically similar texts have nearby vectors. Used by Mem0 to find the top-s most relevant existing memories when deciding how to handle a new candidate fact.

p95 Latency

The 95th percentile of response times — 95% of requests complete faster than this value. A better measure of worst-case user experience than the median (p50). Mem0's p95 total latency is 1.44s vs 17.12s for full-context.

Knowledge Graph (Neo4j)

A graph database where entities are nodes and relationships are labeled edges. Mem0g uses Neo4j to store memory as triplets (source_entity, relationship, destination_entity), enabling structured traversal for temporal and relational queries.

Citation

@article{chhikara2025mem0,
  title   = {Mem0: Building Production-Ready AI Agents
             with Scalable Long-Term Memory},
  author  = {Chhikara, Prateek and Khant, Dev and
             Aryan, Saket and Singh, Taranjeet and
             Yadav, Deshraj},
  journal = {arXiv preprint arXiv:2504.19413},
  year    = {2025},
  url     = {https://arxiv.org/abs/2504.19413}
}