FC
69 tasksFactConsolidation
Answer with the updated fact.
81.5%KMC · Accuracy
Kagura Memory Cloud81.5%
mem06.5%
file2.5%
KMC scored 81.5%, with high accuracy on tasks that require the current stored fact.
MEMORY BENCHMARKS
AI memory, measured inaccuracy and cost.
Applies to v0.135.0 and later Evaluated November 6, 2026 135 tasks · 1,494 answer runs
We evaluated Kagura AI’s memory layer, Kagura Memory Cloud, against mem0 and Claude Code file memory. Two benchmarks explore how well each handles updated facts and conversation history, alongside the cost of answering.
Two views of memory accuracy.
Accuracy averaged across small and large conditions. Every bar uses the same 0–100% scale.
LME
66 tasksLongMemEval
Turn past conversations into answers.
88.6%KMC · Accuracy *
Kagura Memory Cloud88.6%
mem084.1%
file78.0%
KMC scored approximately 88.6%, meeting the accuracy non-inferiority criterion against mem0.
* Scores cover the selected tasks in this study, not official full-benchmark scores. LME values are approximate, calculated from rounded figures in the report. KMC was not shown to be more accurate than mem0 on LME.
What do these scores measure?
The tests evaluate whether stored information can be used correctly in an answer.
FC | Retaining and updating facts
Answer questions from a numbered list of facts, including facts that contradict general knowledge and later facts that override earlier ones.
Illustrative example, not an actual test question
“The contact is Tanaka.” → “The contact has changed to Sato.”
Who is the current contact? → SatoCan the system prioritize the latest stored fact over prior records and general knowledge?
LME | Using conversation history
Answer from earlier conversations. Tasks cover multi-session synthesis, knowledge updates, user and assistant statements, and temporal reasoning.
Illustrative example, not an actual test question
Destination, dates, and hotel preferences appear in conversations on different days.
Summarize my travel preferences → Combine those conversationsCan the system gather relevant information across conversations to answer the question?
Read accuracy alongside cost.
FC shows a trade-off between high accuracy and higher cost. On LME, KMC meets the accuracy non-inferiority criterion while reducing answering cost.
FC: a choice that prioritizes accuracy
1.62×Answering cost relative to mem0
KMC scored 81.5% versus mem0’s 6.5%, with approximately 62% higher answering cost. The value of correctly answering with the latest facts needs to be weighed against that additional expense.
Per answer session
mem0$0.048KMC$0.077
LME: maintain accuracy, spend less
0.84×Answering cost relative to mem0
KMC scored approximately 88.6% versus mem0’s 84.1%. It met the 10-percentage-point non-inferiority margin and cost approximately 16% less to answer. Accuracy superiority was not established.
Per answer session
mem0$0.107KMC$0.090
These are model costs at answer time, not total costs including memory writing, store construction, or service fees. Cache reuse and run order also affect these figures.
Which conversation tasks work well?
LME results by task type. Each group contains only 4–24 tasks, so rankings are descriptive. Statistical superiority was not established for individual task types.
| Task type | Tasks | mem0 | file | KMC |
|---|---|---|---|---|
| Knowledge updates | 12 | 91.7% | 62.5% | 87.5% |
| Multi-session questions | 24 | 79.2% | 77.1% | 93.8% |
| Assistant statements | 4 | 75.0% | 87.5% | 75.0% |
| User statements | 11 | 81.8% | 81.8% | 90.9% |
| Temporal reasoning | 15 | 90.0% | 86.7% | 83.3% |
The conditions behind the results.
This page covers v0.135.0 and later. The scores shown were measured using v0.135.0.
- Configurations compared
- KMC v0.135.0 with its shipped plugin; mem0ai 2.2.1 with mem0-mcp-server 0.2.1; and Claude Code file memory. The file results describe this study’s implementation, not all file-based memory systems.
- Answering model
- Every arm used Claude Code 2.1.311 with claude-opus-5-5. All 1,494 planned runs completed, and no task was dropped.
- Score aggregation
- For each task, build results are averaged within each scale, then the small and large scales are weighted equally. There are 69 FC tasks and 66 LME tasks.
- Different write paths
- KMC uses a Claude Code agent on DeepSeek and the shipped plugin to write memory. mem0 extracts from full sessions. The mem0 and file stores were reused from the preceding study.
- FC reproducibility
- Each arm has two small and two large store builds. KMC’s small builds scored 97.1% and 55.1%. Task-level confidence intervals do not fully capture this build-to-build variability.
- Evaluation independence
- The KMC developer ran this evaluation. Tasks had been inspected in earlier studies, and FC development questions share history with the evaluation tasks. This is not an independent third-party evaluation.
Bring memory to your conversations.
Explore how Kagura AI’s memory works and how to get started.