I Benchmarked 6 AI Agent Memory Strategies: Top Score, Worst Experience

Many agent memory systems accumulate duplicate and contradictory memories over time. To address this, I tested the memory designs of mem0, MemOS, and signetai, compared 6 dedup-and-update strategies, and arrived at a best practice that keeps memory quality high while balancing LLM call costs.

Agent memory comes in two main forms: file-based memory, where the agent directly reads and writes markdown files it maintains itself; and an external long-term memory system, where a backend calls an LLM to extract facts from conversations into a memory store, and later conversations retrieve relevant memories to inject into context. This article is about the latter. As memories accumulate, newly written ones inevitably duplicate or even contradict what's already in the store — whether dedup and update are done well directly determines whether stale, redundant, contradictory memories pollute the context and hurt the user experience.

The following case comes from my own OpenClaw sessions, with mem0 2.0.7 as the memory module. Across 4 conversations, I mentioned living in Seattle four times. After the conversations ended, I checked the memory store: the single fact "lives in Seattle" alone had produced 4 memories, each worded differently:

1. User recently moved to Seattle, as shared on September 17, 2026 (late Thursday night around 2:58 AM local time)
2. User recently moved to Seattle around September 16-17, 2026, coinciding with their recent job change
3. User is living in Seattle as of mid-September 2026, confirmed directly by the user on September 17, 2026
4. User currently lives in Seattle (as of September 17, 2026)

Here's how the memory store evolved as this accumulation happened:

Many memory systems support dedup and update, but handling near-synonymous, subset/superset, and contradictory relationships between memories remains hard, and there is no perfect solution.

In practice, the design involves choices along two dimensions:

The first dimension is deciding whether a newly extracted memory duplicates an old one: before storing a new memory, should the system check whether it duplicates something already in the store? And how do you find the duplicates?

The second dimension is what to do once a new memory is judged to be a duplicate: overwrite the old memory with the new one? Or call an LLM to merge the old and new into a single memory?

Different products choose different strategies. This article analyzes mem0, MemOS, and signetai as examples. The experiments are reproduced on the open-source project NeatMem (https://github.com/kanhaoning/NeatMem), and every command in this article can be run directly.

1. mem0

mem0 (open-source v2.0.7) has no standalone dedup step. Its dedup measure lives in the memory-extraction prompt, which asks the LLM to avoid extracting duplicates. Its extract-and-store pipeline looks like this:

1. Load conversation history

mem0 maintains a sqlite database of conversation records. Before each extraction, it loads the most recent 10 messages from sqlite into the extraction prompt, helping the LLM extract more complete memories with conversational context.

2. Retrieve related old memories

The new messages are embedded, and the 10 most relevant old memories are retrieved from the store and loaded into the extraction prompt — so the LLM can both use the old memories as context and avoid re-extracting what already exists.

3. Call the LLM to extract memories

A system prompt and a user prompt are built to call the LLM for extraction. The user prompt is the raw material for extraction, with this template:

## Summary
(a profile summary of the user from conversation history)

## Last k Messages
(the most recent 10 messages, for resolving references in the new messages)

## Recently Extracted Memories
(memories already extracted in this session, up to 20)

## Existing Memories
(the related old memories retrieved in step 2, 10 of them)

## New Messages
(the new messages this round; extracted content must come only from here)

## Observation Date
(the date the conversation happened; relative time expressions are resolved against it)

## Current Date
(today's date)

mem0 fills in only four of these sections: Last k Messages, Existing Memories, New Messages, and the two dates. Summary and Recently Extracted Memories are left empty on the default path (those two sections are only populated in mem0's closed-source version).

The system prompt contains a series of extraction requirements, of which three rules relate to dedup and one relates to updates:

• Rule 1:

Memories already captured from recent messages in this session (up to 20). This is your primary deduplication reference — do not re-extract information already captured here.

Recently Extracted Memories is the primary dedup reference: don't re-extract what's already been captured. But as noted above, the open-source version never passes this section in.

• Rule 2:

Memories currently in the system relevant to this conversation. … Use these ONLY for deduplication and linking — do NOT extract new memories from Existing Memories. If new information in New Messages is semantically equivalent to an Existing Memory with no meaningful new context, skip it.

Existing Memories may only be used for dedup and linking; if information in the new messages is semantically equivalent to an existing memory with no meaningful new context, skip it.

• Rule 3 (pointing the opposite way):

When in doubt, extract. A slightly redundant memory is far less costly than a missing one. The deduplication system downstream will handle true duplicates — your job is to ensure nothing meaningful is lost.

When unsure whether something is a duplicate, extract it anyway — a slightly redundant memory costs far less than a missed one, and the downstream dedup system will handle true duplicates.

• Rule 4 (for update scenarios):

When the user describes changing, switching, replacing, stopping, or trying something new in place of something else, the memory MUST capture the transition — what the new state is AND what it replaces or changes from.

When the user describes switching, replacing, or stopping something, the memory must spell out the full transition — the new state and what it replaces — in a single memory.

Rules 1 and 2 demand avoiding duplicates; rule 3 demands extracting more. But the "downstream deduplication system" promised by rule 3 doesn't actually exist in the pipeline — after extraction, memories are written straight into the store.

The direct consequence of this design: it can only avoid generating obviously redundant memories. New memories that are synonymous-but-reworded, subset/superset, or contradictory to old ones easily make it into the store — there's no backstop to intercept duplicates. The opening case is exactly this: the repeated "living in Seattle" messages were worded differently each time, were not judged as duplicates of existing memories at extraction, and so 4 duplicate "lives in Seattle" memories were extracted and stored.

But add-only has real advantages too. Lower risk of losing information, zero extra calls on the write path, the lowest latency and cost, and no need to maintain judgment-and-merge logic in the architecture. More importantly, rule 4 gives add-only a way to handle updates: the new memory spells out the full transition from the old one, so at recall time both old and new memories show up together, and the LLM — seeing the new memory — can tell the old one is stale and decline to use it in its answer. As for the cost of redundancy, it shows up when recalled stale memories mislead, and in the thinking tokens the LLM wastes sifting correct memories out of redundant, outdated ones. But the impact on benchmark accuracy (for example, mem0's LoCoMo evaluation recalls 200 memories by default for answering) is not significant: when the correct memory is recalled alongside stale and duplicate ones, the LLM can still find the right one and answer correctly.

2. How MemOS judges duplicates

In contrast to mem0, MemOS (2.0.23) makes duplicate judgment a standalone step: new memories are written to the store without any dedup check, and afterwards a background thread calls the LLM on each new memory to judge whether it duplicates an existing one, executing merges, archiving, and so on based on the verdict. Note this background thread is off by default (reorganize=False); the pipeline below only runs when it's enabled. The judgment flow:

1. Candidate pair prefiltering

For each new memory, vector similarity first recalls a batch of candidate old memories whose similarity reaches a threshold (hardcoded at 0.8); each new-old pair is then judged by a separate LLM call.

2. Three-way classification

For each candidate memory, a single LLM call judges which of three relationships it has with the new memory. The prompt defines them as follows:

contradictory: The two statements describe the same event or related aspects of it but contain factually conflicting details.

redundant: The two statements describe essentially the same event or information with significant overlap in content and details, conveying the same core information (even if worded differently).

independent: The two statements are either about different events/topics (unrelated) OR describe different, non-overlapping aspects or perspectives of the same event without conflict (complementary).

3. RESOLVER merging

New-old pairs judged contradictory or redundant go through one more LLM call. The rules in the RESOLVER prompt:

If the statements are redundant, merge them by preserving all unique details and removing duplication, forming a richer, consolidated version.

If the statements are contradictory, attempt to resolve the conflict by prioritizing more recent information, higher-confidence data, or logically reconciling the differences based on context. If the contradiction is fundamental and cannot be logically resolved, output No.

When the LLM reconciles successfully, the old and new memories are merged into a single new memory that is returned and stored; the two old memories are marked archived, keeping a link to the merged result for traceability. When the contradiction can't be reconciled, the LLM returns No, and the older entry is deleted by timestamp — outright deletion.

MemOS's three-way classification is finer-grained than a binary "duplicate or not" judgment, with contradictions and redundancy handled separately, and old memories archived rather than deleted. But the fine-grained classification has a prerequisite: duplicate pairs must first pass the similarity threshold to enter judgment and subsequent handling. Under this architecture, choosing the similarity threshold is a hard problem — set it too high and large numbers of duplicates slip through unprocessed; set it too low and LLM call volume grows significantly. §6 investigates this empirically.

3. How signetai judges a whole batch at once

The third product, signetai (0.123.22), sits at another combination: each newly extracted memory retrieves a batch of candidate old memories, and the new memory plus the whole batch go into a single dedup-judgment prompt, which decides whether any candidate duplicates the new memory and, if so, picks out the one; the new memory then replaces the identified old memory. The full write path:

1. Call the LLM to extract facts

Facts and entities are extracted from the conversation. Unlike mem0, the extraction prompt contains no dedup requirements — it only extracts.

2. Surprisal gating (no LLM call)

Before writing, compute the maximum cosine similarity between the new memory and existing memories of the same type (preferences, decisions, events, etc., distinguished by type labels at write time); surprisal = 1 − max similarity. If surprisal is below the threshold (i.e., the store already holds something nearly identical), the new memory is dropped outright and never enters the later stages. Constraints, errors, and decisions pass through directly. This layer is pure vector-computation near-duplicate interception with zero model calls.

3. Retrieve candidates, decide the action in one call

For each fact that passes the gate, hybrid BM25 + vector retrieval recalls the 5 most relevant old memories; these are loaded into a prompt together with the new memory for a single LLM call that outputs an action directly. The action definitions in the prompt:

• add: New fact has no good match, should be stored as new memory
• update: New fact supersedes or refines an existing candidate (specify targetId). Ensure the merged result is self-contained
• delete: New fact contradicts/invalidates a candidate (specify targetId)
• none: Fact is already covered by existing memories, skip

When the verdict is update or delete, the call must also return the target old memory's id, indicating which memory is to be updated or deleted, for the next step.

4. Overwrite execution

On an update verdict, no LLM call is made to merge memories — the new memory overwrites the old memory's content wholesale. Before overwriting, the old version's entire row is saved as a complete JSON snapshot into a separate cold-storage table, queryable afterwards. A delete verdict likewise snapshots first, then soft-deletes. What gets written is the fact text exactly as extracted in step 1 (the judgment stage outputs only the action and target id, no generated text), so how much of the old memory's detail is kept depends entirely on how much detail the new memory itself contains.

4. Strategy summary

mem0 doesn't judge duplicates at all; MemOS judges pair by pair and merges; signetai judges a whole batch and overwrites duplicates. Returning to the two dimensions from the introduction, here are the strategies along each.

Three common approaches on the judgment dimension:

• Add-Only (no duplicate judgment): the prompt asks the LLM not to extract memories that duplicate existing ones, and everything extracted is written directly. No extra call cost, low risk of information loss; the price is that a prompt has limited binding power over duplicate-free extraction, and redundancy keeps accumulating in the store.
• PointWise (judge pair by pair): each newly extracted memory is compared against each highly similar old memory in a separate call, classifying the relationship as contradict / redundant / independent. Finer judgment granularity; the price is call volume growing multiplicatively with the number of candidate pairs (concrete numbers below).
• ListWise (judge a whole batch): before writing, the new memory and the retrieved batch of old memories go into a single LLM call that decides everything at once: whether a duplicate exists, which memory it is, and what to do about it. One call per batch, low cost.

The call-volume difference between PointWise and ListWise:

On the handling dimension, there are two things to do once a duplicate is found:

• Replace (overwrite): the new memory replaces the old one wholesale. Simple to implement, but old details the new memory doesn't mention are lost along with it.
• Rewrite (merge): an LLM merges the old and new memories into one. The most complete detail preservation; the price is one extra merge call per update, and merge quality varies with the model.

5. Benchmark overview of each approach

To isolate interference from models, prompts, and retrieval implementations — and attribute differences precisely to the "dedup strategy" itself — I reproduced everything on the open-source project NeatMem. NeatMem is a memory system in the same category as mem0, and it exposes the judgment method (Add-Only / PointWise / ListWise), the handling method (Replace / Rewrite), and several other strategies as independent parameters, making it possible to compare the approaches on a single code base. I also added NeatMem's own dedup scheme (multi-target ListWise + Rewrite, detailed in §8).

Experiment setup: the LoCoMo long-conversation QA benchmark, with identical write, embedding, and scoring models throughout; each configuration ran 5 independent times and results were averaged, with every run starting from an empty memory store and re-ingesting all conversations before evaluation; on the retrieval side, reranking was uniformly disabled with top 200 returned, matching mem0's evaluation configuration; the QA and scoring prompts also follow mem0's evaluation exactly — so the only difference left is the dedup strategy itself. Each of the four approaches corresponds to one evaluation command (model keys must be configured first — see appendix item 2, just 5 export commands; the LoCoMo dataset ships with the PyPI package, no separate download needed):

Add-Only (mem0 approach)

# dedup fully off; extraction results written straight to the store; each of the four configs gets its own output directory
neatmem evaluate --runs 5 --top-k 200 \
  --rerank off \
  --no-dedup \
  --output-dir runs/add-only

PointWise + Rewrite (MemOS approach)

# judgment: pairwise; handling: LLM merge; candidate recall threshold 0.8
neatmem evaluate --runs 5 --top-k 200 \
  --rerank off \
  --dedup-detector pointwise \
  --dedup-resolver rewrite \
  --dedup-recall-threshold 0.8 \
  --output-dir runs/pointwise-rw-08

ListWise + Replace (signetai approach)

# judgment: whole batch; handling: wholesale overwrite; candidate recall threshold 0.4
neatmem evaluate --runs 5 --top-k 200 \
  --rerank off \
  --dedup-detector listwise \
  --dedup-resolver replace \
  --dedup-recall-threshold 0.4 \
  --output-dir runs/listwise-replace

ListWise multi-target + Rewrite (NeatMem, default config)

# dedup on defaults: multi-target judgment + merge + threshold 0.4
neatmem evaluate --runs 5 --top-k 200 \
  --rerank off \
  --output-dir runs/listwise-mt-rewrite

Results:

The four scores fall in a 0.895–0.907 band, and the top one comes from Add-Only, which does no dedup at all. The gaps between approaches are the same order of magnitude as run-to-run variance of a single configuration — the benchmark score can't distinguish dedup strategies. This is exactly the starting point of this article: the basis for choosing a dedup strategy isn't only the evaluation score, but also whether synonymous duplicates pile up in the store, whether stale facts get updated, and whether old details are lost when updates happen. The next three sections analyze the update approaches one by one.

6. PointWise (MemOS approach) in depth

PointWise (the MemOS approach) is the finest-grained of the three judgment methods: each candidate pair gets its own LLM call, the model outputs only a contradict / redundant / independent label, and code executes the merge or archival by label. By design it comes closest to thorough updating, but in practice the similarity threshold for recalling candidate duplicates is hard to tune: too high and duplicates slip through unprocessed; too low and large numbers of pairs get judged, LLM call volume climbs, and the evaluation score drops with it.

Hands-on case

A minimal update scenario: the user first says they've long lived in Seattle, then announces a move to Austin.

Reproduce it directly with NeatMem's demo command (each --say is an independent write; model and key configuration same as §5, see appendix item 2), running PointWise + Rewrite with recall threshold 0.8 (same tier as MemOS):

neatmem demo \
  --dedup-detector pointwise \
  --dedup-resolver rewrite \
  --dedup-recall-threshold 0.8 \
  --say "I currently live in Capitol Hill, Seattle — I've been in this apartment for over three years and I'm pretty used to living here" \
  --say "I moved to Austin last month — renting an apartment in Zilker now, much closer to work"

The second message makes the "currently live in Seattle" statement from the first one stale, yet after the run, two contradictory memories coexist in the store:

[1] User moved from their Capitol Hill, Seattle apartment to Zilker, Austin in August 2026, renting a new apartment that is much closer to their workplace

[2] User lives in an apartment in Capitol Hill, Seattle, and has been residing there for over three years (since before June 2023), feeling well-adjusted to the neighborhood

This outcome is identical to doing no dedup at all, but for a different reason: dedup's first step is finding possibly related old memories by similarity, and only pairs above 0.8 get an LLM judgment. This pair's similarity lands around 0.71–0.75 — below the gate — so the judgment step is skipped entirely and the new message goes in as a new memory. Here's how the store evolved on this run:

Contradictory statements are often worded very differently (one says Seattle, the other Austin), so their similarity is naturally low. This is the recall blind spot: the old memory that should be judged never enters the candidate set because its similarity didn't pass the gate — no matter how accurate the judgment, the pair has to be found first.

Rerunning the same scenario with the threshold lowered to 0.4:

neatmem demo \
  --dedup-detector pointwise \
  --dedup-resolver rewrite \
  --dedup-recall-threshold 0.4 \
  --say "I currently live in Capitol Hill, Seattle — I've been in this apartment for over three years and I'm pretty used to living here" \
  --say "I moved to Austin last month — renting an apartment in Zilker now, much closer to work"

The judgment hits, and the store merges into a single memory:

User moved from Capitol Hill, Seattle to Zilker, Austin in approximately August 2026, renting an apartment that is much closer to their workplace, after having resided in an apartment in Capitol Hill, Seattle for over three years (since before approximately September 2023) where they were well-adjusted to the area.

The same scenario at threshold 0.4, with the judgment hitting and the two memories merging into one:

Score and LLM call volume analysis

Lowering the threshold eliminates the blind spot, but introduces two other costs. First, call volume: PointWise's judgment calls are highly sensitive to the threshold — about 2k per LoCoMo run at 0.8, rising to about 24k at 0.4, a ~12x increase (ListWise at the same threshold is about 5k). Second, the evaluation score:

All told, PointWise has a recall blind spot at high thresholds, and bears higher cost plus score loss at low thresholds — neither direction is satisfactory. The remaining way out is switching the judgment method: signetai's combination (ListWise batch judgment + Replace wholesale overwrite, §3) judges the new memory against the whole candidate set in one LLM call to find duplicate old memories — at the same threshold, judgment calls are about a fifth of PointWise's, and even the merge call after a hit is saved. Can it solve both the blind spot and the cost? The next section tests it on the same set of scenarios.

7. ListWise (signetai approach) in depth

Hands-on case

The same moving scenario, rerun with the signetai combination at threshold 0.4 (two switches flipped relative to the previous command: detector to listwise, resolver to replace — the former targets the blind spot and call volume, the latter follows signetai's default update handling):

neatmem demo \
  --dedup-detector listwise \
  --dedup-resolver replace \
  --dedup-recall-threshold 0.4 \
  --say "I currently live in Capitol Hill, Seattle — I've been in this apartment for over three years and I'm pretty used to living here" \
  --say "I moved to Austin last month — renting an apartment in Zilker now, much closer to work"

After the judgment hits, an overwrite update executes, and the store converges to a single memory, verbatim:

User relocated from their apartment in Capitol Hill, Seattle to a rental apartment in Zilker, Austin in August 2026, motivated by being much closer to work

The whole batch of candidates is judged in one LLM call; after a hit, no LLM merge happens — the new memory's original text overwrites the old memory wholesale. At threshold 0.4 the recall blind spot is largely eliminated, though not one hundred percent: occasionally a candidate doesn't pass the threshold, or makes it into the batch but is still judged as new. Here's how the store evolved on this run:

The blind-spot problem is solved; the next subsection answers the cost question with data. The real price lies in the overwrite action itself.

In the moving scenario above, the historical detail "over three years" disappeared with the overwrite — a detail directly related to that update. There's a more insidious kind of loss: content in the old memory that has nothing to do with the current update also vanishes along with the wholesale overwrite.

A breakfast-routine update scenario verifies this: the user first describes their breakfast routine (making pour-over coffee at home with toast, listening to Spanish podcasts while eating — they're learning Spanish for a work transfer), then says the coffee machine broke and is in for repair, so these mornings they grab an Americano at the coffee shop downstairs instead. The Spanish learning is unrelated to this update, but it lives in the same memory.

Running with replace first:

neatmem demo \
  --dedup-detector listwise \
  --dedup-resolver replace \
  --dedup-recall-threshold 0.4 \
  --say "I make pour-over coffee at home with toast every morning, listening to Spanish podcasts while I eat — I'm learning Spanish for a work transfer to Mexico City next spring" \
  --say "My coffee machine broke and is in for repair, so these mornings I grab an Americano at the coffee shop downstairs instead — still with toast"

When the verdict is update, what replace actually writes, verbatim — the old memory's "listening to Spanish podcasts during breakfast" doesn't make it into the new text:

Old: User starts every morning with pour-over coffee and toast at home while listening to Spanish podcasts during breakfast

New: User's coffee machine broke and is currently in for repair, so they temporarily switched from making pour-over coffee at home to grabbing an Americano at a coffee shop downstairs for their morning coffee, while still eating toast with it

The memory is replaced wholesale, and "listening to Spanish podcasts" disappears from it. (This case has some randomness: if "learning Spanish" gets extracted as a separate memory, it stays in the store.) How the store evolved on the replace run:

Rerunning the same scenario with rewrite (the only difference in the command is --dedup-resolver):

neatmem demo \
  --dedup-detector listwise \
  --dedup-resolver rewrite \
  --dedup-recall-threshold 0.4 \
  --say "I make pour-over coffee at home with toast every morning, listening to Spanish podcasts while I eat — I'm learning Spanish for a work transfer to Mexico City next spring" \
  --say "My coffee machine broke and is in for repair, so these mornings I grab an Americano at the coffee shop downstairs instead — still with toast"

With the same update verdict, the merged result preserves the old details:

The coffee machine broke around September 2026 and is currently in for repair, so the user temporarily switched from making pour-over coffee at home to grabbing an Americano at the coffee shop downstairs each morning, still paired with toast; previously, the morning routine had included making pour-over coffee with toast every morning while listening to Spanish podcasts during breakfast.

How the store evolved on the rewrite run — old details preserved in the merged memory:

These two cases show the listwise judgment step did nothing wrong — both runs correctly identified this as an update to that breakfast memory. The detail loss comes from replace: overwriting goes through no LLM merge, so whether old details are kept rides entirely on the extraction stage — if the new memory happens to be written completely enough, they stay; if not, they vanish with the overwrite.

Score and LLM call volume analysis

Back to the cost question from the end of the last section: with more candidates, why does call volume stay manageable? PointWise's judgment count grows linearly with the number of candidate pairs — every new memory gets a separate LLM call against every candidate old memory. ListWise folds the same batch of candidates into one call, decoupling judgment count from candidate count. Replace additionally saves the merge call after a hit. Measured call volumes per configuration:

Dedup calls run from 1x extraction calls (ListWise at 0.8) to 18x (PointWise at 0.4) — the bulk of the whole pipeline's cost.

On cost, PointWise loses both ways: threshold 0.8 keeps call volume low but has the recall blind spot; dropping to 0.4 eliminates the blind spot at the price of judgment calls rising to nearly 5x ListWise at the same threshold.

On score, replace and rewrite differ by about 1 point (89.52% vs 90.47%). That 1 point, plus the detail preservation seen in the hands-on cases, is what rewrite's extra merge calls get you. Compared with no-dedup's 90.68%, both are within run-to-run variance. The LoCoMo score can't tell these configurations apart; the real difference is in what ends up in the memory store.

8. Multi-target ListWise (NeatMem approach) in depth

Even after switching to ListWise, one problem remains in the judgment step: how many candidate memories can a single LLM call mark for update. The difference lies in the prompt's output format — single-target ListWise returns one JSON object, hitting at most one memory: {"action": ..., "targetId": ...}; multi-target ListWise asks one LLM call to evaluate every candidate and return all hits at once: {"judgments": [{...}, ...]}. When one new memory makes multiple old memories stale at the same time, single-target ListWise can only update one of them. Merging differs accordingly: when multi-target hits several memories, a single merge LLM call combines the new memory and all hit old memories into one, instead of several sequential pairwise merges. The handling difference between the two ListWise + Rewrite variants:

Hands-on case

A scenario to verify this: the user first builds up two separate habits, an Americano every morning and buying coffee at the shop downstairs from the office. Then a single message overturns both at once: they've switched to tea, no more Americanos, and they haven't been to that coffee shop in ages.

Running ListWise + Rewrite (single-target) at threshold 0.4:

neatmem demo \
  --dedup-detector listwise \
  --dedup-resolver rewrite \
  --dedup-recall-threshold 0.4 \
  --say "I drink an Americano every morning" \
  --say "I often buy coffee at the coffee shop downstairs from my office" \
  --say "I've recently switched to tea — no more Americanos, and I haven't been to the coffee shop downstairs in ages"

The result updates only one memory: the Americano entry is updated, while the coffee-shop entry stays in the present tense, directly contradicting the updated one:

[1] User often buys coffee at the coffee shop located downstairs from their office ← still present tense, contradicts [2]

[2] User recently switched from drinking an Americano every morning to tea and no longer visits the coffee shop downstairs from their office.

How the store evolved on this run — one judgment hit only memory 1, and memory 2 was missed and stayed:

Rerunning with the detector switched to the multi-target variant (multi-target is already the default configuration; the flag is passed explicitly here only for a clear comparison):

neatmem demo \
  --dedup-detector listwise_multitarget \
  --dedup-resolver rewrite \
  --dedup-recall-threshold 0.4 \
  --say "I drink an Americano every morning" \
  --say "I often buy coffee at the coffee shop downstairs from my office" \
  --say "I've recently switched to tea — no more Americanos, and I haven't been to the coffee shop downstairs in ages"

A single call judges both old memories as update targets, and a following single merge call combines the new memory and both old ones into one memory written to the store. After the scenario, only one memory remains:

[1] As of September 2026, the user switched from their previous habit of drinking an Americano every morning and often buying coffee at the coffee shop located downstairs from their office to drinking tea instead, and no longer visits that coffee shop as part of their routine.

How the store evolved through the whole process — two memories land one after another, the third message hits both in one judgment call, and one more merge call combines all three into one:

Score and LLM call volume analysis

The multi-target variant's judgment count is essentially flat versus single-target: one call judges all candidates, and more targets don't add judgment calls. What grows is merge calls: when one judgment hits multiple targets, one merge call combines the new memory and all hit targets into one:

(Figures are means over 5 LoCoMo runs, same methodology as the §7 table.)

Judgment calls are essentially flat, merge calls are 1.6x single-target, and the total is still about a quarter of PointWise at the same threshold (27,186); the scores are tied, the gap within run-to-run variance — more thorough updating costs nothing extra.

9. Conclusion

Back to the two dimensions from the introduction — how to judge, how to handle — with the full measured picture side by side:

"/>
Three observations:

1. A higher score doesn't mean a better experience. The two highest-scoring configurations are Add-Only, which does no standalone dedup (90.68%), and ListWise at threshold 0.8 (90.75%) — the latter scores highest but its threshold is so strict that some old memories that should be merged never enter the dedup candidate list. In practice, stale memories left in the store get recalled for answers and noticeably degrade the experience.
2. The cost difference lies in the judgment method. PointWise makes a separate LLM call for every recalled candidate; once the threshold drops, costs can spiral. ListWise puts all candidates into a single LLM call — no matter how many are recalled, it costs one call — so at the same threshold its total LLM call volume is about a fifth of PointWise's, and lowering the threshold to eliminate the recall blind spot only raises it modestly (from 1,263 calls at 0.8 to about 5k at 0.4).

My choice is multi-target ListWise + Rewrite: judgment cost on par with single-target, scores tied, and the single-target incomplete-update problem largely eliminated; Rewrite's merge calls buy the best possible preservation of old-memory details. NeatMem can be installed directly with pip install neatmem; neatmem serve starts a local service, and from Python you use a client whose API and parameters are compatible with mem0 — an existing Python mem0 project migrates by swapping import mem0 for import neatmem and pointing the client address at the local service URL; the default configuration is exactly the scheme in this article. It can also plug in directly as the memory backend for Claude Code, OpenClaw, or Hermes (plugin installation in the README). The code and reproduction instructions for every case in this article are open-sourced on GitHub — if you found this article helpful, a star means a lot: https://github.com/kanhaoning/NeatMem

Appendix: reproduction notes

All experiments in this article are based on NeatMem v0.5.8 and are fully reproducible. Experiment configuration: BM25 on, entity off, reranking off (explicitly specified as --rerank off in the commands), thinking off; everything except --rerank off is a v0.5.8 default — if defaults change in later versions, the repository CHANGELOG prevails:

1. Install: pip install "neatmem[nlp]" && python -m spacy download en_core_web_sm (the nlp extra provides lemmatization for BM25; this article's scores were measured under this configuration — a plain pip install neatmem also works, with BM25 degrading to raw word matching), or install from GitHub source with pip install ..
2. Configure: set model keys in the terminal:

   export LLM_PROVIDER=minimax
   export LLM_API_KEY=your-key
   export LLM_MODEL=MiniMax-M3
   export EMBEDDER_PROVIDER=siliconflow
   export EMBEDDER_API_KEY=your-key

For long-term use, write them into a .env in the working directory — same fields (template at the repository root in .env.example). The write, answer, and scoring models in this article's experiments are all MiniMax-M3, and the embedding model is SiliconFlow's BGE-M3. Supported LLM providers also include deepseek / dashscope / zhipu / moonshot / volcengine / openai / gemini / openrouter / siliconflow and OpenAI-compatible custom endpoints; embedding providers also include openai / dashscope / xinference (local) — swap the corresponding provider/key/model and you're set; the full configuration list is in the README. Other models will run too, but absolute scores will shift; the trends should still be a useful reference.

1. Dataset: the LoCoMo-10 evaluation set ships with the PyPI package; neatmem evaluate loads it by default, no separate download needed.
2. Case reproduction: the neatmem demo commands in §6–§8 run directly; each --say is an independent user-message input, and the final memory store contents are printed when the run finishes.
3. Score reproduction: the four neatmem evaluate commands in §5 — --runs 5 means 5 independent runs averaged; --rerank off --top-k 200 is this article's uniform retrieval methodology (reranking off, 200 memories recalled). Each command has a different --output-dir, so results, logs, and config manifests land in their own directories and the four runs don't interfere with each other (rerunning the same command after an interruption resumes automatically). Mind your quota: all the tables in this article add up to roughly 260k LLM calls. The default concurrency is fine (this article's data was produced with --max-workers 20 behind a multi-key proxy; with a single key, turning it up is not recommended — you'll hit rate limits). For a low-cost environment check first, add --limit 1 --runs 1: this uses only the first long conversation of the dataset (10 in total) for a single run, at a few percent of the full call volume, with scores deviating from the full run.
4. Plugging into an agent: neatmem serve listens on http://localhost:8790; on the Python side, MemoryClient(host="http://localhost:8790"), with an interface shape consistent with mem0's client; you can also call the HTTP endpoints directly (/v1/memories/ etc.).