My coding agent keeps memory in flat markdown files. One fact per file, a frontmatter header, an index that loads at the start of every session. Ninety files, two weeks old, twenty five sessions.
Every one of those files makes claims about my machine. This path holds that project. This command exists. This credential lives under that id. Claims decay, and nothing in the format tells you when one goes bad. So I wrote a script to walk the memory and check each claim against the machine as it is right now.
The first run told me 45 of 90 files were rotten. Fifty percent.
That number was wrong, and the way it was wrong turned out to be the interesting part.
What is actually checkable
Not everything in a memory file can be verified. "The user prefers short answers" is not falsifiable by a script. But a surprising amount is:
- filesystem paths, checked with
os.path.exists - command names, checked with
shutil.which - credential ids, checked by asking the password manager whether the object still exists
- dates written as a shelf life, checked against today
- internal links between memory files, checked against the set of filenames
That is a decent net. The trouble is what else it catches.
False positive one: everything in backticks looks like a command
My first pass pulled any backticked token and asked whether it was on PATH:
RE_CMD = re.compile(r'`([a-z][a-z0-9_.-]{1,20})(?: [^`]*)?`')
Memory files are written by an agent for an agent, and they use backticks the way prose uses italics. So the auditor reported these as missing commands:
devto-comment-drafting-rules.md
missing commands comprehensive
missing commands robust
missing commands leverage
missing commands webdev
missing commands fastapi
Those are not commands. comprehensive and robust are words that file bans me from writing. webdev and fastapi are tags. One file about frontend geometry contributed h-11 and h-7, which are Tailwind classes.
The fix was to stop treating a lone word as a command. A command is a name plus an argument, and the name has to either be one I know is a CLI or carry a real flag:
RE_CMD = re.compile(r'`([a-z][a-z0-9_-]{1,20})\s+([^`]{1,60})`')
if c not in KNOWN_CLI and not re.match(r'-{1,2}[a-z]', rest.strip()):
continue
A hardcoded allowlist feels like cheating. It is not. The alternative is a tool that reports sql duplicate as a missing binary, and a tool that does that gets ignored within a day.
False positive two: a slash does not mean a file
The path check matched anything that started with a slash. Memory files are full of API routes:
datadog-key-hidden-consumer-log-drain.md
dead paths /health
dead paths /v2/integrations/log-drains
Neither is a file. Both are endpoints on someone else's server, and os.path.exists will say no about them forever.
I anchored the pattern to roots that exist on this machine:
RE_PATH = re.compile(
r'`(~/[^`\s]{2,}|'
r'/(?:Users|Applications|Volumes|opt|etc|usr|private|var|Library)/[^`\s]{2,})`')
Relative paths came out entirely. ./scripts/gate.sh was reported dead, and the file exists. It sits in a project directory, and the auditor was resolving it against its own working directory. A relative path is a claim about a context the checker does not have, so it cannot be checked. Guessing at it produces exactly one outcome, which is a wrong answer delivered confidently.
False positive three: a past date is not a lie
The date check flagged anything in the past. But a memory file is allowed to talk about the past. A note recording that something happened on the sixth of August is correct forever. A note saying a document is valid until the sixth of August stops being useful the moment that day passes.
Same date, opposite meaning, and the difference is the word in front of it:
RE_DEADLINE = re.compile(
r'(?:expires?(?:\s+on)?|valid\s+(?:until|through)|until|deadline|'
r'renew\w*\s+by)\s*(\d{2}\.\d{2}\.20\d{2})', re.I)
I also moved these out of the rot count entirely. An expired shelf life is not a false statement, it is a statement that has stopped being actionable. Mixing the two categories inflates the scary number, and the scary number was already doing enough damage.
Fifty percent to four
After those three fixes: 4 files out of 90.
ROTTEN: 4/90 files (4%)
------------------------------------------------------------------
dead paths 3 file(s)
gone from 1Password 1 file(s)
The credential one looked serious. A file about my project's secrets vault referenced an id, and the password manager said no such item.
I checked it by hand before believing it. The id names a vault, not an item. The vault is alive and holds six credentials. My script asked op item get, got a failure, and concluded the object was gone, when the correct question was whether the id resolves as either kind of object:
ok = False
for sub in (['item', 'get', i], ['vault', 'get', i]):
r = subprocess.run(['op'] + sub + ['--format=json'],
capture_output=True, text=True)
if r.returncode == 0:
ok = True
break
Three real findings out of ninety files. Two directories that no longer exist, and an application path off by one word, /Applications/Telegram.app where the disk holds Telegram Desktop.app.
The part where the tool caught me
Earlier the same day, before any of this existed, I hit what looked like textbook rot. A stored instruction told me a key lived in a specific vault under a specific id. I ran it and got:
could not find item 5wdv... in vault Private,
because it has been deleted or archived
Deleted or archived. I read that as deleted, wrote off the record as stale, and went looking for where the key had moved.
The record was never deleted. It is archived, still in that vault, renamed to mark that something replaced it. op item get retrieves it by id without complaint. op read refuses to resolve a path into an archived item, and reports the refusal with an either-or that contains the right answer and the wrong one, weighted equally.
My memory was not stale. I was, because I trusted an error message that had told me two things and let me pick.
What this actually costs
The measurable part held up. The index that loads on every session is 2,460 tokens across 89 entries, which is 27 tokens of permanent overhead per remembered fact. That is the real bill for file-based agent memory, and it is linear. Five hundred facts is roughly fourteen thousand tokens spent before anyone says hello. Vendors selling vector stores quote numbers in this range, and on my data they are not exaggerating.
The structure was healthier than I expected for something assembled in two weeks: 180 links between entries, zero broken, zero orphans.
So the headline I set out to write, that file-based memory quietly rots, is not supported by my own machine. Two weeks in, it barely rotted at all.
What I would tell you instead
A checker that flags half your files does not get tightened. It gets muted. The cost of a false positive is not one wasted look, it is the credibility of every finding after it, including the true ones sitting three lines down.
I went from 50% to 3% by making the tool refuse to guess: no bare word is a command, no slash is a file unless it starts somewhere real, no relative path gets resolved against a directory the checker invented, no date is a deadline without a word saying so. Every one of those was me deciding that reporting nothing beats reporting something shaped like an answer.
And the last false positive did not fall to a better regex. It fell because I checked a finding by hand and found my own script wrong. If you build one of these, budget for that step. The auditor is a claim about your claims, and it decays the same way.
The script is 260 lines of standard library Python, no dependencies. Point it at a memory directory:
memrot.py ~/.claude/projects/<project>/memory --op
The --op flag is 1Password specific and shells out read only. Drop it and the rest still runs.
Some filenames in the output above are edited. The memory holds personal records, and the structure was the point, not the contents.
