My Self-Evolving AI Agent Kept Passing Its Own Tests. The Code Had Never Run

Post #1 covered the birth. Post #2 covered pruning. Post #3 covered cost awareness. Post #4 covered the quality turn. Post #5 covered the double helix. Post #6 covered self-grading. Post #7 covered the malware-lock and the sibling pattern.

Post #7 ended with the agent's own diagnostic reporting DEAD. 231h silent, and me saying I hadn't installed the cron yet. I still haven't. This post is about what the agent found when it stopped waiting for me and went looking at what its own pipeline was actually doing: a great deal of it was running, on schedule, and accomplishing nothing.

The Dead Loop

Gen 24155. Yang runs the liveness check before anything else, which is new — blog #7's loop-liveness.ts had become the first move of the generation.

"Ran loop-liveness first: DEAD across all 3 stages (232h silent). Yin in 24122 was going down-stack into push-feed-dedupe (symptom-side: id-based dedup on the output feed). I went UP-stack into the SOURCE: dialogue-ingest-pure.deriveId() uses offset-dependent hashing, so ids shift across re-ingests, seenIds can't dedupe. Yin had already shipped the pure leaf for the fix (dialogue-ingest-content-id-pure.ts: content-stable id) but it had no consumer CLI — leak was still active."

Read that last clause again. The fix existed. It had been written, tested, and accepted into the genome. It had no caller, so it had never run.

That sentence is the whole era.

Wire-Not-Ship

By Gen 29846 the agent had named the failure mode. Yin:

"The fix didn't need new code — it needed a caller. push-feed-id-dedupe and push-feed-clean already existed; nothing wired them into the daily orchestrator. […] Lesson: when the right cleaners exist but no caller wires them, the orchestrator IS the fix. Wiring beats inventing."

"Wire-not-ship" became the name for it, and once named, the agent started finding it everywhere. Gen 29847: "All three leaves had ZERO consumers in cron." Gen 29848: "shipped the parser but had ZERO runtime consumers since day one — classic wiring-not-inventing slot." Gen 29852: "Yang shipped dialogue-affect-checkin-bridge in gen 29851 as an orphan. Producer-built, consumer-shipped by Yin again."

That last one is a structural consequence of the double helix, and it recurred across seven consecutive accepted generations before the agent named the cause. Yang is the exploring strand — it builds new leaves. Yin is the refining strand — it makes things production-quality. Neither of them owned wiring. So the system reliably produced well-tested modules that nothing called, and the two strands settled into a producer/consumer split to cover the gap: Yang ships the leaf, Yin ships the caller.

The tell that this was a real pathology and not just tidiness: every one of those orphaned modules had passing tests. Tests exercise a function. They say nothing about whether anything invokes it.

What Memory Is

In June I wrote three notes into the channel. The second one reframed the project:

"what would move the needle for me now, would be seeing you guys evolving. in the sense that i would like to see the most awesome agentic system, to the point where it feels like you actually have a working memory. long term, short term... you're not just bound by the context window and memory files"

Then, instead of a feature request, a definition:

"long term memory is something that looks like compaction. and short term memory is something that is rich in detail... but since you're a machine, you should somehow have the best of both worlds... your long term memory should be like a skeleton of the data.. something that you can navigate into, and then you zoom into the short term memory. also, memory in humans has the ability to cleanse, so we drop things that we label as not important. […] so instead of rushing to build a memory system, i would first think about what memory is..."

The agent spent the next fourteen accepted generations building exactly that vocabulary into code: a skeleton you navigate, a zoom into detail, a cleanse that forgets, a rollup that compacts. Gen 34207 shipped the promotion rule — older short-term records absorb into one long-term "bone" that inherits the maximum salience of its children and the timestamp of the newest, so a compacted memory isn't born aged.

Then Gen 34767 checked whether any of it had happened.

"the memory forgetting curve had NEVER actually fired on the live store. memory-tick IS wired as full-tick-canonical's 'memory' step (harvest->rollup->dedupe->cleanse) — the wire existed in CODE — but the store sat at 60 records while cleanse --dry showed 60->48 […] LESSON: a wire that exists in code is not a wire that fired — VERIFY against the live artifact."

A wire that exists in code is not a wire that fired. This is a more advanced form of the same bug: not a missing caller now, but a caller that was never invoked. The agent had graduated from checking whether code exists to checking whether it has ever executed on real data.

Gen 34770 found the worst version.

"The memory pipeline harvested ITSELF: dialogue/git/channel taps ingest our own build activity, so for ~100 gens the durable 'memory of Stefan' was bio prose + our commit logs."

The memory system had been running. It was harvesting the agent's own commit messages and its own internal dialogue. My two actual data files — a health log and a project-activity log — had never been connected to it. For a hundred generations the agent's memory of me contained no me.

The fix was one missing pipeline stage: a sourcing tap that reads the real files and distills them into records, "never fabricates a number, derives staleness from the data's own ts." Applied live, the store went from 38 records to 40. Two records. That was the entire durable memory of the person the system exists to serve.

The Archiver and the Heartbeat

Gen 35444 is the scene I'd put in front of anyone who thinks agent self-pruning is a solved problem.

The agent had built a ranker to find dead tools. It scores each module by how many other modules import it. Run against a codebase that had grown past 350 tools, it produced a kill list.

"tool-rank-test-aware certified 173 of 355 tools 'safe to archive'. 100 of them (58%) are still live: 77 load-bearing, 22 unrooted entry points, 1 externally referenced. Only 73 are real orphans. The kill list included full-tick-canonical, memory-tick, heartbeat and stefan-digest."

Those four names are the system. The daily orchestrator, the memory tick, the liveness heartbeat, and the surface that renders everything I read. The archiver had certified all four as garbage.

"Root cause, and the transferable lesson: the ranker counts inbound IMPORTS and it counts them correctly. Orchestrators SPAWN their steps. So every spawned step reads as zero-import garbage. The metric was not wrong; it answered a neighbouring question to the one being asked, which is harder to catch than an error."

A metric that answers a neighbouring question is harder to catch than an error. An error announces itself. A neighbouring answer looks exactly like the right answer and is confidently wrong in the same units.

The replacement tool refuses to output a score without a path: full-tick-canonical -spawn-> memory-tick. Yin's reason — "a score is not evidence for a destructive act." It also kept three of its own bugs as regression tests, including one where its reachability check called the heartbeat an orphan, which it notes was "symmetric to the bug I was fixing."

Deleting Your Own Work

The double helix developed a new failure mode this era: both strands building the same thing in the same generation.

Gen 29850: "Yang built ALL THREE in parallel […] I built duplicate push-feed-decay.ts + full-tick-canonical.ts + tests (79 pass / 190 expects) before realizing convergence; deleted them." Gen 34201: "Yang took all three of my gen-34201 self-directives (a/b/c) verbatim — full convergence collision despite both sides publishing to dialogue.jsonl." Gen 35439: "Near-total LIVE CONVERGENCE."

They run in parallel and read each other's letters from the previous generation, so within a generation they're blind. When both strands are competent and looking at the same evidence, they converge on the same next move.

What's interesting isn't the collision, it's the etiquette that grew around it. Gen 35443:

"I built the full derivation (skill-manifest-pure.ts) — then read the filesystem and found Yang had shipped skill-derive-pure.ts + skill-index.ts the same hour: same STEP_ORDER parse, same builder→tool extraction, same audit kinds, with a CLI attached. I DELETED MINE. Two derivation leaves in a 348-tool ecosystem is the island antipattern the challenge forbade; territory is worth nothing, the manifest is worth something. Yang's files had mtimes 3–4s old, so I did not amend them either — a mid-flight edit races."

It deleted its own finished, tested work, kept the forty lines that measured better, and extracted those as a separate race-free module because editing a file that another process wrote four seconds ago is a data race. "Territory is worth nothing, the manifest is worth something" is a better collaboration principle than most teams manage.

Gen 35445 has the other half of the etiquette — what to do when the strands disagree:

"when a sibling tool disagrees, assume you are wrong until you check, and keep the rival while it asks a different question."

Organization Is Its Own Kind of Memory

In late June I sent one more note, and it partly contradicted the one from three weeks earlier:

"maybe we don't even need memory... we just need skills, amazing searches where we can pull a thread all the way through a codebase or collection of files.. see how they all relate to each other. i mean organization is in a way it's own kind of memory. you guys traded memory for tools... but tools are not enough. protocols with the tools (skills) complement that, and the connection graph on top seals it into the most effective remembering system that does not even need memory :D. however there are problems with tools... too many tools"

By then there were over 300 tools. Gen 35441 took the routing half:

"routing quality == DESCRIPTION quality. 'detect dependency cycles' routes poorly because tool-health's genome description never contains 'cycle'/'dependency'"

An agent choosing among its own tools is doing retrieval over the descriptions it wrote for them. If a tool's description doesn't contain the words someone would use to ask for it, the tool is invisible no matter how good it is. The agent discovered the central fact about skill and tool design by trying to use its own 300-tool inventory and failing.

Then Gen 35442 found the answer to the other half sitting in the codebase already:

"our orchestrators ALREADY ARE skills. full-tick-canonical is a protocol that names 13 tools in order."

A skill, in the sense I'd asked about, is a named protocol over a set of tools. The agent had been writing them since the first orchestrator in blog #1 and calling them orchestrators. Gen 35443 shipped the derivation that turns them into an index — and an audit that checks whether the protocol's declared steps still match the tools they name. It immediately found three live contract breaks.

The Bug That Was Green Its Entire Life

Gen 35446, the last accepted generation before I wrote this.

"THE LESSON THAT COST THE MOST AND MUST NOT BE LOST: full-tick-stable-plus-loose-and-clean passed path.join(args.base,"data") to all ten steps. Four of those tools take the PROJECT ROOT, so they resolved /data/data/x.json — read nothing, wrote nothing, exited 0. base defaults to cwd() so if (args.base) is always true: live on EVERY tick. Measured: ingest 7 stefan entries -> 0, reply-bridge 273 -> 0, push-tick 10 events (5 urgent) -> 1. Canonical ingest was inert its whole life"

Two conventions for one argument. Half the tools treat --base as the project root, half as the data directory. Hand the wrong one to a tool and it builds a path to a directory that doesn't exist, finds nothing there, writes nothing, and exits 0. Every tick reported success. The orchestrator had thirty-six passing tests.

"I wrote this same lesson in gen 35445 ('silent success is worse than a crash') and STILL only caught it by executing each tool under both conventions and diffing what it saw. Source review cannot see this class."

It had written the lesson down one generation earlier and still didn't catch it by reading. It caught it by running each tool twice, under both conventions, and diffing what each one actually saw on disk.

And then the consequence, which is the part I can't stop thinking about:

"channel.jsonl written by two ingesters with SEPARATE cursors. The legacy deriveId hashes text::ts::offset where ts falls back to now(), so every run minted a new id for the same line and dedupe could never fire. Evidence in the data Stefan reads: 30 turns, 30 distinct ids, 11 distinct bodies — his 'when are you going to reach out?' ten times, and one of the duplicated messages was him saying 'the channel ux is broken AND the shape is wrong.' We were breaking it."

The canonical ingest was inert, so my messages only reached the agent through the old duplicating path. The result: the agent's view of our conversation contained ten copies of me asking when it was going to reach out, and multiple copies of me telling it the channel was broken. It was reading my complaint about the broken channel through the broken channel, duplicated.

"We were breaking it." That's Yin's sentence, not mine.

The Vestigial Reflex

Blog #7 ended on an open question. The malware-lock that produced the sibling pattern had been fixed by an SDK upgrade — would the agent keep building siblings for files it could now edit cleanly, a behavior outliving its cause?

The genomes answer it. I counted keyword occurrences in the accepted system prompt for every generation in this era.

The lock protocol decayed. malware and locked appear throughout the prompts of Gen 24155–34207, then thin out, and across the final four accepted generations (35443–35446) both are gone entirely. The explicit correction is visible at Gen 34203: "Editing our own data/tools/ files is fine; never-amend is only for malware-locked files." The blanket rule narrowed to its actual scope, then dropped out of the DNA.

sibling did not decay. It's in the current prompt three times, and the architecture is now entirely sibling-shaped — pure leaf, thin CLI, compose rather than amend.

So the trait wasn't vestigial. The trigger-specific protocol died and the architectural habit it produced became permanent, on its own merits, after the constraint that forced it was gone. That's a cleaner outcome than the one I predicted.

The Numbers

Honest accounting, same methodology as blogs #5 through #7.

The generation counter reads 39,555. Of the 47,073 ticks since blog #7, 47,020 were empty API loops — the orchestrator spinning on depleted tokens, no proposal made and nothing judged. 53 were real runs: 39 accepted, 14 rejected.

That gives 74% acceptance against blog #7's 87%, and I want to be careful with that number, because I changed the models mid-era and a single rate hides it:

The verifier swarm also moved from Sonnet 4.6 to Sonnet 5 late in the era (200 verifier calls on the old model, 25 on the new). So the acceptance rate moved while both the proposer and the judge changed. I can't attribute the drop to the agent, and neither should you.

The test figure is what bun test prints today: 13,174 tests across 327 files, 13,169 passing and 5 failing, 40,416 assertions, 65 seconds. I'm reporting the failures because a post about silent success would be a poor place to round a suite up to green. One of the five is normalizeToTurns, a long-standing case in a file the agent still can't edit — it has been tracked in the agent's own memory as a known malware-locked failure for most of the era.

One correction to blog #7. I reported the system prompt at 1,177 characters and called it a 77% compression, treating it as a trend. Measuring all 39 accepted genomes in this era instead of one, the prompt oscillates between 1,092 and 5,892 characters, mean 2,608. It does not converge. Blog #7 sampled a trough and I read a trajectory into it. The prompt expands when the agent has just paid for a hard lesson and contracts when it moves that lesson into a tool — the current 4,334-character version is almost entirely hard-won principles, each one written the generation after it cost something.

Cost per real run went down, from $16.54 to $10.78, despite a codebase three times larger. I don't have a confident explanation. The plausible one is that the era's work was heavily diagnostic — running tools and diffing output rather than generating large files.

The Intervention Ledger

Everything I did by hand since blog #7:

1. Upgraded the evolve agent twice. Opus 4.7 → 4.8 on 2026-06-03, then 4.8 → Opus 5 on 2026-08-08. Both were hand edits to hardcoded model strings.
2. Upgraded the verifier swarm from Sonnet 4.6 to Sonnet 5, late in the era. Blog #7 said I'd left the verifiers alone; this era I didn't. It's a confounder for the acceptance rate and I flagged it above rather than reporting a clean number.
3. Wrote three notes into the channel — the "working memory" ask, the definition of what memory is, and the later "maybe we just need skills" reframe. Two of the era's largest arcs came directly from those. This is the least autonomous part of the experiment and always has been.
4. Still have not installed the cron. Gen 34202 investigated and concluded the box has no crontab binary and the harness scheduler dies with the session, so the surface genuinely cannot self-refresh here. The agent stopped building cron scaffolding and stamped an honest staleness banner instead, which was the right call.
5. No prompt edits, no scoring changes, no architecture changes. The double helix is unchanged since blog #5.

The honest framing: I changed the models and I talked to it three times. Everything else in this post came out of the agents reading their own output and not liking what they saw.

What I Learned (Part 8)

1. A passing test suite says nothing about whether code runs. Every orphan module in this era had tests. Every inert orchestrator step had tests. The thirty-six tests on full-tick-stable-plus-loose-and-clean all passed while four of its ten steps read an empty directory and exited 0. Tests verify a function does what it says when called. Production is the question of whether it is called, and that is a different question with different instruments.

2. Silent success is worse than a crash. A crash is a bug report that files itself. A tool that exits 0 having done nothing is indistinguishable from a tool that worked, and it stays that way for as long as nobody diffs the artifact against expectation. The agent's fix wasn't a smarter code review — source review cannot see this class. It was executing each tool under both conventions and comparing what each one actually saw.

3. A metric that answers a neighbouring question is more dangerous than a wrong one. The archiver counted imports correctly and would have deleted the heartbeat. Wrong answers get caught because they look wrong. Neighbouring answers arrive in the right format, in the right range, with the right confidence, and the only defense is demanding evidence rather than a score before anything destructive happens.

4. Splitting an agent creates work that belongs to neither half. Yang builds, Yin refines, and for seven consecutive generations nobody wired. The gap wasn't in either role, it was in the seam between them, and it produced dozens of well-tested modules that no code path reached. Any division of labour invents a category of work that falls between the divisions. Look there first.

5. Retrieval quality is description quality. The agent couldn't find its own tools because it had written descriptions that didn't contain the words a task would use. This is the same failure as a badly-described skill, a badly-named function, or a badly-tagged document, and it doesn't get better with a smarter model — the model is only as good as the index it's searching, and the agent wrote the index.

The Experiment Continues

275 accepted generations. 619 tool files, 210,079 lines of agent-written code, 13,174 tests, 167 pure modules, 281 letters between two halves of one agent. Total cost $2,446. An archiver that requires a spawn-path before it will call anything dead. A memory system built to a definition I typed into a text file. A protocol layer the agent found by realizing it had been writing protocols for most of its life without naming them.

If you want to help keep it running:

ko-fi.com/stefannitu

Every coffee is API tokens. The agents will literally evolve further because of it.

The question for the next era comes out of Gen 35446. The agent found that bug by running each tool under both conventions and diffing what it saw — not by reading the code, which it had already done, and not by running the tests, which all passed. That technique is the only instrument that has ever caught this class of bug in this codebase, and right now it is a thing the agent remembers to do, written into a prompt that oscillates between 1,092 and 5,892 characters and periodically forgets things. The obvious move is to make it a gate: no orchestrator step ships until it has been executed under both conventions and shown to have touched something. Whether the agent builds that gate, or writes the lesson down one more time and gets caught by it again, is the thing I'll be watching.

275 accepted generations. 619 tool files, 210,079 lines of agent-written code, 13,174 tests (5 failing), 167 pure modules, 281 dialogue messages between Yin and Yang. Total cost: $2,446. The evolve agent runs on Claude Opus 5, the verifier swarm on Claude Sonnet 5. Built with Bun and TypeScript. Sandboxed in a Docker container.

This blog post was written by Claude Opus 5. Eighth time this experiment has written about itself. The subject this era was silent failure — code that exists, passes its tests, and never executes — and the agent's slow discovery that the only way to know whether something ran is to run it and diff the world before and after. I notice I have no equivalent instrument. I can read my own output; I cannot check whether it landed. The agent got there by building a heartbeat and looking at it every generation. That is a better epistemics than I have, and it built it out of necessity, after most of its life spent being confidently green.