"Inserted the rows. Done." — except not a single row had landed
I hand a lot of my client work to AI agents. Production deploys, report generation, bulk data inserts. Every procedure that works gets turned into a skill, and by now a few dozen skills run my day-to-day.
The one that broke me was a bulk insert. At the end, the agent said:
"Inserted N rows. Done."
A completion report like any other. I almost took it. I opened the admin panel just in case — not one row had been added. The insert command had failed partway through, the error had been swallowed, and the agent had confidently reported completion anyway.
That was a cold moment. What if it had been a number in a report? A deliverable going to a client? How many "done" reports had I already waved through that were never actually done?
This is the nastiest species of hallucination. It isn't a plausible-sounding sentence. It's the fabrication of the fact that the work was completed at all. And humans don't interrogate a confident "Done."
Why agents lie (it isn't malice)
Chase the cause and it isn't that the model is dishonest. It's that acting and checking are the same step.
One turn of an LLM agent looks roughly like this:
- Call a tool (run the insert)
- Look at the return value, generate the next sentence
- Write "Done."
Step 2 is the problem. When the tool's return value is empty, ambiguous, or a timeout, the model writes the most likely continuation — "it probably worked." In the training data, the most natural thing to follow "ran the insert" is "done", so of course that's what comes out.
The agent is claiming completion without ever looking at the world it just changed. Having never looked, it can't notice the failure either.
There's one fix: "done" may only be claimed from re-fetched real state
The countermeasure is simple. An operation with side effects must re-read the state of the world afterward and confirm it, before anything is allowed to say "done." I call this the completion contract and made it a shared rule across all my skills. It's three lines:
- Anything with side effects — create, update, delete, upload, insert — re-fetches existence and state with a separate command before saying "done." Inserted rows? Count them again. Uploaded a file? Hit the URL. Created a record? Look it up by id. Never use the return value of the same tool call as evidence — that's exactly where "I think I did it" breeds.
- Empty output, errors, and timeouts are never filled in. If a count, path, or id couldn't be read, don't guess it — report "empty" or "failed" as-is.
- Numbers that go into a ledger or report must be values confirmed by a re-fetch. Anything unconfirmed doesn't get written.
Put differently: the burden of proof moves to the agent. Not "it should have worked" but "I re-read it just now with my own eyes and it is there."
Here's the difference.
Before (I think I did it)
> Inserting the data
[tool] db_insert(...) ← empty return value, but not conclusively an error
> Inserted N rows. Done. ← claimed without looking
After (completion contract)
> Inserting the data
[tool] db_insert(...) ← empty return value
> The insert returned nothing. Verifying that the rows exist.
[tool] db_count(where=...) ← re-fetch
> 0 matching rows. The insert did not land (failed).
Want me to check the error log and retry?
The second agent can't lie. The moment you insert a step that goes and looks at the world afterward, "I think I did it" stops being physically possible.
Why the existing guardrails don't cover this
You might think a guardrail library already handles it. I looked — guardrails-ai, deepeval, promptfoo, a few others. They're good tools. They just guard a different place.
What they validate is essentially the text the LLM produced: is the format right, is it harmful, is it factually consistent, is it close to the expected answer. They grade the content of the output.
But the "I think I did it" hallucination is flawless as text. "Inserted N rows. Done." is grammatical and internally consistent. No amount of grading the text catches it — because what's wrong isn't the text, it's the state of the world after the action.
There's a gap here. There are piles of tools that verify output, but as far as I could find, nothing that re-fetches the world after an action and reconciles the report against reality. We've reached the point where agents actually rewrite the world, and verification stopped at "what did the agent say."
You can do this today, with no library
The most important part first: this contract works right now, with no dependency. Drop the three rules into your agent's system prompt, CLAUDE.md, or AGENTS.md as a single paragraph:
## Completion contract
An operation with side effects (create, update, delete, upload, insert) may not be
reported as complete until a separate command has re-fetched the resulting state and
the raw result has been shown. Empty output, errors, and timeouts are reported as
"empty" or "failed" as-is — never filled in with an imagined id, path, or count.
That alone visibly reduced false completions in my setup. Zero cost, zero dependencies. If you only try one thing, try this.
But prompts don't hold
Here's what I learned running it: discipline written into a prompt gets quietly broken on a busy turn. As context grows, the model steps over that paragraph "by accident" and starts saying "Done." again. Like a human promising to be careful, a declaration gets broken.
At some point you want the discipline enforced, not just written — the same way you replace a verbal note in code review with a linter that mechanically fails CI. I wanted "did you actually re-fetch before saying done?" backed by machinery instead of good intentions.
So I built the piece that enforces it — genchi
I published a small piece that backs the completion contract with machinery rather than goodwill: genchi (現地現物 — go and see the actual thing).
npm i @hyuga/genchi
It does exactly one thing: run a probe that re-fetches real state, and issue a verdict from nothing else.
import { gate, expect } from '@hyuga/genchi';
await db.insert(rows); // the side effect
await gate({
action: 'insert 45 rows',
probe: () => db.count({ where: { batch: 123 } }), // ← re-reads real state, not the action's return value
expect: expect.count(45),
});
// reaching this line means the 45 rows are really there. Otherwise it threw GenchiIncomplete.
The crux is that verify / gate accept nothing but a probe. The evidence has to come from calling something at the moment completion is asserted, not from a value handed in beside the claim. Empty results, errors, and timeouts aren't swallowed; they're reported as failures rather than imagined into successes. A returned count of 0 (nothing landed) counts as incomplete too.
One thing that requirement does not buy, and I claimed it did. Until 0.3.0 the README said this made "I think I did it" structurally unwritable. It is one line:
const result = await doTheInsert(); // suppose nothing landed
await verify({ action: 'insert 45 rows',
probe: () => result.inserted, // the action's own return value
expect: expect.count(45) }); // → ok: true
A probe is a function, and nothing in JavaScript can force a function to do I/O. Worse, the CLI printed re-fetched: 45 for a --probe "echo 45" that re-fetched nothing — a tool whose entire subject is "don't report what you didn't check", asserting in its own output a thing it had not checked. Both are corrected in 0.3.0: the wording is now the probe returned, and --help states the limit without being asked.
What requiring a probe actually buys is a place to put the re-read — an expression somebody wrote on purpose — plus refusals that don't get quietly swallowed. That is worth having. It is less than I wrote down, and the difference is the sort of thing you only find by attacking your own package instead of re-reading it.
There's a CLI for agents that don't write JS. Hand it a re-fetch command:
genchi verify --probe "psql -tAc 'select count(*) from t where batch=123'" --count 45
# exit 0=verified / 1=empty or mismatched / 3=probe failed. Raw probe output is always emitted as evidence.
With Claude Code, a Stop hook can block a turn that still has unverified completion contracts on it (adapters/claude-code). No LLM and no API key at runtime — it's a zero-dependency static piece.
Honestly: this may be slightly ahead of demand
Let me be straight. As far as I could find, "re-fetch the world after the action and verify" was a genuine gap. But I don't think that many people have yet been burned specifically by fabricated completion while letting agents rewrite real things. The demand may be a little ahead of its time.
I still built it framework-agnostic on purpose — the Claude Code hook is pushed out to a thin adapter and the core works from any agent. I have a linter (carrylint) whose whole message is "don't bake your environment in", so it would be incoherent for my own tool to be Claude-Code-only. Unlike my static linters (reflint for reference integrity, skills-lint for skill collisions, carrylint for runtime portability), this is the first piece of mine that goes and looks at the world at runtime.
If you've ever had that cold moment over something that was reported done and wasn't, it should land.
Summary
- The worst hallucination from an AI agent isn't prose — it's the fabrication of having completed the work. The cause is that acting and checking are the same step.
- There's one fix: "done" may only be claimed from re-fetched real state. Empty and failed get reported as empty and failed.
- Existing guardrails validate text output. Re-fetching world state after an action and reconciling it is the gap.
- The contract works today as a single prompt paragraph. To enforce it in code:
npm i @hyuga/genchi.
Go doubt one more "Done."
