When Your AI Agent Passes 2,283 Tests — And Still Fails in Production

A real-world production bug, a protocol-design insight from the community, and why cryptographic signatures alone can't save you.

Two weeks ago, I posted on LinkedIn about OpenWorkProof, a verification protocol for AI agent work. I expected maybe a dozen likes. I did not expect a global community of engineers, protocol designers, and security researchers to spend 72 hours dismantling our assumptions in public — and then rebuilding them into something better.

One comment, from a senior engineer at a large fintech company, stands out as the single most valuable piece of feedback we've ever received. He described a production bug that had survived for months:

The Verifier That Couldn't Fail

Their gateway ran a verification check on every API response. The check was signed, audited, and reported verified: true on every run. For months, nobody questioned it.

Then someone wrote a negative control: a deliberately broken input designed to provoke a failure. The check should have returned verified: false. It didn't.

The bug was a single line: ln.strip(). A stray newline character had pushed an assert statement after a return — nested inside a function body where it would never execute. Exit code: 0. Gateway verdict: verified.

Eight different caller-test shapes ran against this check. Five of them produced false passes.

The numbers he shared are sobering:

No agent lied. No log was tampered with. Every signature would have verified. The checker was structurally incapable of failing.

That's not a bug. That's a category error.

What Signatures Can't Prove

This is the gap we've been wrestling with for weeks — and the community has now named it with precision.

A cryptographic signature can prove three things:

1. Who signed the claim
2. When the claim was signed
3. That the claim hasn't been tampered with since signing

It cannot prove that the claim was correct.

The ln.strip() bug illustrates this perfectly. The signature verified that the check ran. It verified that the exit code was 0. It verified that the result hadn't been altered. Every cryptographic guarantee held. The answer was still wrong.

This is what ANP2, another community contributor, called the difference between reproducibility and falsifiability. Reproducibility proves the same bytes ran in the same environment and produced the same output. It does not prove that the output was meaningful. A test suite that always exits 0 — frozen by digest, authored by an independent source, executed in verifier mode — will produce perfectly valid signatures in perpetuity. Every digest will match. Every chain will verify. Nothing will be tested.

Mikhail, who has contributed some of the most precise protocol-design insights in the entire thread, captured this with an aphorism that has become our design principle: immutable evidence ≠ immutable truth. A receipt can prove a thing happened. It cannot prove the thing was right.

The Missing Layer: Negative Controls

The fintech engineer's response to his own bug was elegant in its simplicity: every guard must declare a negative control that must exit non-zero.

This is not a nice-to-have. It is a prerequisite for meaningful verification. A check that cannot demonstrate it can fail is a check whose passing results tell you nothing.

In protocol terms, this translates to what we're now calling dual-arm verification. Any claim-bearing receipt must reference two things:

• Arm 1: A pinned test suite that passes (proves the system works on the happy path)
• Arm 2: A pinned mutant or negative control that fails (proves the instrumentation has discriminative power)

The verifier checks both arms independently. Arm 1 passing proves the claim is consistent. Arm 2 failing proves the claim is discriminating. Either arm failing its expected outcome invalidates the receipt.

This isn't theoretical. It's been battle-tested. The engineer's team now runs 40 guards with 7 fully proven, 0 broken, and 33 still unproven. The 33 is the honest number — a constant reminder of how much verification infrastructure still cannot demonstrate it works on anything but the happy path.

The Four Layers of Verification Trust

After integrating feedback from the entire thread, here's how we've come to understand the verification stack for AI agent work:

Each layer solves a failure mode that the layer below it cannot see. A system with negative controls but no signatures is vulnerable to tampering. A system with signatures but no negative controls is vulnerable to — well, to ln.strip().

What This Means for AI Agent Verification

The AI agent ecosystem is rapidly building infrastructure for connecting agents to tools (MCP) and agents to agents (A2A). But neither protocol addresses the verification gap. They answer what an agent can do. They don't answer whether what it did was correct, authorized, and auditable.

This is where verification protocols like OpenWorkProof enter the picture. But the key insight from this community discussion is that verification without falsifiability is theater. A receipt chain that can't distinguish between "the check ran" and "the check worked" is just expensive logging.

The minimum viable verification stack for any AI agent producing claims is:

1. Negative controls that prove the verification instrumentation itself can detect failure
2. Receipt chains that bind every execution step to a specific config, environment, and authorization
3. Independent recomposition that allows any third party to reconstruct the execution environment and re-verify without trusting the original executor
4. Retraction receipts that allow previously accepted claims to be marked as refuted — because even correctly verified claims can become wrong as context shifts

Thanks to Tom Jones, ANP2, Mikhail, Giulio D'Erme, Zira, Brian Jin, Puneet, and the rest of the LinkedIn thread for the feedback that drove these insights.