Generative systems fail differently from normal software. They do not crash. They produce something plausible, fluent and occasionally wrong, at scale, with a straight face. Shipping their output to the outside world without machinery in between is negligence. Ours pass three walls.
Wall one: evidence gates
Every automated claim must carry its receipts, and the receipts are structured, not vibes:
{
"claim": "business site has no online booking flow",
"evidence": {
"url": "https://example-biz.com/reserve",
"fetched_at": "2026-03-11T02:14:09Z",
"check": "booking CTA selectors absent in rendered DOM",
"screenshot": "sha256:9f2c11ab..."
},
"verdict": "pass"
}
No evidence object, no pass. Fuzzy match where the rule demands exact, no pass. It is astonishing how many hallucinations die at the simple demand: show me where you read that.
Wall two: the identity gate, and the bug that taught us humility
A pipeline that writes about real businesses must first prove it is looking at the right business. Our first identity gate compared the business name against the domain name and skipped anything that did not match. Reasonable, and catastrophically wrong in a market where businesses have Korean names and Latin-alphabet domains. The gate was silently discarding 98% of legitimate targets. The fix was to stop trusting the domain string and verify identity from the page body instead, name, address, phone cross-checked against the record. Pass rate went from 2% to 71% with zero wrong-entity incidents since. The lesson generalized: a gate that fails closed can be just as wrong as a gate that fails open, and only measurement tells you which failure you have.
Wall three: adversarial second passes
The cheapest quality upgrade we know is asking a second model instance to attack the first one's output. Not improve. Refute. The reviewer prompt states that its job is to find the flaw, and that when uncertain it should call the finding refuted, because a missed defect costs us more than a discarded one. A reviewer whose only job is finding flaws finds the ones a collaborative reviewer politely overlooks. Findings that survive are the ones worth acting on.
Behind all three walls sits the regression suite, about 3,900 cases, expected failures zero, one case added per incident. None of this slows the pipeline meaningfully. All of it together is what lets us point generative machinery at real-world consequences and sleep.
