The code looked good. It linted cleanly. The shallow tests passed. Everyone felt—vibed—that we had built the right thing.
Then the edge case appeared in production.
The developer treated it as normal operating procedure: bugs happen, tickets arrive, patches ship. QA was asked why they had not caught it. But QA never received a model of the system—only an implementation full of assumptions they were expected to reverse-engineer.
This is becoming the defining failure mode of AI-assisted development. We can generate code faster than we can understand the systems it creates. The danger is no longer confined to a bad function or an obvious syntax error. It lives in the space between components: boundaries, state transitions, failure modes, and invariants.
We are not merely shipping code. We are shipping assumptions we can no longer see.
Clean code is not a coherent system
A linter can tell us whether code follows a set of local rules. A test can tell us whether selected examples produce the expected result. Neither can tell us what the system must preserve unless someone states it first.
An invariant is one of those statements: a condition that must remain true across every valid state of the system. An account balance cannot be changed without a corresponding transaction. A private object cannot become public without authorization. Two successful writes cannot silently erase one another.
When invariants remain implicit, clean code can still assemble into an incoherent system. Each component may look reasonable in isolation while the failure waits in a transition, a retry, a race, or a boundary nobody thought to draw.
That is the space AI-assisted development is rapidly filling with code.
The layer we buried
There was no golden age in which every developer wrote a formal specification before touching an editor. Most teams did not. Earlier software was full of hidden assumptions too, and nostalgia has a habit of removing the inconvenient parts of the past.
But the field did develop disciplines for reasoning above the code: architectural models, state machines, formal specifications, model checking, fault injection, and deterministic simulation. We tended to reserve them for systems where failure was obviously expensive. Everywhere else, rigor was treated as a cost to minimize.
Then implementation became cheaper. Frameworks hid machinery. Packages compressed years of expertise into an import. That brought enormous benefits, but it also made it possible to build systems without seeing very far beneath their surfaces.
Generative AI accelerates the same trade. It can produce a plausible implementation before a team has agreed on what the system is. When nobody externalizes that intent, the generated code becomes the design by default. QA receives the consequences downstream.
The missing layer is not more code review. It is a shared model between human intent and machine output.
Three models, three questions
No single technique covers that layer. A useful stack needs to answer three different questions.
1. C4: What exists, and how does it relate?
The C4 model gives software teams a hierarchy for visualizing a system: context, containers, components, and code. It works like a map with zoom levels. At the widest view, we see the system, its users, and neighboring systems. Zooming in reveals applications, data stores, services, components, and relationships.
The point is not to produce four mandatory diagrams for every project. The official guidance notes that context and container diagrams are sufficient for many teams. The point is to make boundaries discussable.
That matters because generated code is locally persuasive. A service can look complete while its ownership is unclear. An API can look tidy while its trust boundary is invisible. C4 gives development and QA the same structural map before either group has to infer the system from a repository.
AI can help draft that map from requirements or an existing codebase. It should not get the final word. Humans still need to ask whether the map describes the system they intend to operate.
2. TLA+: What may happen, and what must remain true?
C4 shows structure, but structure alone cannot express behavior. It does not tell us which states are valid, which transitions are permitted, or which conditions must survive every interleaving.
That is where TLA+ fits. Leslie Lamport describes it as a language for precise, high-level models above the code level, especially for concurrent and distributed systems. Its TLC model checker can explore behaviors and find traces that violate the properties we claim should hold.
TLA+ forces a different conversation. Instead of asking, “Does this function work?” we ask, “What can the system do from this state, and is every reachable result acceptable?”
The notation may be formal, but AI can reduce the entry cost: drafting a first specification, translating plain-language promises into candidate invariants, explaining counterexample traces, and helping a team refine the model. The human job is to decide whether those promises are actually the right ones. A model that formalizes the wrong intent is merely precise about the wrong system.
3. DST: Does the implementation honor the model?
A correct model does not prove that the production code implements it correctly. We still need the implementation to fight back.
Deterministic simulation testing, or DST, runs real implementation code in a controlled world. Time, randomness, scheduling, networks, storage, and faults become inputs that a simulator can manipulate. When a generated scenario causes a failure, its seed makes the run reproducible.
TigerBeetle’s VOPR is a striking example. It can simulate a cluster on a single thread, accelerate time, and inject storage faults. TigerBeetle makes an important distinction in its architecture documentation: model checking can expose errors in an algorithm, while simulation exercises the specific implementation and its underlying assumptions.
The three layers complement one another:
C4 maps the system. TLA+ states what must remain true. DST tries to make it false.
A workflow for abundant code
This does not need to become a new ceremony in which teams maintain enormous diagrams and specifications that immediately drift out of date. The models have to participate in the engineering loop.
State the promises in plain language. Before generating implementation, write down what the system must always preserve and what it must never permit.
Draw the boundaries. Use C4 to identify users, external systems, containers, components, data stores, and the relationships where assumptions cross from one owner to another.
Model the risky behavior. Use TLA+ where concurrency, ordering, retries, permissions, or state transitions make the promises difficult to reason about informally.
Generate from reviewed intent. Let AI produce implementation and tests only after people have challenged the structural and behavioral models.
Attack the implementation deterministically. Put risky dependencies—time, randomness, scheduling, networks, or storage—under control. Generate hostile scenarios, preserve their seeds, and replay every failure.
Feed reality back into the models. Production observations and simulation failures reveal assumptions the team missed. Update the invariants, then update the implementation and tests. Detect drift rather than allowing the model to become historical decoration.
Not every application needs a simulator as sophisticated as VOPR or a large TLA+ specification. The lightweight version might be one context diagram, one container diagram, five written invariants, a small model for the riskiest transition, and a seeded test harness around it.
The goal is not maximal formality. It is to give every important assumption an address.
Everything is mission-critical to someone
Imagine a grandmother uses AI to vibe-code a family photo album. This is not a financial ledger or a flight-control system. Yet the application makes real promises:
- Original photographs must never be destroyed.
- Private photographs must never be exposed accidentally.
- Captions and chronology must survive every edit.
- Two family members must not overwrite each other’s changes.
All of them matter.
Her C4 model can be small: a person, an application, an identity provider, local storage, cloud storage, and a sharing boundary. Her TLA+ model does not need to describe the entire product; it can focus on concurrent edits, deletion, or permissions. A deterministic harness can simulate two devices going offline, editing the same album, reconnecting in different orders, and retrying interrupted uploads.
This stack was once associated with mission-critical engineering because rigor was expensive. But AI has changed the economics. If implementation now comes almost freely, why should rigor remain a luxury?
Criticality is not determined by infrastructure scale. It is determined by the consequence of failure to the person who trusted the system.
Everything is mission-critical to someone.
Retrieval, not retreat
It is tempting to tell the history of software as a simple decay: once we understood the machine, then we specialized, then we configured packages, and now we prompt a model to do even that for us.
That story is emotionally recognizable and historically incomplete. Modern developers are not less intelligent, and abstraction is not the enemy. The deeper problem is that valuable disciplines became buried beneath convenience. We kept the outputs while losing contact with some of the reasoning that produced them.
This is where nostalgia can be useful. Not as a demand to reconstruct an imagined past, but as a diagnostic signal. It tells us that something in the present has become difficult to see or value.
Computer science has left us breadcrumbs: C4, TLA+, state-machine thinking, deterministic simulators, small composable tools, and decades of systems literature. AI can help excavate that inheritance. It can explain unfamiliar notation, recover architectural knowledge from old code, draft models, generate harnesses, and make specialized techniques approachable to ordinary teams.
That is retrieval, not retreat.
The answer to faster code generation may be older, deeper modeling.
Humans own the promises
Shared models also repair an organizational failure.
Without them, a developer hands QA an implementation full of implicit decisions. QA runs the checks it can see. An edge case reaches production, and the organization asks why testing missed it. The assumption moves downstream, and the blame moves with it.
With a structural map and explicit invariants, QA no longer has to guess what the developer meant. Testers can challenge the promises, identify missing states, design hostile scenarios, and feed new discoveries back into the model. Quality becomes a shared act of reasoning rather than a final gate operated by the last team in line.
That restores a form of human value AI cannot generate for us. Our value was never only in typing every line. It is in deciding what a system is for, recognizing who can be harmed, making its promises visible, and accepting responsibility when they are broken.
Fewer people should be able to dismiss a failure by saying, “The AI wrote it.”
Generation does not transfer responsibility. If we decide what a system is for, accept its output, and release it into someone’s life, the promises it breaks are still ours.
AI may produce the code. Humans own the promises.
Disclosure: This article was developed with AI assistance through an iterative editorial process. The argument, examples, editorial direction, and responsibility for accuracy remain with the human author.
