Treat Runbook Status as Cached Data

The most dangerous runbook is not completely wrong. It is mostly right.

Completely broken documentation earns suspicion. A mostly accurate document earns trust, which is why one stale sentence can steer an investigation, review or release decision in the wrong direction.

A recent committed documentation correction made this concrete. Several neighbouring statements described the current state of one operational path. A narrow host probe and recent pipeline history contradicted two of them. Another nearby configuration statement remained true. That surviving truth made the stale prose feel credible and helped produce a false blocker.

The engineering lesson is simple: present-tense operational prose is cached data.

Procedure and status age differently

Runbooks usually mix two kinds of information.

Durable procedure changes when the system design changes:

• how to recover a service;
• which gate precedes a promotion;
• where an approval belongs; and
• what a destructive operation must never touch.

Current status can change without a documentation commit:

• an environment exists;
• recent runs are healthy;
• a feature switch is enabled;
• a migration has landed; or
• a dataset is actively used.

Treating those categories alike is the root problem. A procedure may remain useful for months. A current-state claim can expire minutes after it is written.

One true neighbour proves very little

Operational prose often arrives in clusters: “the host is absent, deployments fail, and promotion is disabled.” Reviewers naturally compress that into one belief.

But each clause may have a different authority and a different rate of change. Host reachability belongs to a network or HTTP probe. Deployment health belongs to recent run history. A switch belongs to configuration state. Verifying one does not validate the others.

This is the same discipline we apply to data models: do not infer one field from a nearby field merely because they usually move together.

Give every volatile claim provenance

If a sentence can change while the file remains untouched, attach four things:

1. Scope — exactly which environment, component or workflow the statement describes.
2. Probe — the smallest read-only check that can confirm or reject it.
3. Checked time — when the evidence was collected.
4. Recheck trigger — the event or age that makes the evidence untrustworthy.

A compact status block can be enough:

Claim: the pre-production host is reachable
Scope: one named deployment target
Evidence: read-only name-resolution and health probes
Checked: <date and time>
Recheck when: host configuration changes, a deployment fails, or seven days pass

The point is not paperwork. It is to make the next verification cheap.

Match the probe to the claim

Choose evidence with the narrowest useful authority:

A shallow probe can answer the wrong question. A process-health endpoint may be green while a database path is broken. One failed run does not mean every run fails. A zero count is weak evidence if the database connection itself is stale.

Good evidence is not necessarily elaborate. It is simply aligned with the claim.

Separate observation, assumption and desired state

Documentation becomes misleading when plans are written as facts.

Use explicit language:

• Observed: a read-only probe returned this result at this time.
• Assumed: the team expects this to remain true, but has not rechecked it.
• Desired: this is the state we intend to create.
• Procedure: these are the steps to create or recover it.

That vocabulary prevents “not yet verified” from quietly becoming “does not exist.” It also stops a completed setup appendix from continuing to look like outstanding work.

When a procedure completes, update its status in the same change. Otherwise the implementation and its operational map begin drifting immediately.

Prefer event-based expiry where possible

A fixed review date is useful, but operational claims often have better invalidation signals.

Recheck host status after infrastructure changes. Recheck deployment health after a workflow edit or failed run. Recheck a gate after configuration changes. Recheck usage before estimating migration risk.

These triggers are more meaningful than “review quarterly” because they follow the events that can invalidate the claim. A time-based expiry remains a helpful backstop when no reliable event exists.

The engineering trade-off

Evidence has a cost. Probes need access, time and ownership. Some systems cannot be queried safely by every contributor. Continuously verifying every sentence would turn documentation into a monitoring platform.

The balanced approach is risk-based:

• keep probes narrow and read-only;
• verify at decision points rather than on every edit;
• automate only stable, valuable checks;
• record access limitations honestly; and
• delete volatile claims that do not help a decision.

The return is not perfect freshness. It is fewer confident decisions built on expired truth.

A runbook should teach re-verification

Good operational documentation does more than preserve the answer somebody found last time. It preserves the method for finding the answer again.

The next time a runbook says an environment is absent, a pipeline is always red or a switch is off, ask four questions: what is the scope, what probe supports it, when was it checked, and what has happened since?

Old truth is still old. Make the path back to current truth part of the document.