An OpenAPI diff is a judgement call, not a line diff

An API can return 200 OK and still break every client that depends on its
contract. Uptime monitoring answers whether an endpoint responds. It does not
answer whether the shape it responds with is still the shape your consumers
compiled against.

This is about the judgement around an OpenAPI diff: where the baseline comes
from, how to reason about direction, what should fail automatically, and what
still drifts after merge.

Start from a reproducible baseline

Compare the proposed specification with the exact OpenAPI artifact shipped by
the base revision. A mutable documentation URL is a poor pull-request baseline
because it may change between two reviews of the same commit. Store or generate
both versions so the identical pair can be reproduced locally when someone
disputes the report.

Validate both documents before diffing them. A malformed or partially generated
spec produces a misleading compatibility report, and reviewers stop trusting the
gate after the first false alarm.

Reason about direction, not diff lines

For requests, widening what the server accepts is safer than narrowing it. For
responses the perspective reverses: the server producing a new shape may exceed
what an old client tolerates.

Removing a response property is breaking because consumers may read it. A
property that was always present but becomes optional is risky in the same
way — clients dereference it without an absence check, so the value can become
null or undefined at runtime.

Enum edits are directional too:

• removing a value from a request enum rejects calls that used to be valid;
• adding a value to a response enum can break a consumer that compiled an exhaustive switch with no fallback branch.

A diff that assigns one severity to every enum edit will be both too loud and
too quiet.

Separate what fails from what needs review

Not every flagged change deserves the same response. Removing an operation
should fail the build. Tightening numeric bounds, changing a string format,
editing a default value, or dropping an optional request parameter can change
behaviour while the JSON type stays identical — those need a human, not a red
build. Documentation-only edits should stay visible without paging the same
people as a deleted endpoint.

Four buckets keep alerts actionable: breaking, potentially breaking,
additive, documentation.

A worked example

GET /v1/users/{id} previously returned an object with id and email. The
proposed contract removes email, adds a required expand query parameter, and
adds an optional displayName response field.

That report tells a reviewer exactly what existing clients must change. A raw
line diff of generated YAML does not.

Running it

You can check a pair of specs without installing anything permanent:

npx @semihbugrasezer/cryonel diff base.yaml head.yaml

It takes a repository policy file for changes a team has already accepted and
exits non-zero on a breaking change, which is what makes it usable as a CI gate.

Honest limits, because this is the part most comparison posts skip: the diff
focuses on operations, parameters, request-body requiredness, and direct
successful-response schemas. It reads OpenAPI 3.0 and 3.1. It does not
resolve remote $ref URLs, and it does not replace integration tests.

Monitor drift after merge

Pull-request checks cover planned source changes. They do not catch a gateway
configuration edit, a generated artifact published from another repository, or a
manual production change.

Scheduled monitoring compares the deployed contract with the last known baseline
and records when the difference first appeared — which is the question an
incident review actually asks. The document can also stay identical while
pagination order, defaults, rate limits, or authorization policy change, so this
narrows risk rather than eliminating it.

Which tool

If you want a CLI in CI and nothing else, oasdiff
is mature and I recommend it for most teams — I wrote up an
honest comparison of it
against Optic, Redocly CLI and the Atlassian tool.

I build Cryonel, which covers two things those don't: a
browser check with no install, and scheduled post-merge drift monitoring. The
diff tool runs locally in the tab, and
there's a no-signup demo of the
classification if you want to see the rules before trusting them.

What I'd like feedback on

1. Which classification above would you argue with?
2. Does your team gate on contract diffs in CI today, or only review by eye?
3. Has post-merge contract drift ever caused you an incident?