Catching API contract drift in CI without codegen

You can catch API contract drift in CI without running codegen there: fetch the backend’s live OpenAPI document, diff it against the copy the frontend was built from, and report only conflicts on elements both sides share. Mine is 150 dependency-free lines. It has caught three real breaks, and been red on 52 of its last 71 runs.

The frontend is built ahead of the backend it calls

The client project is a data platform: a Python backend that publishes its OpenAPI document at /openapi.json, and a React frontend I build for it. Types are generated, not hand-written — a script curls the spec into src/data/openapi.json and runs openapi-typescript over it, which today produces a 14,616-line types.d.ts from 139 paths, 171 operations and 268 schemas. When the frontend work started, that spec had 89 paths and 169 schemas; it has grown by half in three months, and every one of those changes arrived from a different team on a different deploy cadence.

That cadence is the whole problem. Frontend work regularly lands ahead of the backend deploy it depends on: the endpoint is agreed, the spec is regenerated from a feature branch, the UI is built against it, and the integration environment catches up days later. So the committed spec is routinely a superset of the live one — and any check that asks “is the committed spec identical to live?” fails on every branch, every day, for reasons that are nobody’s fault.

Why regenerating types in CI is the wrong check

The obvious freshness gate is the one I use elsewhere for committed build artifacts: regenerate in CI and git diff --exit-code. For a generated type file it fails twice over. First, the frontend-ahead workflow means the diff is non-empty by design most of the time. Second, openapi-typescript output moves for reasons that are not contract changes at all — key ordering, a docstring edit on the backend, a formatter version — and a 1,400-line diff in a generated file tells you nothing about which of those lines would break a request. The most recent regeneration commit on the project touched 1,435 lines of types.d.ts; the underlying contract change was one required field.

The signal I actually want is narrower and directional: of the things both sides agree exist, has the backend changed one in a way that breaks a request the frontend already makes?

A directional diff over the shared surface

The comparator takes two parsed OpenAPI documents, committed and live, and walks the intersection. The rules are short enough to list in full:

• Committed-only anything is ignored. A path, method, schema, parameter or property that exists only in the committed spec is the frontend being ahead. Not a conflict.
• Live-only optional additions are ignored. A new optional property or query parameter on the backend can’t break an existing request.
• Live-only required additions are conflicts. A required property or parameter that the committed spec doesn’t know about means every request the frontend sends today is now a 422.
• Shared elements are compared shallowly on type, format, enum set and $ref target. An enum losing a value, a string becoming an integer, an inline object becoming a named schema — each is a conflict with a one-line reason.
• A shared field turning required is a conflict. Only 2xx response schemas are checked, because that is what components consume; error shapes are approximated by the mocks anyway.
• $refs are never resolved. Depth into named schemas comes from the components loop walking every shared schema, not from recursion.

The output is a list of { where, detail } pairs — components.schemas.X.a: type changed: committed=string live=integer — which is the level a reviewer can act on. The whole module is pure, has no dependencies, and is covered by 13 unit tests that each pin one rule: “ignores a path present only in committed”, “flags a live-only required param on a shared path”, and so on.

A thin runner around it fetches the live document from the integration environment with a 30-second timeout and exits 0 for clean, 1 for conflicts, and 2 when the fetch itself failed. That third code matters: an unreachable integration environment is an infrastructure problem, and it must never read as “the contract drifted”. In the last 100 pipelines it fired zero times.

The blind spot, written down

Directionality buys the frontend-ahead tolerance at a price, and the price is documented in the module header rather than discovered later: a server-side removal of a shared property is invisible, because “committed has it, live doesn’t” is exactly what a frontend-ahead addition looks like. Enum values are compared as whole sets, so a backend dropping a status value is caught — and was, in August, when a pending state was removed from a service-status enum and the canary named it on every branch that had not yet picked up the frontend change dropping the dead status. But a backend deleting a property outright would sail through. There is a unit test asserting that limitation, so nobody fixes it by accident and calls the new behaviour a regression.

What it caught in twelve days

Pulling the canary’s line out of the last 100 pipelines — 24 August to 4 September — gives this:

Three of the conflicts were real, and each has the shape the design was built for. On 24 August the backend renamed an entire submission-status vocabulary — seven enum values replaced by seven different ones — and the canary’s message listed both sets side by side; the frontend aligned four days later. On 27 August a status field on a shared row schema became required and changed from an inline object to a named $ref, two conflicts on one line, which is a rename-plus-tightening you would otherwise find in a browser console. And on 4 September a required organization_types property appeared on a documentation-upload request body that the frontend does call. That last one is red on the integration branch as I write this.

Warn-only was right in June and wrong by August

The CI step has been npm run contract-drift || true since 9 June: exits non-zero on drift, never fails the job. Warn-only was the correct first setting. The comparator was new, its false-positive rate was unknown, and a gate that blocks merges on a check nobody trusts yet gets deleted, not fixed.

Twelve days of pipelines say what the steady state became. 52 red runs out of 71 is a 73% red rate, and most of that is one conflict: a required checksum field added to an upload-slot request on 26 August, on an endpoint the frontend does not call, which stayed red across roughly 40 pipelines until a “re-generate types” commit on 2 September cleared it. For eight days every branch’s test job printed a conflict that no reviewer needed to act on, and the one on 4 September that a reviewer does need to act on prints in exactly the same place, in exactly the same format, into a job that goes green regardless. The warning channel has the same problem as the spec-drift check I run on this site: a report nobody is forced to read is a report that stops being read.

Scope it to what the frontend calls, then block

The fix is not to make the check blocking as-is — that would have blocked every branch for eight days over an endpoint nobody uses. The fix is to narrow the surface first. The frontend references 83 distinct paths of the 139 in the spec; the canary compares all 139. Filtering the intersection down to the paths the frontend actually calls, plus the schemas reachable from them, removes the entire class of “true, but not ours” conflicts — the checksum field would never have fired — and leaves a check whose red means this frontend will break. That is the version that earns || true being deleted, and it is the next change on the project.

The general lesson I take from it is about the order of operations for any canary. Build it directional so it tolerates the workflow you actually have; ship it warn-only so its false-positive rate becomes a measured number instead of a fear; then use that number to scope it before you make it block. Skipping the middle step gets the check deleted. Staying in it gets the check ignored.