# Running spec-drift detection on a real repo Spec-drift detection asks whether an implementation still matches what its specs declare. This log runs sdx check on this site — the first real Astro consumer of specdx's framework-agnostic declared artifacts — and reports the results: 25 checks across three specs, a 100% first run that mostly proves freshness, two deliberately broken declarations to verify both failure modes, and the drift classes the checker still can't see. Published: 2026-08-09 Updated: 2026-08-12 Canonical: https://umar.codes/spec-drift-detection ## Revisions - 2026-07-30 — Created as an outline; body waits on the specdx alpha that ships declared artifacts. - 2026-07-30 — Wrote the full body from the 0.4.0-alpha.8 run; sdx check added to the site's verification gate. - 2026-08-09 — Published, pulled forward from the planned 2026-09-28 date. Added a dated update note: the suite grew to four specs and 29 declarations since the logged run, exercising pending planned artifacts. - 2026-08-12 — Corrected the description of tokens.css, which had claimed to be the only place colour may be defined — three static assets mirror it. Added the correction as a worked example of the third drift class the checker can't see. --- Spec-drift detection checks that an implementation still matches its specs without anyone re-reading both. This site is the test subject: three specs, an Astro codebase, and `sdx check` now wired into the verification gate. Before declared artifacts landed in specdx, the honest answer here was exit code 3 — nothing checkable. Here's what changed, verified. ## Exit code 3 is an honest zero [specdx](/sdx)'s `check` command finds drift by extracting checkable surfaces from the implementation — routes, types, tests — and those extractors were framework-specific. Pointed at this Astro repo, the scan came back `framework=none detected`, routes and tests not scanned, and exited 3: "coverage not assessed". That exit code is a design decision worth copying: zero errors because nothing was checked is not a pass, and the tool refuses to let the two look alike. Under this site's [dogfooding contract](/dogfooding-specdx) that friction became an upstream issue rather than a workaround, and the fix that closed it went framework-agnostic instead of adding an Astro extractor: any spec can now declare the `artifacts:` it claims to produce — file paths, plus named exports per file — and `check` verifies them with no framework knowledge at all. The design went in as the specdx suite's first ADR; per-framework extraction stayed deferred. ## Declaring artifacts in three specs The declarations took twelve entries across the site's three specs. The crawler-logger design names its four files — the middleware, the export and serve scripts, and the bot-list module with its two exports. The project-context spec declares the modules its rules govern: the identity constants (`ENTITY_STRING` and friends), the content schema with its reserved-slug list, the structured-data builders, the OG renderer, and `tokens.css` — the source of truth for every colour the site's pages render. The content-calendar epic declares the plan file, the posts directory (directories verify as paths), and the trigger-eval harness entry point. Twelve paths plus thirteen named exports makes 25 checks. A declaration asserts existence — this path is real, this export is present — and nothing more; that limit is the price of working on any stack. ## The first run scored 100%, which mostly proves freshness `sdx check` on the declared suite: exit 0, `artifacts: 25 verified`, 100% implementation coverage. The honest reading is that this proves freshness more than correctness — the crawler logger shipped four days before this run, and its spec was corrected *during* implementation, so there'd been no time to drift. A checker that has never failed is an unverified checker, so I broke it twice on purpose. First a plausible export rename: the upstream issue's illustrative example had guessed the bot list was called `BOT_SIGNATURES`; the code's real export is `BOT_USER_AGENTS`. Declaring the guessed name — exactly what pasting the example blind would have done — dropped coverage to 96% and exited 1 with a finding that names both legal fixes: "Export BOT_SIGNATURES from src/lib/bots.ts or update the artifacts list in crawler-logger." Then a missing file: declaring a renamed script produced the mirror finding ("Create scripts/serve-static.mjs or update the artifacts list"). Both failure modes work, and both findings treat drift as symmetric — the code may be wrong or the spec may be, and the tool doesn't presume which. With that verified, `specdx check` joined the site's verification gate: `pnpm check-specs` now runs validate, lint, and check on every commit and in CI. The interesting findings arrive later, the first time a refactor moves a declared file and the gate — not a reader — notices. ## Drift the checker still can't see Three classes stay invisible, deliberately. Config-key assertions were deferred with the ADR, so the redirect rules in `vercel.json` and the spec suite's own wiring can drift silently. Route extraction for Astro was deferred too — the framework override list still reads express, hono, nextjs — so no spec can claim "this page exists" and have it checked. And existence-checking says nothing about content: `check` proves `ENTITY_STRING` is exported, while the rule that its bytes stay identical everywhere is enforced by a different tool in the same gate (`check-content`). Layered checkers, each honest about scope, beat one checker pretending to cover everything. The reopen criterion from the ADR is recorded: if declarations prove insufficient on a real write-up, the deferred extractors get revisited. This write-up is the first data point, and declarations were sufficient. **Update, 2026-08-09.** The suite has grown since the run logged above: a fourth spec — a scheduled drain moving crawler logs to durable storage — entered as a draft declaring artifacts it *plans* to produce, and the gate now reports four specs and 29 declarations. The two unbuilt ones (a cron endpoint file, one middleware export) show as pending, not failing: planned artifacts are excluded from the coverage score until the spec's status flips to approved, at which point they become enforceable. A checker that treats "not built yet" as different from "missing" is the same honesty as exit code 3, applied to the other end of a spec's life. **Update, 2026-08-12.** The clause describing `tokens.css` above used to read "the only place colour may be defined". It had already stopped being true when this post was published, and nothing here caught it. A later branch added a contrast and palette check, which found three files carrying literal hex outside `tokens.css`: the OG renderer, which runs in Node with no stylesheet to read; `favicon.svg`; and `favicon.ico`, whose colour is binary pixels. `sdx check` verified the `tokens.css` declaration on every run throughout — the path was real and the exports were present, which is all a declaration claims. So this is the third drift class above, caught on this post rather than hypothesised: existence held while the prose describing it rotted, and no checker in the gate was looking at the sentence. The repair was not to soften the claim. Two of the three mirrors are now asserted against `tokens.css` by the accessibility check, and the third is documented as hand-maintained — which is the only move that makes the corrected sentence stay true without someone remembering to re-read it.