# When the search index disagrees with the source of truth Give each field one owner instead of trusting one system. On a Shopify storefront, a third-party search index lagged the catalogue and product cards showed retired colourways. Imagery moved to the catalogue, read per product and cached for 10 minutes. Ranking, filters, price, stock and badges stayed with the index. When a vendor sent an exact variant id, that id won over our own colour matching. Published: 2026-10-28 Canonical: https://umar.codes/search-index-vs-source-of-truth ## Revisions - 2026-10-28 — Created from the storefront repository's commits, code comments and context document (anonymized), read that day. - 2026-10-28 — Published as a backfill of the missed Oct 26 slot; auto-drafted by the publishing run because no draft existed for the row. --- Give each field one owner. Do not pick one system to trust for everything. On a Shopify storefront I worked on, a third-party search index lagged the catalogue, and product cards showed colours the shop no longer sold. Imagery moved to the catalogue, cached for 10 minutes. Ranking, filters, price, stock and badges stayed with the index. That sentence took four fixes to earn, over nine days in August 2026. Each fix found a place where the old owner of a field was still being read. The client is anonymized; the numbers come from the commits and code comments. ## How a lagging index put a retired colour on a product card The storefront's collection pages render from a search and merchandising vendor's index, not from Shopify directly. The index gives each card its position, its filter facets, its price and stock, and a gallery of images. The vendor syncs from Shopify on its own schedule, so the index runs behind the catalogue. When a colourway's images were missing from the index gallery, the card fell back to the product's featured image. On 27% of the bestsellers collection, that featured image showed a discontinued colourway. A shopper looking at a card for one colour saw a photo of a different colour that was no longer for sale. A first fix tried to read the variant images from the index instead. It was reverted: on the US index, those fields were empty, and the gallery field held at most 50 images per product. The index could not supply the answer, however it was asked. ## Fix one: move imagery to the catalogue and leave everything else in the index The page loader now adds an `imagesByColour` field to each index record, from a cached read of Shopify's Storefront API for that product. A pure function picks the images for each colour in a fixed order: 1. Media whose alt text or filename matches the colour. 2. That colour's representative variant image. 3. The product's featured image, but only when it is not an orphaned colourway. 4. Nothing, so the card keeps its own fallback. Step 2 is the one that holds. The variant image was present for all 682 colourways sampled across four collections, including every case where filename and alt-text matching both failed. The index's gallery is cleared only when every colour on the product resolved, so a partial read never leaves a card worse than before. The cost is one Storefront API request per product. I measured two shapes: 24 uncached products read in parallel took 233 ms, and one batched query for the same 24 took 917 ms. The per-product read also caches better, because a batched cache key changes with every filter and sort. An unbounded page size would turn into one origin request per record (`?limit=500` meant 500), so the enrichment is capped at 48 products, one page. It runs in the deferred part of the response, so it adds nothing to time to first byte, and on any failure the records stay as the index sent them. ## Fix two: every reader of the old field has to change too The cards looked right after fix one, but a recommendations vendor still received the retired colour's image URL. Its tracking code read the product's featured image from the records before the step that replaced it had run. No shopper saw it; the vendor's data was wrong. The fix was small: read `imagesByColour` first, then a colour-matched image from the index. A route test now pins that the vendor gets the enriched images. The lesson was not small. Moving ownership of a field means finding every reader of that field, and the reader on screen is only the easiest one to find. ## Fix three: a 24-hour cache rebuilt the lag that fix one removed A week later, two colourways of a knit jacket on a preview collection showed only the packshot. Shopify had the model shots. Merchandising had uploaded them colour by colour, hours after our cache read the product. The commit message says it in one line: imagery moved off the index because the index lags, and a day-long cache rebuilt the same lag. The fresh window went from 86,400 seconds to 600. The stale-while-revalidate window stayed at 604,800 seconds, seven days, so no request ever waits for a refresh; the stale window carries the performance, and the short fresh window carries the correctness. Shortening the window was not enough on its own. The framework's subrequest cache ages a stored entry against the cache policy it was written with, so entries already stored under the 24-hour policy would have stayed fresh for the rest of their day. The fix adds a version number to the query variables, so a change of version is a change of cache key and works as a purge. A comment in the query text would not have worked, because the query is minified before it is hashed and the minifier strips comments. The proof was the jacket itself. Before the fix, the cached entry held one image for each of the two colourways. With the cache bypassed, Shopify returned six and seven. ## Fix four: when the vendor is right and our code is wrong The last fix went the other way. Campaign cards on product pages come from the recommendations vendor, and one showed a tote bag in a different colour from the one the vendor's own widget showed. The vendor sent an exact variant id. Our batch endpoint ignored it for matching, looked for the product page's colour name exactly, missed, and fell back to the product's first variant. The variant id was used only as part of the cache key. Now the endpoint resolves in a fixed order: the variant id, then an exact colourway name, then a colourway whose name contains the colour. The variant id step is opt-in, used only by the vendor's cards; cards driven by the index or the CMS keep colour as the authority. The resolver has 14 test cases, and the nine other request shapes return byte-identical results to before. ## Each field has one owner, and the owner is the system that changes it After the four fixes, the code and the repository's context document state the same rule: imagery comes from Shopify, never from the index, and only imagery moves. | Field | Owner | Why | |---|---|---| | Order and ranking | Index | Merchandising order is set there, not in the catalogue. | | Filters and facets | Index | The index computes them over the whole result set. | | Price, stock, badges | Index | Kept there by decision. Its one price fault was a different bug, below. | | Imagery | Catalogue | Merchandisers upload images colour by colour; the index catches up hours later. | | Variant on a vendor's card | Vendor's variant id | The id names the exact variant; a colour name only approximates it. | The price fault was real, but it was not lag. The index's product-level price collapsed to the lowest price across every colourway, so a $110 colour showed as $77 with a Sale badge because a sibling colour was on sale. The fix read the per-variant price from the same index. The field kept its owner; the code had been reading the wrong level of it. Two things I would carry to the next project: - **A cache is part of the ownership decision.** Moving a field to the fresher system buys nothing if the cache in front of it is older than the lag you moved away from. Set the fresh window from how fast the field changes, not from what is cheap. - **Ownership drifts in documents too.** It is the same [drift between a spec and its code](/spec-drift-detection), one level down. As I write this, the context document on the main branch still says card images are cached for 24 hours, ten weeks after fix three made it 10 minutes. The index was not the only thing that lagged its source of truth.