Checking for staleness

npx shotlist --check

Re-shoots every recipe and compares each against the committed image, reporting the ones that changed. Exits non-zero if any did, so it can run in CI. A recipe that installs nowhere is skipped: there is nothing to compare against.

Writing a diff image

To see what changed rather than how much, add --diff. For every shot that moved it writes a three-up into <paths.out>/diff/: the committed image, the re-shot one, and the re-shot one again with every changed pixel tinted. A percentage alone does not tell you whether to bless it or fix it.

  CHANGED  order-row — 2.13% of pixels differ
           committed: content/guide/images/order-row.png
           re-shot:   screenshots/out/order-row.png
           diff:      screenshots/out/diff/order-row.png

When the two are different sizes there is nothing to overlay, so the image is the two panels and the size change is the whole story.

Reporting the run as JSON

--json puts the report on stdout and moves everything written for a person to stderr, which is what makes --check --json > report.json leave a usable file.

{
  "changed": 1,
  "total": 2,
  "drift": [{ "field": "chromium", "was": "141.0.0.0", "now": "139.0.0.0" }],
  "results": [
    {
      "name": "order-row",
      "status": "changed",
      "ratio": 0.0213,
      "shot": "screenshots/out/order-row.png",
      "against": "content/guide/images/order-row.png",
      "diff": "screenshots/out/diff/order-row.png"
    }
  ]
}

status is one of same, changed, new, skipped or failed, and drift is what the machine-mismatch warning would have said — so a job can tell a re-render apart from a regression without parsing prose.

The machine the images were taken on

--install writes shotlist.baseline.json beside the config, recording the shotlist, Playwright and Chromium versions and the platform. Commit it with the images.

A different Chromium rasterises text differently, and a different platform has different faces to rasterise — either moves pixels without the site moving at all. When --check runs somewhere else, it says so before the results, so a difference is not read as a regression that was never there.

! this is not the machine the committed images were taken on:
    chromium: 141.0.0.0 → 139.0.0.0
    platform: darwin → linux
  Differences below may be that, rather than the site.

Leaving a region out of the comparison

To keep a region in the image but out of the comparison, list a query for it under check.ignore. Masking covers a region over; this leaves it visible and excuses only its contents. Use it for the figure the screenshot is about — a result, a live total — which has to stay in the shot and will not be the same twice.

name: roll-form
clip: { css: '.card' }
check:
  ignore:
    - { css: 'output#result' }

Only the contents are excused, not the geometry. The region is blanked where it resolves to now, in both images — so if the box moves or changes size, what it used to cover is still compared, and the difference is reported. A shot whose widget renders nothing at all fails outright, because the query matches nothing. That is the assertion check: false gives up: it says the shot still works, without saying anything about what is inside.

A result that skipped something says so — same order-row (1 region not compared) — so a pass is not read as covering the whole image.