What: /api/search/dataset/ can serve a stale total for up to an hour after a dataset is published.
Why it matters: search and the sitemap disagree, and the prod read-only check test_sitemap_datasets_count_matches_backend fails and rolls back the deploy.
Evidence: frontend prod deploy 35085603469 — sitemap 238 datasets, search total 237 before and after the crawl; minutes later every cache key read 238.
Fix: not fixed. Two likely causes in api/signals/dataset_signals.py / paginated_elastic_view.py, unconfirmed:
- The version bump runs in
pre_save, before the save commits and ES reindexes, so a search in that window re-caches the old count under the new version.
CACHES sets no TIMEOUT, so the version key expires after Django's default 300s and resets to 0, while results are cached for 3600s — old entries become reachable again.
What:
/api/search/dataset/can serve a stale total for up to an hour after a dataset is published.Why it matters: search and the sitemap disagree, and the prod read-only check
test_sitemap_datasets_count_matches_backendfails and rolls back the deploy.Evidence: frontend prod deploy 35085603469 — sitemap 238 datasets, search total 237 before and after the crawl; minutes later every cache key read 238.
Fix: not fixed. Two likely causes in
api/signals/dataset_signals.py/paginated_elastic_view.py, unconfirmed:pre_save, before the save commits and ES reindexes, so a search in that window re-caches the old count under the new version.CACHESsets noTIMEOUT, so the version key expires after Django's default 300s and resets to 0, while results are cached for 3600s — old entries become reachable again.