ci: actually run the test suite - #2
Merged
Merged
Conversation
This repo has 51 tests and no workflow has ever run them. Adds tests.yml (push to main + PRs) and a smoke-gate step in nightly.yml. Two decisions worth recording: paths allowlist, not paths-ignore. scrape.yml commits data/ every 15 minutes (~21-28 commits/day). A naive `on: push` fires on all of them and buries real failures. An allowlist is used because a future scraper output directory would silently re-enable that storm under paths-ignore but stays excluded here. The nightly step runs BEFORE the backfill, not after. Once db/termo.db exists, 15 skipif-gated tests un-skip and assert frozen DATA constants (universe_size == 947, median_pt_days == 22, top slug pt-modul-toporasi). universe_size is derived from data/harta.html, which the scraper rewrites from CMTEB's live map - so the day CMTEB adds one PT, a post-backfill pytest would fail the nightly and leave the site silently serving stale data. Running before the backfill keeps those 15 skipped and the step a ~10s code gate. Data invariants stay pipeline.validate's job, which already gates the release. Actions are pinned to SHAs (SEC005); permissions are contents: read only, since this job never commits, uploads or deploys. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Verified: paths-allowlist YAML is valid and matches real directories (pipeline/, scraper/, scripts/, tests/); the new pytest check on this PR already ran and passed in ~10s, self-confirming the trigger and the skip-count reasoning. Nightly gate is correctly placed before backfill so the 15 db-gated tests stay skipped and can't block a release on a legitimate CMTEB universe change. permissions: contents: read is correctly scoped, actions are SHA-pinned, no untrusted input reaches any run: block.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
51 tests exist in this repo. No workflow has ever run any of them.
What
tests.yml(new) — pytest on push-to-main and PRsnightly.yml— a smoke-gate stepTwo decisions worth reviewing
pathsallowlist, notpaths-ignore.scrape.ymlcommitsdata/every 15 minutes (~21-28 commits/day). A naiveon: pushfires on every one and buries real failures in the run history. I used an allowlist rather thanpaths-ignore: ['data/**']because a future scraper output directory would silently re-enable that storm underpaths-ignore, but stays excluded under an allowlist.The nightly step runs BEFORE the backfill. This is the load-bearing bit. Once
db/termo.dbexists, 15skipif-gated tests un-skip and assert frozen data constants —universe_size == 947,median_pt_days == 22, top slugpt-modul-toporasi.universe_sizecomes fromdata/harta.html, which the scraper rewrites from CMTEB's live map. So the day CMTEB adds one PT to that map, a post-backfill pytest would fail the nightly, and since validate/upload/deploy-hook all sit downstream, the site would silently serve stale data indefinitely.Before the backfill,
db/termo.dbdoes not exist, those 15 stay skipped, and the step is a ~10s pure-code gate. Data invariants remainpipeline.validate's job, which already gates the release properly.Expected CI result
36 passed, 15 skipped— the skips are correct, not a misconfiguration. A run reporting51 passedwould mean a stale local database was present.Security
permissions: contents: read— this job never commits, uploads or deploys${{ github.event.* }}reaches anyrun:block🤖 Generated with Claude Code