Skip to content

Track heuristic typing mismatches in a baseline file - #239

Open
krzema12 wants to merge 2 commits into
mainfrom
typing-mismatch-baseline
Open

krzema12 wants to merge 2 commits into
mainfrom
typing-mismatch-baseline

Conversation

@krzema12

@krzema12 krzema12 commented Nov 21, 2026 •

Copy link
Copy Markdown
Member

Stacked on #244.

Adds typing-mismatches.yml: a checked-in baseline of every case where a typing in this catalog differs from the type inferred by TypingHeuristics (1,655 entries initially).

Regenerate it with:

DUMP_TYPING_MISMATCHES=true .github/workflows/test.main.kts

compareTypings() is the single source of truth used by both the dump and typing validation, so the baseline cannot drift from what CI checks.

CI fails on all baseline inconsistencies for the validated actions: a missing mismatch, a stale entry where the heuristic now returns the actual type, or an entry with outdated actual/inferred typings. All issues across all actions and inputs are collected into the validation error report from #244, so the run reports everything at once.

PRs keep validating only changed typings; main and the nightly run validate the whole catalog. Only .yml files are stored, with no heuristic reason/description text, so explanatory wording changes don't churn the baseline.

@krzema12
krzema12 changed the base branch from main to graphite-base/239 November 21, 2026 13:01
@krzema12
krzema12 force-pushed the typing-mismatch-baseline branch from 7a0c7f6 to 0822173 Compare November 21, 2026 13:01
@krzema12
krzema12 changed the base branch from graphite-base/239 to refactor-typing-comparison November 21, 2026 13:01

krzema12 commented Nov 21, 2026 •

Copy link
Copy Markdown
Member Author

@krzema12
krzema12 force-pushed the typing-mismatch-baseline branch from 0822173 to 36870a1 Compare November 21, 2026 13:07
@krzema12
krzema12 force-pushed the refactor-typing-comparison branch from 32ae653 to ff81ba5 Compare November 21, 2026 13:36
@krzema12
krzema12 force-pushed the typing-mismatch-baseline branch from 36870a1 to c3bf024 Compare November 21, 2026 13:36
krzema12 added a commit that referenced this pull request Nov 21, 2026
Preliminary refactoring for the typing mismatch baseline work (stacked
under #239).

Moves the inference/actual-typing comparison in `validateTypings()` into
`compareTypings()`, which returns structured `TypingInference` records.
Validation still prints exactly the same warning lines and compares
actual vs inferred typings inline — no behavior change.

Also extracts two small helpers: `Path.toActionCoords()` for parsing the
action path into coordinates, and `Map<*, *>.names()` for reading
input/output names from a manifest or typings file.
Base automatically changed from refactor-typing-comparison to main November 21, 2026 17:51
@krzema12
krzema12 force-pushed the typing-mismatch-baseline branch 3 times, most recently from d1fa41c to 239bfe6 Compare November 21, 2026 18:39
@krzema12
krzema12 changed the base branch from main to graphite-base/239 November 21, 2026 18:50
@krzema12
krzema12 force-pushed the typing-mismatch-baseline branch from 89e97ae to 8e13756 Compare November 21, 2026 18:50
@krzema12
krzema12 changed the base branch from graphite-base/239 to graphite-base-ref-workaround November 21, 2026 18:51
@krzema12
krzema12 force-pushed the typing-mismatch-baseline branch from f3434a9 to f6891f0 Compare November 21, 2026 18:53
@krzema12
krzema12 force-pushed the graphite-base-ref-workaround branch from e9b46a4 to 19f4b62 Compare November 21, 2026 18:53
krzema12 added a commit that referenced this pull request Nov 21, 2026
Graphite reports a synthetic `graphite-base/<PR>` branch as
`github.base_ref` for stacked PRs. That branch does not exist on the
remote, so `Validate typings` fails when it tries to fetch it to detect
changed actions.

This PR resolves the real base branch through the GitHub API whenever
`github.base_ref` has the `graphite-base/` prefix. It is split out from
#239 so the baseline change stays focused.
Base automatically changed from graphite-base-ref-workaround to main November 21, 2026 18:54
@krzema12
krzema12 force-pushed the typing-mismatch-baseline branch from f6891f0 to 9553d56 Compare November 21, 2026 18:54
@krzema12
krzema12 changed the base branch from main to graphite-base/239 November 21, 2026 19:00
@krzema12
krzema12 force-pushed the typing-mismatch-baseline branch from 9553d56 to 6940e45 Compare November 21, 2026 19:00
@krzema12
krzema12 changed the base branch from graphite-base/239 to collect-validation-errors November 21, 2026 19:00
krzema12 added a commit that referenced this pull request Nov 21, 2026
Refactors `validateTypings()` so action-level validation problems
(lowercase names, schema issues, missing manifests, input/output
mismatches) are collected into a list instead of being printed and
failed on immediately. At the end of the run all issues are printed
together, sorted, with the affected action included in each message.

No behavior change: the same conditions still fail validation. Split out
of #239 to keep the baseline change focused.
Base automatically changed from collect-validation-errors to main November 21, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant