Add an ERT test suite and run it in CI - #66
Merged
Merged
Conversation
This package had no automated tests -- the Eask `test' script was a
placeholder that only errored -- and CI ran `make all', which compiles
but never executes anything. Every bug fixed recently (a destructive
`nconc', a dropped command name, a version probe that read the launcher
instead of PHPStan, a swallowed parse fallback) was invisible on
inspection and only showed up when the code ran, so it is exactly the
kind of thing a test suite should pin down.
The tests cover the pure and stubbable functions where those bugs lived:
* test/phpstan-test.el -- container detection, version parsing, editor
mode gating (through the version cache, so it does not shell out),
path normalization, and `phpstan-get-command-args' (that it keeps the
command name for the `(STRING . (ARGS...))' form, does not mutate
`phpstan-executable' or `:options', and rewrites the config path for
a container). Project and filesystem access is stubbed with
`cl-letf', so the tests are hermetic.
* test/flycheck-phpstan-test.el -- that the JSON report is found even
when a container runtime prefixes it with progress on STDERR, and
that a no-JSON failure is surfaced as a warning rather than dropped.
Each test was checked to fail when its bug is reintroduced, so they
verify behaviour rather than merely describe it.
`make all' now runs `eask test ert' after compiling, so the existing
CI (which calls `make all') exercises the suite on every matrix job,
including the Emacs 27.2 one via Eask.27. The 27.2 helper script runs
the tests too. `Eask' gains a `test' script for `eask run script test';
`Eask.27' is kept in sync.
zonuexe
force-pushed
the
test/add-ert-suite
branch
from
July 17, 2026 16:03
5e91ff2 to
a634fd0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the package's first automated tests and runs them in CI.
Why
There were no tests: the Eask
testscript was a placeholder that only errored, and CI ranmake all, which compiles but executes nothing. Every bug fixed in the recent PRs (#62–#65) — a destructivenconcgrowingphpstan-executable, the(STRING . (ARGS...))form dropping the command name, the version probe readingdocker --versioninstead of PHPStan, the parse fallback silently discarding failures — was invisible on inspection and only surfaced when the code ran. That is exactly what a test suite should catch.What
test/phpstan-test.el— container detection (phpstan--container-runtime-command/-executable-p), version parsing (phpstan--version-from-output), editor-mode gating (through the version cache, so it never shells out), path normalization, andphpstan-get-command-args: that it keeps the command name for the(STRING . (ARGS...))form, does not mutatephpstan-executableor the caller's:options, and rewrites the config path to the container mount point. Project and filesystem access is stubbed withcl-letf, so the tests are hermetic.test/flycheck-phpstan-test.el— that the JSON report is found even when a container runtime prefixes it with progress on STDERR, and that a no-JSON failure is surfaced as a warning instead of dropped.Each test was verified to fail when its bug is reintroduced (e.g. swapping
appendback tonconcfails the two mutation tests; reverting the JSON detection tostring-prefix-pfails the stderr-prefix test), so they pin behaviour rather than merely describe it.Wiring
make allnow runseask test ertafter compiling, so the existing CI — which already callsmake all— runs the suite on all nine matrix jobs with no workflow change, including the Emacs 27.2 job throughEask.27. The local.github/run-emacs-27-2.shhelper runs the tests too.Easkgains atestscript (eask run script test);Eask.27is kept in sync.Verification
13/13 pass on system Emacs and on Emacs 27.2 (via nix-emacs-ci, MELPA Stable deps).
make allruns compile + test cleanly.