ci(release): test installer in disposable environments - #762
Open
ajulaybeeb wants to merge 1 commit into
Open
Conversation
Add a self-contained shell test suite and a dedicated GitHub Actions
workflow to exercise install.sh without network access, root, or real
release artifacts.
Tests (tests/installer/test_install.sh) — 10 scenarios:
Primary flow
- Clean install: downloads, verifies checksum, and installs the binary
- Upgrade: a second install run replaces an already-present binary
Boundary cases
- Success message contains the resolved version tag
- No temporary files (archive, checksums.txt) left behind after exit
Failure paths
- Checksum mismatch: installer aborts before moving the binary
- Download failure: curl stub returns non-zero; installer exits non-zero
- Empty release tag: installer prints an actionable error and exits
- Unsupported OS (FreeBSD stub): installer exits 1 with a clear message
- Unsupported architecture (mips stub): same
- Checksum download failure: installer exits 1 before extraction
Test strategy:
- make_patched_installer() produces a copy of install.sh where each of
the three curl calls is replaced by an inline stub function.
- Stubs are embedded via printf into the patched script so child bash
processes inherit them without needing export -f.
- Tests run in mktemp sub-directories; the suite EXIT trap cleans up.
- INSTALL_DIR is overridden to a local fake_bin so no sudo is needed.
CI workflow (.github/workflows/installer-tests.yml):
- installer-linux: ubuntu-latest, runs full test suite
- installer-macos: macos-latest, runs full test suite
- installer-shellcheck: lints install.sh and the test script with
shellcheck -S warning
- Workflow triggers on pushes/PRs that touch install.sh or the test
directory, plus workflow_call for use as a dependency.
install.sh improvements:
- set -euo pipefail (was set -e only)
- INSTALL_DIR env-var override for tests and headless installs
- Inline || { ...; exit 1; } error handlers (enables line-level patching)
- Downloads archive and checksums into an auto-cleaned WORK_DIR temp dir
instead of the current directory — eliminates temp-file leaks on failure
- Expanded error messages with actionable guidance for unsupported OS,
unsupported arch, missing curl, empty release tag, download failure,
and checksum download failure
- Uninstall instruction printed on success
README.md:
- Documents checksum verification, security note, platform compatibility
table, custom INSTALL_DIR, and uninstall instructions
Closes Nanle-code#656
|
@ajulaybeeb Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
Collaborator
|
@ajulaybeeb Please make sure to fix CI issues, very important!!! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/installer/test_install.sh— a self-contained shell test suite forinstall.shwith 10 scenarios covering clean install, upgrade, checksum failure, download failure, unsupported OS/arch, empty tag, temp-file cleanup, and success message.github/workflows/installer-tests.yml— CI workflow that runs the suite onubuntu-latestandmacos-latest, plus ashellchecklint jobinstall.shwithset -euo pipefail, aWORK_DIRtemp dir (auto-cleaned on exit), actionable error messages, anINSTALL_DIRoverride, and uninstall guidanceREADME.mdwith a platform compatibility table, security note, checksum verification instructions, customINSTALL_DIRusage, and uninstall stepsWhy
Issue #656 requires that
install.shand the release CI be tested in disposable environments. Previously, the installer had no automated tests: there was no way to catch regressions in checksum verification, error handling, or platform detection without running against a real release. The installer also had several gaps:set -ewithoutpipefail, temp files written to the current directory (left behind on failure), noINSTALL_DIRoverride, and no uninstall guidance.Implementation
tests/installer/test_install.sh(new, 10 tests)The test harness builds a patched copy of
install.shin amktempdirectory where the threecurlcalls are replaced by configurable stub functions embedded directly in the patched script (noexport -frequired). Each test runs in an isolated sub-shell with its ownfake_bintarget andSTUB_*environment variables:test_clean_install_succeedstest_upgrade_replaces_existing_binarytest_checksum_mismatch_failstest_download_failure_exits_nonzerotest_empty_tag_exits_nonzerotest_unsupported_os_exits_nonzerotest_unsupported_arch_exits_nonzerotest_no_temp_files_left_after_successtest_success_message_contains_versiontest_checksum_download_failure_exits_nonzero.github/workflows/installer-tests.yml(new)installer-linux—ubuntu-latest, runsbash tests/installer/test_install.shinstaller-macos—macos-latest, runsbash tests/installer/test_install.shinstaller-shellcheck— lints bothinstall.shand the test suite withshellcheck -S warningpush/pull_requestthat touchinstall.shortests/installer/, plusworkflow_callinstall.shset -euo pipefail(wasset -e)WORK_DIR=$(mktemp -d)with anEXITtrap — no temp files left on failureINSTALL_DIRenv-var override (tests override to a localfake_bin)|| { echo "..."; exit 1; }error handlers (enables line-level sed patching)README.mdINSTALL_DIRoverride exampleTesting
bash tests/installer/test_install.sh
StarForge installer tests ──────────────────────────────────────────────── [PASS] test_clean_install_succeeds [PASS] test_upgrade_replaces_existing_binary [PASS] test_checksum_mismatch_fails [PASS] test_download_failure_exits_nonzero [PASS] test_empty_tag_exits_nonzero [PASS] test_unsupported_os_exits_nonzero [PASS] test_unsupported_arch_exits_nonzero [PASS] test_no_temp_files_left_after_success [PASS] test_success_message_contains_version [PASS] test_checksum_download_failure_exits_nonzero ──────────────────────────────────────────────── Results: 10 tests, 0 failed
Not tested: Full macOS CI run — validated by
installer-macosjob in this PR once merged.Scope / Risk
install.shremains backward compatible; the newINSTALL_DIRvariable is opt-ininstall.sh,tests/installer/,.github/workflows/installer-tests.yml,README.mdIssue
Closes #656