Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/backlog/COR-103-verification-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Each check fails first on one named past mistake.
| COR-103-T6 | Signal and refuses commands | agent | COR-103-T3 | done |
| COR-103-T7 | One install and uninstall path | agent | - | done |
| COR-103-T8 | Update through the target updater | agent | COR-103-T7 | todo |
| COR-103-T9 | Sign off an isolated checkout | agent | COR-103-T7 | todo |
| COR-103-T9 | Sign off an isolated checkout | agent | COR-103-T7 | done |
| COR-103-T10 | Release with a receipt | agent | COR-103-T7, COR-103-T9 | todo |
| COR-103-T11 | macOS job in the template | agent | - | done |
| COR-103-T12 | Author path from the published tag | agent | - | done |
Expand Down
6 changes: 4 additions & 2 deletions docs/north-star/contract/extension-repository.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,10 @@ The template refuses `main` and each Go keyword as a domain, such as `type`. It
A push can skip Lefthook, so its GitHub workflow runs `just check` again on `ubuntu-24.04`.
Public repositories also run the same steps on `macos-15`, and both runners are free.

The repository tests itself through the [test kit](test-kit.md). Its `justfile` has six recipes:
The repository tests itself through the [test kit](test-kit.md). Its `justfile` has seven recipes:

- `just check` runs every check: `go mod tidy -diff`, `gofmt`, `go vet`, and `go test`, whose test kit installs the extension into scratch folders
- `just signoff` checks the committed SHA in isolation, then signs that SHA, as [Local checks](../decisions/D016-local-ci.md) defines
- `just install` builds the executable and exports its sidecars from that build. It replaces the previous install of the slug in the extensions folder of [Discovery](discovery.md), the executable last
- `just uninstall` removes the install of the slug from that folder, the executable first. With nothing installed, it says so and exits 0
- `just acceptance [fixture]` runs `acceptance.sh` as an agent would, with `--json` and `--no-input`. It installs the extension and the `ncly` of the pinned Nicely version into scratch XDG folders. It works on a copy of the fixture and checks that the fixture keeps its checksum. Without a fixture, it makes an input
Expand All @@ -42,9 +43,10 @@ The repository tests itself through the [test kit](test-kit.md). Its `justfile`

`just install` grants no key. A human grants one after reviewing the extension, as [Grants](grants.md) defines.

The SDK package `extensionrepo` owns install and uninstall.
The SDK package `extensionrepo` owns install, uninstall, and signoff.
The template's `internal/export/` program calls `extensionrepo.Main(Extension)`.
The recipes each call that program in one line: `go run ./internal/export install` or `go run ./internal/export uninstall`.
Signoff calls `go run ./internal/export signoff`.
Its `export <folder>` subcommand writes the embedded sidecars of the staged build.

Install stages both builds beside the destination with one `go build`.
Expand Down
22 changes: 22 additions & 0 deletions docs/north-star/decisions/D016-local-ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,26 @@ Decided 2026-10-05. Changed 2026-10-29 by [#373](https://github.com/pascalandy/n

**Why.** Local checks are faster, and `just signoff` proves the exact commit that reaches `main`. A push can skip the pre-push hook of Lefthook, so a second run on a clean runner shows a broken `main`. Nicely is public, so standard GitHub runners run that check for free.

`just signoff` uses `sdk/extensionrepo` in core and extension repositories.
The program freezes the full HEAD SHA and creates a detached scratch worktree at that SHA.
It clears `JUST_*` and `GIT_*` from its child environment before it resolves the repository or runs a check.
It replaces caller `GOFLAGS` with a blank value, which also masks a `GOFLAGS` that `go env -w` keeps. Overlays and module flags can replace committed source or dependencies.
It disables `GOWORK`, which can select external workspace modules.
Other persisted Go settings, such as `GOPRIVATE`, stay.
It runs that worktree's `just check` once, including its dependencies.
A failed check, a changed HEAD, or a changed tracked file prevents signoff.
Uncommitted changes in the caller cannot fix the checked commit.
After the checks pass, `gh signoff --commit <sha>` signs only the frozen SHA.
Success prints `Signed off on <sha>` with the full SHA.

Each run prints the path of its JSON receipt under Nicely's cache folder, in `signoff/`.
That folder must be outside the repository.
The receipt records the SHA, host OS, tool versions, and each invoked gate's exit code.
The current gate is `just check`.
The receipt exists before signoff and then records the exit code of `gh signoff`.
The run also keeps its check log, including failures.
The scratch worktree goes when the program returns.
After SIGKILL, run `git worktree remove --force <receipt-folder>/worktree` from the caller, using the printed receipt's folder.
Every run repeats the checks, even when a receipt already exists.

**Rejected.** Local checks alone, because a skipped hook then leaves a broken `main` unseen, as #373 found. GitHub Actions as the only gate, because Pascal often hits its limits, and local checks are faster. A required `check` status, because `signoff` already gates `main`.
231 changes: 180 additions & 51 deletions scripts/signoff-test.sh
Original file line number Diff line number Diff line change
@@ -1,93 +1,222 @@
#!/usr/bin/env bash
# signoff-test.sh proves just signoff for just test: it runs the check once,
# then gh signoff, even under JUST_NO_DEPS=true or --no-deps, and it signs
# nothing after a failed check. It runs on macOS and Linux, with bash 3.2.
#
# The caller's justfile is a copy of the real one, where only check is a
# stub. The stub writes CHECK-RAN from a dependency of check, because
# JUST_NO_DEPS and --no-deps skip exactly those. gh is a stub too, so no case
# reaches the network. See #213.
# Prove isolated signoff through core's recipe, with real Git and just.
# gh is always a stub. A dirty fix must not make a broken commit pass.
set -Eeuo pipefail
trap 'printf "%s: line %s failed: %s\n" "${BASH_SOURCE[0]##*/}" "${LINENO}" "${BASH_COMMAND}" >&2' ERR

SIGNOFF_TEST_ROOT=""
root=""
test_case="setup"

# shellcheck disable=SC2317,SC2329
fct_cleanup() {
if [[ -n "${SIGNOFF_TEST_ROOT}" ]]; then
rm -rf "${SIGNOFF_TEST_ROOT}"
fi
# Keep artifacts when trash is absent, such as on a hosted runner.
if command -v trash >/dev/null; then trash "${root}"; fi
}

fct_fail() {
cat "${root}/output" >&2
printf 'signoff-test: %s: %s\n' "${test_case}" "$*" >&2
exit 1
}

fct_setup() {
local repo_root
local repo_root variable
for variable in $(compgen -e); do
case "${variable}" in GIT_* | JUST_*) unset "${variable}" ;; esac
done
repo_root="$(cd "${BASH_SOURCE[0]%/*}/.." && pwd -P)"
root="$(mktemp -d "${TMPDIR:-/tmp}/ncly-signoff-test.XXXXXXXX")"
SIGNOFF_TEST_ROOT="${root}"
mkdir -p "${root}/bin" "${root}/caller/scripts"
export NCLY_SIGNOFF_TEST="${root}" PATH="${root}/bin:${PATH}"
mkdir -p "${root}/bin" "${root}/caller/scripts" "${root}/caller/signoff-fixture" "${root}/cache"
export NCLY_SIGNOFF_TEST="${root}" PATH="${root}/bin:${PATH}" XDG_CACHE_HOME="${root}/cache"
cat >"${root}/bin/gh" <<'SCRIPT'
#!/usr/bin/env bash
set -Eeuo pipefail
if [[ "$*" != signoff ]]; then
echo "gh stub: no answer for: $*" >&2
exit 1
case "$*" in
--version)
if git symbolic-ref --quiet HEAD >/dev/null; then
echo 'version probe is not isolated' >&2
exit 1
fi
echo 'gh version test'
exit 0
;;
signoff | signoff\ --commit\ *) ;;
*) echo "gh stub refuses: $*" >&2; exit 1 ;;
esac
if [[ "$*" == signoff\ --commit\ * ]]; then
jq -e --arg sha "${3}" '.sha == $sha and .checks[0].exit == 0 and .signoff == null' ../receipt.json >/dev/null
fi
echo SIGNOFF-RAN >>"${NCLY_SIGNOFF_TEST}/log"
printf '%s\n' "$*" >>"${NCLY_SIGNOFF_TEST}/signed"
git rev-parse HEAD >>"${NCLY_SIGNOFF_TEST}/signed-head"
[[ ! -e "${NCLY_SIGNOFF_TEST}/gh-fails" ]]
SCRIPT
chmod 755 "${root}/bin/gh"
cp "${repo_root}/justfile" "${root}/caller/justfile"
cp "${repo_root}/go.mod" "${repo_root}/go.sum" "${root}/caller/"
cp -R "${repo_root}/sdk" "${root}/caller/sdk"
cp "${repo_root}/scripts/signoff.sh" "${root}/caller/scripts/signoff.sh"
cat >>"${root}/caller/justfile" <<'JUST'

set allow-duplicate-recipes

# The stub check of scripts/signoff-test.sh replaces the real one, and only lists a dependency, as the real one does
check: signoff-test-check

signoff-test-check:
@echo CHECK-RAN >>"${NCLY_SIGNOFF_TEST}/log"
@test ! -e "${NCLY_SIGNOFF_TEST}/fail"
@git rev-parse HEAD >>"${NCLY_SIGNOFF_TEST}/checked"
@test -z "${GIT_DIR-}${GIT_WORK_TREE-}${GIT_SIGNOFF_TEST-}${JUST_SIGNOFF_TEST-}${JUST_NO_DEPS-}"
@test "$(cat fix.txt)" = fixed || { echo COMMITTED-FAILURE >&2; exit 1; }
@go test -count=1 ./signoff-fixture
@go env GOPRIVATE >"${NCLY_SIGNOFF_TEST}/goprivate"
@if [[ -e "${NCLY_SIGNOFF_TEST}/move-head" ]]; then git checkout --quiet --detach HEAD~; fi
@if [[ -e "${NCLY_SIGNOFF_TEST}/change-tracked" ]]; then echo changed >fix.txt; fi
@if [[ -e "${NCLY_SIGNOFF_TEST}/move-caller" ]]; then git -C "${NCLY_SIGNOFF_TEST}/caller" -c commit.gpgsign=false commit --quiet --allow-empty --message 'caller moved'; fi
JUST
cp "${repo_root}/scripts/signoff.sh" "${root}/caller/scripts/signoff.sh"
printf 'package fixture\n\nfunc fixed() bool { return false }\n' >"${root}/caller/signoff-fixture/fix.go"
cat >"${root}/caller/signoff-fixture/fix_test.go" <<'GO'
package fixture

import "testing"

func TestFix(t *testing.T) {
if !fixed() {
t.Fatal("COMMITTED-FAILURE")
}
}
GO
git -C "${root}/caller" init -q
git -C "${root}/caller" config user.name 'Signoff test'
git -C "${root}/caller" config user.email 'test@example.com'
git -C "${root}/caller" config core.hooksPath /dev/null
git -C "${root}/caller" config commit.gpgsign false
printf 'broken\n' >"${root}/caller/fix.txt"
git -C "${root}/caller" add .
git -C "${root}/caller" commit -qm 'broken commit'
sha="$(git -C "${root}/caller" rev-parse HEAD)"
: >"${root}/output"
: >"${root}/signed"
: >"${root}/signed-head"
}

# fct_expect <passes|fails> <log line>... runs the three forms of just signoff, and requires each to end as given and to write exactly these lines, in this order.
fct_expect() {
local outcome="${1}" form status
# Run core's recipe and check the observable result, including the checked SHA.
fct_run() {
local outcome="${1}" status=0
shift
printf '%s\n' "$@" >"${root}/log.expected"
for form in "just signoff" "env JUST_NO_DEPS=true just signoff" "just --no-deps signoff"; do
: >"${root}/log"
status=0
# shellcheck disable=SC2086
(cd "${root}/caller" && ${form}) >"${root}/output" 2>&1 </dev/null || status=$?
if [[ "${outcome}" == passes && "${status}" != 0 || "${outcome}" == fails && "${status}" == 0 ]]; then
cat "${root}/output" >&2
fct_fail "${form} exited ${status}"
fi
if ! cmp -s "${root}/log.expected" "${root}/log"; then
diff "${root}/log.expected" "${root}/log" >&2 || true
fct_fail "${form} ran other steps"
fi
done
: >"${root}/checked"
: >"${root}/signed"
: >"${root}/signed-head"
(cd "${root}/caller" && "$@") >"${root}/output" 2>&1 </dev/null || status=$?
if [[ "${outcome}" == passes && "${status}" != 0 || "${outcome}" == fails && "${status}" == 0 ]]; then
fct_fail "$* exited ${status}, expected ${outcome}"
fi
[[ "$(cat "${root}/checked")" == "${sha}" ]] || fct_fail 'check did not run once on the frozen SHA'
}

fct_main() {
trap fct_cleanup EXIT
fct_setup
test_case="passing check"
fct_expect passes CHECK-RAN SIGNOFF-RAN
test_case="failing check"
: >"${root}/fail"
fct_expect fails CHECK-RAN
printf 'just signoff runs the check once before gh signoff, under JUST_NO_DEPS and --no-deps too, and signs nothing after a failed check\n'
trap fct_cleanup EXIT
test_case="dirty fix"
printf 'fixed\n' >"${root}/caller/fix.txt"
printf 'package fixture\n\nfunc fixed() bool { return true }\n' >"${root}/caller/signoff-fixture/fix.go"
(cd "${root}/caller" && just check) >"${root}/output" 2>&1
fct_run fails just signoff
grep -qF COMMITTED-FAILURE "${root}/output" || fct_fail 'the committed defect did not fail the check'
[[ ! -s "${root}/signed" ]] || fct_fail 'signed the unfixed commit'
fct_receipt 1 null

git -C "${root}/caller" add fix.txt
git -C "${root}/caller" commit -qm 'committed fix'
sha="$(git -C "${root}/caller" rev-parse HEAD)"
test_case="dirty Go fix without overlay"
fct_run fails just signoff
grep -qF COMMITTED-FAILURE "${root}/output" || fct_fail 'the committed Go defect did not fail the check'
[[ ! -s "${root}/signed" ]] || fct_fail 'signed the unfixed Go commit'
fct_receipt 1 null
jq -n --arg fixed "${root}/caller/signoff-fixture/fix.go" '{Replace:{"signoff-fixture/fix.go":$fixed}}' >"${root}/overlay.json"
printf 'GOFLAGS=-overlay=%s/overlay.json\n' "${root}" >"${root}/go-env"
local form
for form in GOFLAGS GOENV; do
test_case="dirty Go fix through ${form}"
case "${form}" in
GOFLAGS) fct_run fails env GOFLAGS="-overlay=${root}/overlay.json" just signoff ;;
GOENV) fct_run fails env GOENV="${root}/go-env" just signoff ;;
esac
grep -qF COMMITTED-FAILURE "${root}/output" || fct_fail 'the overlay bypassed the committed Go defect'
[[ ! -s "${root}/signed" ]] || fct_fail 'signed the overlay-fixed commit'
fct_receipt 1 null
done
git -C "${root}/caller" add signoff-fixture/fix.go
git -C "${root}/caller" commit -qm 'committed Go fix'
sha="$(git -C "${root}/caller" rev-parse HEAD)"
printf 'dirty caller\n' >"${root}/caller/fix.txt"
printf 'untracked caller\n' >"${root}/caller/untracked.txt"
git -C "${root}/caller" diff >"${root}/before"
mkdir "${root}/decoy"
git -C "${root}/decoy" init -q
printf 'check:\n @exit 1\n' >"${root}/decoy/justfile"
printf 'GOPRIVATE=example.com/private\n' >"${root}/go-env-private"
for form in normal no-deps-env no-deps-flag caller-env caller-go-env; do
test_case="passing commit, ${form}"
case "${form}" in
normal) fct_run passes just signoff ;;
no-deps-env) fct_run passes env JUST_NO_DEPS=true just signoff ;;
no-deps-flag) fct_run passes just --no-deps signoff ;;
caller-env)
fct_run passes env GIT_DIR="${root}/decoy/.git" GIT_WORK_TREE="${root}/decoy" GIT_SIGNOFF_TEST=leak \
JUST_SIGNOFF_TEST=leak JUST_NO_DEPS=true JUST_JUSTFILE="${root}/decoy/justfile" JUST_WORKING_DIRECTORY="${root}/decoy" \
just --justfile "${root}/caller/justfile" --working-directory "${root}/caller" signoff
;;
caller-go-env)
fct_run passes env GOENV="${root}/go-env-private" just signoff
[[ "$(cat "${root}/goprivate")" == example.com/private ]] || fct_fail 'signoff dropped a persisted Go setting'
;;
esac
[[ "$(cat "${root}/signed")" == "signoff --commit ${sha}" ]] || fct_fail 'gh did not receive the frozen full SHA'
[[ "$(cat "${root}/signed-head")" == "${sha}" ]] || fct_fail 'gh ran outside the frozen checkout'
grep -qxF "Signed off on ${sha}" "${root}/output" || fct_fail 'missing landing-script output'
fct_receipt 0 0
done
git -C "${root}/caller" diff >"${root}/after"
cmp -s "${root}/before" "${root}/after" || fct_fail 'signoff changed the caller'
[[ "$(cat "${root}/caller/untracked.txt")" == 'untracked caller' ]] || fct_fail 'signoff removed caller files'
[[ "$(git -C "${root}/caller" worktree list --porcelain | grep -c '^worktree ')" == 1 ]] || fct_fail 'scratch worktree remains'

test_case="caller HEAD moves during the check"
: >"${root}/move-caller"
fct_run passes just signoff
[[ "$(cat "${root}/signed")" == "signoff --commit ${sha}" ]] || fct_fail 'signed the moved caller HEAD'
fct_receipt 0 0
[[ "$(git -C "${root}/caller" rev-parse HEAD)" != "${sha}" ]] || fct_fail 'caller HEAD did not move'
sha="$(git -C "${root}/caller" rev-parse HEAD)"
mv "${root}/move-caller" "${root}/caller-moved"

test_case="gh failure"
: >"${root}/gh-fails"
fct_run fails just signoff
if grep -q '^Signed off on ' "${root}/output"; then fct_fail 'reported success after gh failed'; fi
fct_receipt 0 1
mv "${root}/gh-fails" "${root}/gh-failed"
for form in move-head change-tracked; do
test_case="${form} during the check"
: >"${root}/${form}"
fct_run fails just signoff
[[ ! -s "${root}/signed" ]] || fct_fail 'signed a changed checkout'
grep -q 'during the check' "${root}/output" || fct_fail 'failed before the checkout guard'
mv "${root}/${form}" "${root}/${form}-done"
done
printf 'signoff: dirty fixes fail, frozen SHAs pass, caller files stay, and failed gates sign nothing\n'
}

fct_receipt() {
local receipt
receipt="$(sed -n 's/^signoff: receipt in //p' "${root}/output")"
[[ "${receipt}" == "${root}/cache/nicely/signoff/"*/receipt.json ]] || fct_fail 'receipt is not outside the repository'
jq -e --arg sha "${sha}" --argjson check "${1}" --argjson signoff "${2}" '
.sha == $sha and (.os == "linux" or .os == "darwin") and
([.tools.git, .tools.go, .tools.just, .tools.gh] | all(type == "string" and length > 0)) and
(.checks | length == 1) and .checks[0].command[0] == "just" and .checks[0].exit == $check and
(if $signoff == null then .signoff == null
else .signoff.command == ["gh", "signoff", "--commit", $sha] and .signoff.exit == $signoff end)
' "${receipt}" >/dev/null || fct_fail 'receipt does not describe the actual run'
[[ -s "${receipt%/*}/check.log" ]] || fct_fail 'receipt has no check log'
}

fct_main
exit 0
13 changes: 2 additions & 11 deletions scripts/signoff.sh
Original file line number Diff line number Diff line change
@@ -1,15 +1,6 @@
#!/usr/bin/env bash
# signoff.sh runs just check, then marks the pushed HEAD green with
# gh signoff, for just signoff. The check runs here, not as a dependency of
# the recipe, because JUST_NO_DEPS and --no-deps skip dependencies. It runs on
# macOS and Linux. scripts/signoff-test.sh proves it. See #213.
# Run the shared isolated-checkout signoff program. scripts/signoff-test.sh proves it.
set -Eeuo pipefail
# A caller can export JUST_NO_DEPS or JUST_JUSTFILE, which would run nothing, or the check of another justfile.
for just_variable in $(compgen -e); do
if [[ "$just_variable" == JUST_* ]]; then unset "$just_variable"; fi
done

root="$(cd "${BASH_SOURCE[0]%/*}/.." && pwd -P)"
cd "$root"
just --justfile "$root/justfile" --working-directory "$root" check
gh signoff
exec go run ./sdk/extensionrepo/cmd/extensionrepo signoff
27 changes: 26 additions & 1 deletion scripts/template-recipes-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ fct_test_files() {
fct_has "${repo}/.github/workflows/check.yml" " os: \${{ github.event.repository.private && fromJSON('[\"ubuntu-24.04\"]') || fromJSON('[\"ubuntu-24.04\",\"macos-15\"]') }}"
fct_has "${repo}/.github/workflows/check.yml" " runs-on: \${{ matrix.os }}"
fct_has "${repo}/qa.md" "# ${slug} QA"
for recipe in install uninstall; do
for recipe in install uninstall signoff; do
[[ "$(just --justfile "${repo}/justfile" --show "${recipe}" | sed '/^#/d')" == "${recipe}:"$'\n'" go run ./internal/export ${recipe}" ]] ||
fct_fail "just ${recipe} must call the SDK program in one line"
done
Expand Down Expand Up @@ -82,6 +82,31 @@ fct_test_uninstall() {
(cd "${repo}" && just uninstall) >"${scratch}/uninstall.out"
fct_has "${scratch}/uninstall.out" "uninstall: nothing to remove, since ${extensions} holds no ncly-${slug}"
fct_test_install_order "${repo}" "${slug}" "${extensions}" "${scratch}"
fct_test_signoff "${repo}" "${scratch}"
}

# Run the rendered recipe with real Git and just. Only gh is a stub.
fct_test_signoff() {
local repo="${1}" scratch="${2}" sha
sha="$(git -C "${repo}" rev-parse HEAD)"
mkdir -p "${scratch}/signoff-bin"
cat >"${scratch}/signoff-bin/gh" <<'SCRIPT'
#!/usr/bin/env bash
set -Eeuo pipefail
if [[ "$*" == --version ]]; then echo 'gh version test'; exit 0; fi
[[ "$#" == 3 && "$1" == signoff && "$2" == --commit ]]
[[ "$(git rev-parse HEAD)" == "$3" ]]
if git symbolic-ref --quiet HEAD; then exit 1; fi
printf '%s\n' "$3" >"${NCLY_TEMPLATE_SIGNOFF}"
SCRIPT
chmod 755 "${scratch}/signoff-bin/gh"
if ! (cd "${repo}" && PATH="${scratch}/signoff-bin:${PATH}" XDG_CACHE_HOME="${scratch}/signoff-cache" \
NCLY_TEMPLATE_SIGNOFF="${scratch}/signed-sha" just signoff) >"${scratch}/signoff.out" 2>&1; then
cat "${scratch}/signoff.out" >&2
fct_fail 'rendered just signoff failed'
fi
fct_has "${scratch}/signoff.out" "Signed off on ${sha}"
fct_has "${scratch}/signed-sha" "${sha}"
}

# fct_test_install_order blocks manifest publication in $3, then tries just install in $1.
Expand Down
Loading
Loading