Skip to content

fix: stop on SIGTERM when started with SIGINT ignored, and event-driven interrupt tests - #1531

Merged
kvlonge merged 3 commits into
mainfrom
fix/event-driven-interrupt-tests
Nov 17, 2026
Merged

kvlonge merged 3 commits into
mainfrom
fix/event-driven-interrupt-tests

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Why

Interrupt and SIGTERM tests sent their signal on a timer and asserted elapsed time, so they failed under load and in runners that start commands with SIGINT ignored. That exposed a product gap. When sqb is started with SIGINT ignored, SIGTERM cancelled statements and exited 143, but the blocking step, such as a task or a slow sink import, ran to its end. Background jobs and CI runners often start processes that way; orchestrators SIGKILL after a 10–30 s grace.

Changes

  • Tests: the custom-rule host and Recall SIGTERM and Ctrl-C tests signal once the blocking step has started, or once the main thread is blocked in Recall's close wait. They assert outcomes and counts, not time; a bound fails only a hang.
  • New test: interrupts the single-host fallback and fails if the pending interrupt is lost.
  • Fix: when Python doesn't handle SIGINT, the command handles an unused signal (SIGRTMIN, or SIGUSR2) while it is at its default action, and the SIGTERM watcher interrupts the main thread with it. Exit 143 as before. Ignored SIGINT stays ignored. Interactive Ctrl-C and Windows are unchanged. The handler sits in sqlbuild-python-base beside the Ctrl-C handler.
  • Crate cap: sqlbuild-python grows by 8 code lines (wiring into the termination handler). Its cap moves from 66,756 to exactly 66,764.
  • E2E: native sqb build with SIGINT ignored, SIGTERM once its task runs. On main the task still finished its 60 s sleep.
  • Docs: Recall page, SIGTERM paragraph.

No DuckShop case: this is CLI signal handling, not a project shape, value or lifecycle.

Verification

  • Host and Recall tests started with SIGINT ignored and default: 148 passed each way.
  • New E2E fails on main's extension, passes with the fix.
  • SIGINT ignored: SIGINT leaves a build running; SIGTERM ends it, 143.
  • Under load (16 at once, 8 cores): closing test 0 hangs in 44; Ctrl-C test -2 in 56/56.
  • make check-ci passes. make test: 12,310 passed; 4 need Docker, absent here.

… time

The busy-split-hosts interrupt tests and four Recall SIGTERM and Ctrl-C
tests sent their signal on a timer and asserted elapsed time, so they
failed in runners that start commands with SIGINT ignored or under load.

They now send the signal once the blocking step has started (rule
invocations started, the task printed, the store shows the step running)
and assert outcomes and counts: the interrupt was raised, hosts launched,
fewer invocations than models, no child processes or host inputs left,
the exit code, no statement without an end, and that the step's
post-sleep output is absent. A generous bound remains only to fail a
hang. Processes the tests interrupt get default SIGINT handling.

A new unit test interrupts the single-host fallback and fails if the
pending interrupt is ever lost.
After SIGTERM, the command cancels its running statements and interrupts
its main thread by sending it SIGINT, but only when Python handles
SIGINT. A process started with SIGINT ignored, as background jobs of
scripts and many process managers and CI runners are, therefore kept
running its blocking step, such as a task or a slow sink import, to the
end. Orchestrators then SIGKILL it after their grace period, before it
cleans up.

When SIGINT is not handled, the command now installs a handler on an
otherwise unused signal (SIGRTMIN, or SIGUSR2 where there is none) while
that signal is at its default action, and the SIGTERM watcher interrupts
the main thread with it. The handler raises the same termination error,
so the command exits 143 as before, and is restored when the command
ends. SIGINT itself is unchanged: when it was ignored, Ctrl-C stays
ignored. Windows is unchanged.

An end-to-end test starts the native sqb build with SIGINT ignored and
sends SIGTERM once its task is running.
@kvlonge
kvlonge enabled auto-merge November 17, 2026 02:50
@kvlonge
kvlonge added this pull request to the merge queue Nov 17, 2026
@kvlonge
kvlonge removed this pull request from the merge queue due to a manual request Nov 17, 2026
@kvlonge
kvlonge added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit f113542 Nov 17, 2026
35 checks passed
@kvlonge
kvlonge deleted the fix/event-driven-interrupt-tests branch November 17, 2026 03:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant