Skip to content

fix: name the node each run event is about, and steady the run page on long runs - #1571

Merged
kvlonge merged 9 commits into
mainfrom
fix/run-events-name-their-node
Nov 18, 2026
Merged

kvlonge merged 9 commits into
mainfrom
fix/run-events-name-their-node

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Why

A run's event log did not say which node an event was about, so node, check and audit lines were hard to follow on a busy run. The run page also had rough edges on long runs: it repeated nodes, showed blank lines for Windows line endings, clipped durations on narrow timelines and redrew more often than it needed to.

Changes

  • The server names the node of every node, check and audit event in a run's events. Events stored before node events carried resource columns are named from their payload. The read selects only those identity fields in SQL instead of whole payloads.
  • The run page lists each node once, says which nodes are waiting, and keeps an open running run up to date with throttled refreshes.
  • Printed output shows the node it came from and renders as a terminal would, including CRLF line endings.
  • A model's build rows key as the run keys the model (model:<name>), so they match the run's node.
  • On the timeline, zoomed-out pills keep both ends of a long name and show a running job's progress and the selection. A duration that would run past its row reads on its bar's left.

Verification

  • scripts.run_native_tests -p sqlbuild-orchestration --lib: 187 tests on SQLite and PostgreSQL, including a new test that stores node, check and audit events, rewrites one to the older payload-only shape, and reads all of them back named. 186 passed in the full run. The one failure was the catalog listing timing guard, which passed when re-run alone; this branch does not touch that path.
  • UI: vitest (581 passed), svelte-check (0 errors), Playwright compact, run-page, layout and selection specs (73 passed). A new run-page check fails on the previous timeline at 760px.
  • make check-ci passed on the final head.

On a real 93-seed run the run page fell ever further behind and looked
frozen. A running run sends a change about once a second, and each one made
every page reload whole listings:
- the sidebar's counts, including the 500 KiB checks listing, with no
  coalescing at all;
- on the run page, all of the run's events, statements, output and cost.
At 6× CPU slowdown, as PERF.md assumes, the run page was 81% busy and
/settings 78%, so a slower machine never caught up.

- The sidebar's counts reload at most every 10 s while changes stream in,
  and once more after the last change (`throttled`).
- `createResource` takes `every`: the least time between reloads that
  signals start.
- A change signal drops shared in-flight listings, so a reload never gets
  an answer from a request started before the change. Before, that could
  leave a run's last state unshown.

Measured on the real run with a change each second at 6× CPU: /settings
78% → 5% busy, the run page 81% → 15%. PERF.md gains the budget and the
rule.
…ut as a terminal would

A run's log said `run` in the Node column for every node event. The UI read
node identity from `resource_id` as a `["<invocation>", "<node key>"]` pair,
but the server sends an opaque hash there, with the node in `resource_kind`
and `resource_name`. So no event or statement named its node, and parsing
the hash threw for every event. The node key is now `<kind>:<name>`, as
the run keys its nodes, with the old pair still read. The scheduler's own
run_node_started and run_node_finished events carry no node identity, and
that needs a server change.

Printed output arrives as raw chunks full of spinner redraws and escape
codes. It now reads as the terminal left it, lines joined across chunks,
so the run's printed plan ("Plan ready  93 selected", the seed list) is
legible.

Node keys show without their kind prefix (`seed:` is said elsewhere),
with the full key on hover.
…lls smoothly

- While a run goes on, nodes that haven't started are "Waiting", in their
  own group and on the timeline. "Not run" is for finished runs only.
- A run of more than 40 nodes lists them once, on the timeline. The panel
  beside it becomes a summary: each outcome's count, which narrows the
  timeline to that outcome ("Waiting only · 60 of 93 · Show all"). Only
  failed and skipped nodes are listed there, each with why. Small runs keep
  the full lists.
- The log, statements, output and cost reload at most every 3 s while a run
  streams changes; node states still reload after each burst.
- "Plan it now" beside Details opens the job's build in the plan page. The
  run's own plan is in its printed output; the server keeps no plan
  document for a run.
- Timeline scrolling: rows out of view are neither laid out nor painted
  (`content-visibility`). Bars are placed by their times and one --pps, so
  zooming changes one value instead of every bar. Running bars' stripes
  move by transform, without repainting. Measured on a real 93-node run at
  6× CPU, scrolling went from p90 83 ms / max 133 ms to p90 33 ms / max
  67 ms. Scrolling sideways while zoomed in went from p90 50 ms / max
  267 ms to p90 17 ms / max 133 ms.
…ng job's progress, and show selection

- A name too long for its pill loses its middle, keeping a third from its
  start and the rest from its end. Siblings differ at either end:
  `stg_orders` / `dim_orders`, or the `__` part after a shared prefix.
  Cutting the end left `order_type_…` and `region_usab…` for several siblings
  alike. The tail rule (the shortest `__` ending unique among siblings in
  the same box) is unchanged; it was the end cut that hid the distinguishing
  part. Tested with names shaped like a real project's.
- While a job's latest run is active, its graph's cards and pills show that
  run's node states, running, succeeded or failed, ahead of the last build
  and stale marks.
- A selected pill has a primary-coloured edge, a ring around it and a
  tinted fill, folders included.
A run's event list showed no node for the scheduler's run_node_started,
run_node_finished, run_check_started and run_check_finished events, nor
for audit_completed. Recall fills resource_kind and resource_name only
from top-level payload fields. The scheduler stored a report as
{"change": ...}, with the node key nested inside it, and an audit names
its node as attached_target_kind and attached_target_name. On a real
93-seed run, all 186 node events and 123 audit results had no node.

- Node and check reports now also store their `<kind>:<name>` node key as
  top-level resource_kind and resource_name, as member retry events
  already do.
- A retried report is compared by its `change` only. The fields derived
  from it may differ between server versions, so a report stored by an
  older server still matches its retry instead of being refused.
- The run events read fills the node of older scheduler events and of
  audit results from their payloads. Only those identity fields are
  read; the payload itself is still left out.
… CRLF lines

A build records a model under its materialization (table, view, custom
or snapshot), but the run keys every model as `model:<name>`, so
selecting a model hid its own failure row and statements. Map those
kinds to `model`. A line ending in a carriage return before its newline
showed as blank; drop trailing carriage returns before keeping what
follows the last one.
A run's events read took the whole payload of every node, check and
audit event without resource columns, only to find which node it was
about. The read now selects the node key and audit target fields from
the payload in SQL, so large audit payloads are no longer fetched.
Timeline rows paint only within their own box, so a duration that ran
past a narrow chart's last 10% was cut off. A duration now reads on its
bar's left whenever its measured width would run past the row and there
is more room on the left.
Brings in the ignored audit failures listing and the paged seed plan
sections, keeping both sides of the shared run models, run recall and
UI run client files.
@kvlonge
kvlonge marked this pull request as ready for review November 18, 2026 11:10
@kvlonge
kvlonge added this pull request to the merge queue Nov 18, 2026
Merged via the queue into main with commit 1cd8411 Nov 18, 2026
36 of 62 checks passed
@kvlonge
kvlonge deleted the fix/run-events-name-their-node branch November 18, 2026 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant