Repository navigation
fix: name the node each run event is about, and steady the run page on long runs - #1571
Merged
Merged
Conversation
On a real 93-seed run the run page fell ever further behind and looked frozen. A running run sends a change about once a second, and each one made every page reload whole listings: - the sidebar's counts, including the 500 KiB checks listing, with no coalescing at all; - on the run page, all of the run's events, statements, output and cost. At 6× CPU slowdown, as PERF.md assumes, the run page was 81% busy and /settings 78%, so a slower machine never caught up. - The sidebar's counts reload at most every 10 s while changes stream in, and once more after the last change (`throttled`). - `createResource` takes `every`: the least time between reloads that signals start. - A change signal drops shared in-flight listings, so a reload never gets an answer from a request started before the change. Before, that could leave a run's last state unshown. Measured on the real run with a change each second at 6× CPU: /settings 78% → 5% busy, the run page 81% → 15%. PERF.md gains the budget and the rule.
…ut as a terminal would
A run's log said `run` in the Node column for every node event. The UI read
node identity from `resource_id` as a `["<invocation>", "<node key>"]` pair,
but the server sends an opaque hash there, with the node in `resource_kind`
and `resource_name`. So no event or statement named its node, and parsing
the hash threw for every event. The node key is now `<kind>:<name>`, as
the run keys its nodes, with the old pair still read. The scheduler's own
run_node_started and run_node_finished events carry no node identity, and
that needs a server change.
Printed output arrives as raw chunks full of spinner redraws and escape
codes. It now reads as the terminal left it, lines joined across chunks,
so the run's printed plan ("Plan ready 93 selected", the seed list) is
legible.
Node keys show without their kind prefix (`seed:` is said elsewhere),
with the full key on hover.
…lls smoothly
- While a run goes on, nodes that haven't started are "Waiting", in their
own group and on the timeline. "Not run" is for finished runs only.
- A run of more than 40 nodes lists them once, on the timeline. The panel
beside it becomes a summary: each outcome's count, which narrows the
timeline to that outcome ("Waiting only · 60 of 93 · Show all"). Only
failed and skipped nodes are listed there, each with why. Small runs keep
the full lists.
- The log, statements, output and cost reload at most every 3 s while a run
streams changes; node states still reload after each burst.
- "Plan it now" beside Details opens the job's build in the plan page. The
run's own plan is in its printed output; the server keeps no plan
document for a run.
- Timeline scrolling: rows out of view are neither laid out nor painted
(`content-visibility`). Bars are placed by their times and one --pps, so
zooming changes one value instead of every bar. Running bars' stripes
move by transform, without repainting. Measured on a real 93-node run at
6× CPU, scrolling went from p90 83 ms / max 133 ms to p90 33 ms / max
67 ms. Scrolling sideways while zoomed in went from p90 50 ms / max
267 ms to p90 17 ms / max 133 ms.
…ng job's progress, and show selection - A name too long for its pill loses its middle, keeping a third from its start and the rest from its end. Siblings differ at either end: `stg_orders` / `dim_orders`, or the `__` part after a shared prefix. Cutting the end left `order_type_…` and `region_usab…` for several siblings alike. The tail rule (the shortest `__` ending unique among siblings in the same box) is unchanged; it was the end cut that hid the distinguishing part. Tested with names shaped like a real project's. - While a job's latest run is active, its graph's cards and pills show that run's node states, running, succeeded or failed, ahead of the last build and stale marks. - A selected pill has a primary-coloured edge, a ring around it and a tinted fill, folders included.
A run's event list showed no node for the scheduler's run_node_started,
run_node_finished, run_check_started and run_check_finished events, nor
for audit_completed. Recall fills resource_kind and resource_name only
from top-level payload fields. The scheduler stored a report as
{"change": ...}, with the node key nested inside it, and an audit names
its node as attached_target_kind and attached_target_name. On a real
93-seed run, all 186 node events and 123 audit results had no node.
- Node and check reports now also store their `<kind>:<name>` node key as
top-level resource_kind and resource_name, as member retry events
already do.
- A retried report is compared by its `change` only. The fields derived
from it may differ between server versions, so a report stored by an
older server still matches its retry instead of being refused.
- The run events read fills the node of older scheduler events and of
audit results from their payloads. Only those identity fields are
read; the payload itself is still left out.
… CRLF lines A build records a model under its materialization (table, view, custom or snapshot), but the run keys every model as `model:<name>`, so selecting a model hid its own failure row and statements. Map those kinds to `model`. A line ending in a carriage return before its newline showed as blank; drop trailing carriage returns before keeping what follows the last one.
A run's events read took the whole payload of every node, check and audit event without resource columns, only to find which node it was about. The read now selects the node key and audit target fields from the payload in SQL, so large audit payloads are no longer fetched.
Timeline rows paint only within their own box, so a duration that ran past a narrow chart's last 10% was cut off. A duration now reads on its bar's left whenever its measured width would run past the row and there is more room on the left.
Brings in the ignored audit failures listing and the paged seed plan sections, keeping both sides of the shared run models, run recall and UI run client files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A run's event log did not say which node an event was about, so node, check and audit lines were hard to follow on a busy run. The run page also had rough edges on long runs: it repeated nodes, showed blank lines for Windows line endings, clipped durations on narrow timelines and redrew more often than it needed to.
Changes
model:<name>), so they match the run's node.Verification
scripts.run_native_tests -p sqlbuild-orchestration --lib: 187 tests on SQLite and PostgreSQL, including a new test that stores node, check and audit events, rewrites one to the older payload-only shape, and reads all of them back named. 186 passed in the full run. The one failure was the catalog listing timing guard, which passed when re-run alone; this branch does not touch that path.vitest(581 passed),svelte-check(0 errors), Playwrightcompact,run-page,layoutandselectionspecs (73 passed). A new run-page check fails on the previous timeline at 760px.make check-cipassed on the final head.