Skip to content

feat: report job group cycles at compile and deliver invocation starts first - #1539

Merged
kvlonge merged 10 commits into
mainfrom
fix/job-group-cycles
Nov 17, 2026
Merged

kvlonge merged 10 commits into
mainfrom
fix/job-group-cycles

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Why

A job group whose members wait on each other could compile cleanly and only fail when a location published, and publication named one cycle at a time. Separately, lifecycle sinks could receive an invocation's terminal events before its invocation_started, because the export queue delivers by priority.

Changes

  • sqb compile reports every job group cycle as one D017 error at the group's name line, one line per cycle, with a hint when a wait reads a seed or a node several members build. Compile and publication derive member waits with one shared function from the same node lineage, so they agree; jobs whose selectors fail are left to their own error.
  • Publication rejects groups with all their cycles listed, not just the first.
  • The wait traversal visits each node once per member, so large groups stay fast.
  • Python node deps and intermediate loader sources come from one ProjectSelection owner used by both compile and publication.
  • The lifecycle export queue delivers a queued invocation_started before any other queued event of the same invocation. Each CLI invocation has one dispatcher, so this orders every event a command emits; other invocations keep their priority order.
  • Docs: job groups and the errors table in the orchestration concept page, and the generated skill reference.

Verification

  • New E2E: compile fails with all three cycles and passes once the group is split.
  • Integration: publish rejects the DuckShop group with every cycle; orchestration snapshot, job selector, compile selection and publication suites pass (132 tests).
  • Rust: wait graph cases, a 50-member, 10,000-node group within a 2 s bound (0.4 s locally), and event queue ordering tests.
  • make check-ci, fensu (0 faults) and clippy pass.
  • Not run: a real-project compile timing comparison.

DuckShop: adds edge_cases/orchestration/group_cycles, a job group with three cycles, one through a seed.

The lifecycle queue delivers by priority, so a terminal fact such as
operation_completed could reach a sink before invocation_started. When the
event the queue would pick next has a queued invocation_started of the same
invocation, the queue now picks that start first. Events of other
invocations, and events whose start is not queued, keep their priorities.
…every cycle

group_wait_graph takes member selections and node dependencies, so publish
and compile can share it. It finds every set of members that wait on each
other, names one cycle per set with a reason per wait, and flags cycles
whose waits read a seed or a node several members build. Publication keeps
its message, one line per cycle, plus a hint for shared reads.
…blish and compile

WIP: the publish test passes. The compile E2E fails until compile checks
job group cycles, which is the next commit.
Compile derives each job group's member waits from the same lineage publication
uses and reports every cycle as one D017 error at the group's name line. The
Python node deps publication records come from one shared selection method, and
the wait traversal pushes each node once so large groups stay linear.
Compile raises the cycle error through discovery's existing failure path, the
dispatcher reads an event's invocation through the native record reader, and
the wait traversal, cycle naming and invocation-start lookup are named helpers.
The large-group test states its expected waits as data.
@kvlonge
kvlonge disabled auto-merge November 17, 2026 06:44
@kvlonge
kvlonge added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 1dce3d9 Nov 17, 2026
34 checks passed
@kvlonge
kvlonge deleted the fix/job-group-cycles branch November 17, 2026 08:09
kvlonge added a commit that referenced this pull request Nov 17, 2026
Moves event exporting from the bindings crate into
sqlbuild-python-output, as the slice did, keeping the large-project
tree's behaviour. The slice's base already carried main's
invocation-start delivery (#1539), which the tree does not have; that
change is left out here and arrives with the main merge.

Validation: cargo check --workspace --all-targets passes with no
warnings, the layout matches the slice, and every item of the tree's
event exporting is still present. Nothing is built or tested yet. fensu
reports 349 faults, unchanged from the previous slice.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant