Skip to content

feat(lifecycle): stream --progress ndjson from up, down and restart - #853

Merged
roodboi merged 4 commits into
nextfrom
claude/hack-2218-lifecycle-progress
Nov 15, 2026
Merged

roodboi merged 4 commits into
nextfrom
claude/hack-2218-lifecycle-progress

Conversation

@roodboi

@roodboi roodboi commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

taking R6 (HACK-2218). This PR is the Compose slice; the native part is listed under "Not in this PR".

The desktop app's actions need to show which service is at which step while hack up, down and restart run. With --json today they print only the final envelope.

With hack up|down|restart --json --progress, stdout becomes ndjson:

stateDiagram-v2
  [*] --> start: schema hack.lifecycle-progress.v1
  start --> progress
  progress --> progress
  start --> envelope
  progress --> envelope: one compact line
  envelope --> [*]
Loading
  • First line: {"schema":"hack.lifecycle-progress.v1","type":"start","ts","action","project"}. It is written once the project is resolved.
  • Then {"type":"progress","ts","service","phase"} lines.
  • The last line is the same {ok,data} / {ok:false,error} envelope that --json prints today, on one line. Exit codes are unchanged.
  • Refusals before the stream starts (usage errors, project not found) stay the usual single envelope.
Phase service Source
images, starting null the existing Compose startup phase callback
stopping, stopped null around the Compose down of down and restart
waiting, installing, verifying, ready, failed name dependency-cache progress, which already went to stderr
running, completed, stopped, failed name the services the closing envelope reports, written just before it

Only signals the code already observed are used: no TTY parsing and no new polling. Some phases can be skipped (see docs/reference/lifecycle-progress.md).

Design (please check):

  • There is one process-wide stream. It is armed by the handler, starts when the Compose project is resolved, and closes in emitLifecycleResult (every lifecycle envelope goes through it).
  • An exception that escapes after start is closed by the top-level handler with one compact envelope line, never the pretty one.
  • renderCliResult gains an optional compact flag. The default output of other commands is unchanged.
  • --progress requires --json.

Not in this PR (R6 follow-up, native):

  • native up (its detached IPC carries no per-service events today)
  • native restart --json, which is refused today
  • native down --json, which prints bare JSON

Native backend and native-authored projects refuse --progress before doing any work.

Validation (Linux, pinned Bun 1.4.4):

  • 6 unit tests: line shapes, schema only on the first line, compact closing envelope, and closing after an exception thrown mid-stream.
  • 6 CLI tests with a stub docker: up success and failure, down, restart, --progress without --json, and the native refusal.
  • The new CLI tests and the mid-stream-exception test fail without the change.
  • Existing lifecycle tests pass. 15 native dispatch/refusal tests fail identically on plain next on this host.
  • bun run typecheck, bun run check (warnings only), docs:prepare and docs:check pass.
  • Not covered by a test: per-service dependency-cache phases in a full CLI run (one-line forward next to the existing stderr write), and a live Docker run.

Release intent: feat.

🤖 Generated with Claude Code


Show it working
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

roodboi and others added 2 commits November 14, 2026 18:05
`hack up|down|restart --json --progress` now writes an ndjson stream on
the Compose backend: a start line carrying the
hack.lifecycle-progress.v1 schema, progress lines
{type, ts, service, phase}, and the ordinary --json envelope as one
compact closing line. Phases come only from signals the Compose path
already observes: the detached startup phase callback (images,
starting), down/restart stop (stopping, stopped), dependency cache
markers, and the envelope's final per-service states.

--progress requires --json. The native backend and native-authored
projects refuse it before any work. Refusals before the stream starts
keep the single pretty envelope.

Documented in docs/reference/lifecycle-progress.md (HACK-2218 R6).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An exception that escapes after the start line is rendered as the
stream's compact closing envelope instead of the global pretty one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hack-docs Ready Ready Preview Nov 14, 2026 6:20pm UTC

Request Review

@linear-code

linear-code Bot commented Nov 14, 2026

Copy link
Copy Markdown

HACK-2218

@roodboi

roodboi commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-14T18:24:33.982905Z
We couldn’t complete this review. Comment @codex review to try again.
dfa6af8 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

roodboi and others added 2 commits November 14, 2026 18:13
SIGINT/SIGTERM after start now ends the stream with one compact
E_CANCELLED envelope while lifecycle cleanup and signal exit stay as
before; a closed stream swallows later envelopes. Remote (dispatch)
targets refuse --progress before dispatch, since dispatch output would
precede the start line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Install the signal handlers when the stream is armed rather than at
start, and write stream lines synchronously, so a signal right after
start cannot kill the process before the closing envelope is written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi
roodboi marked this pull request as ready for review November 14, 2026 18:24
@roodboi

roodboi commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@roodboi

roodboi commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

Ready for lead: dfa6af8

Review: one persistent Codex reviewer (gpt-6.1-sol, codex-dio) ran 2 rounds and returned CLEAN at this head. Round 1 had two blockers:

  • Signal termination left a started stream with no closing line. Now SIGINT or SIGTERM after start writes one compact E_CANCELLED envelope. Lifecycle cleanup still runs, and the process still exits by the signal. A closed stream swallows any later envelope.
    • Stream lines are written synchronously, and the signal handlers are installed when the stream is armed. Both choices fix a real flake in the new subprocess test: 2 failures in 12 runs before, 0 in 15 after.
  • Remote dispatch printed human output before start. Remote targets now refuse --progress before dispatch; remote targets are unsupported anyway.

I also fixed one gap before review: an exception thrown after start now closes the stream with one compact envelope.

The implementation was drafted by a subagent; I reviewed and fixed it.

Bots:

  • computer pr review-status: CLEAR at dfa6af8.
  • The Codex bot review never ran: every request got "usage limits reached".

Hosted checks at this head: 29 pass and 2 fail.

  • test shard 1/8 (macos-15-arm64) fails in tests/python-free-download.test.ts, test "bounded cancellation refuses uncertain read-error settlement…". It hit the 5 s timeout and the mock was called 2 times instead of 1; a second cancellation test in that file then failed on the replaced globalThis.fetch.
  • test is the summary job and fails only because of that shard.

This is the same failure as on #795's head. This PR does not touch that file or the download code. It is not one of the named flakes (HACK-2235, HACK-2243), so I did not rerun it. Please rerun the failed job, or tell me to.

Ran locally (Linux, pinned Bun):

  • 13 lifecycle-progress unit and CLI tests.
  • 113 tests across the lifecycle-json, startup-state and runtime-backend files.
  • bun run typecheck, bun run check (warnings only) and the docs checks.

Not verified:

  • A live Docker run.
  • A CLI test of the per-service dependency-cache phases; that code is a one-line forward next to the existing stderr write.
  • A CLI test of the remote refusal; setting up a remote target needs control-plane config.
  • Native backends refuse --progress; that is the follow-up R6b.

Decisions to check:

  • Schema id hack.lifecycle-progress.v1.
  • The closing line is the existing envelope, compacted to one line.
  • Remote targets refuse --progress.
  • renderCliResult gains an optional compact flag.

@roodboi
roodboi merged commit b731b50 into next Nov 15, 2026
65 of 74 checks passed
@roodboi
roodboi deleted the claude/hack-2218-lifecycle-progress branch November 15, 2026 02:16
roodboi added a commit that referenced this pull request Nov 15, 2026
…p turn (HACK-2250) (#858)

## What

`tests/python-free-download.test.ts`, "bounded cancellation refuses
uncertain read-error settlement and preserves the primary failure", hit
Bun's 5 s default timeout on hosted macOS shard 1/8 (#853). The next
scenario then failed `toHaveBeenCalledTimes(1)` with 2, and the
global-state guard reported `globalThis.fetch replaced`.

## What it was really waiting on

The test drove a fake clock after a single `setTimeout(0)`:

```ts
await drain();      // one event-loop turn
clock.advance(5);   // expire the 5 ms cleanup grace
```

`transferAttempt` (`src/lib/release-transfer.ts`) awaits a real `open()`
of the target file before reading. Only after the size failure does
`settleCancellation` arm the 5 ms grace on the fake clock. If `open()`
takes more than one turn (a loaded runner), the advance fires nothing.
The grace is armed afterwards at the new time and never expires, so the
download never settles. The test then hangs until the 5 s timeout, and
its `finally { mock.mockRestore() }` never runs. The next scenario's
`spyOn(globalThis, "fetch")` reuses the leaked spy, which already has
one call: hence 2, and the guard error.

```mermaid
sequenceDiagram
  participant T as test
  participant D as download
  participant C as fake clock
  T->>D: start
  T->>T: one setTimeout(0)
  T->>C: advance(5): nothing armed yet
  D->>D: open() completes (real I/O, late)
  D->>C: arm 5 ms cleanup grace (never advanced)
  Note over T: hangs to the 5 s timeout, finally skipped, fetch spy leaks
```

The same assumption (one turn is enough) sat in three sibling tests in
this file: "HTTP transfer budget … progress / total" and "five-file …
cleanup". They fail the same way when `open()` is slow.

## Fix (test-only)

- `until(ready, what)` drains in a loop with a **2 s real-time bound**,
well under the 5 s default. A missed step fails with `Fixture step did
not happen within 2000 ms: <what>` instead of a timeout.
`within(promise, what)` bounds the final settlement the same way.
- Before advancing the 5 ms cleanup grace, the test waits for
`clock.isArmed(5)`. `FixtureClock` now records each timer's delay; the
other timers are 30 and 100 ms.
- Before an inactivity tick, the test waits for an actual pending read.
The fixture streams use `highWaterMark: 0` and flag `pull()`, which runs
only while the download is reading. A file-existence check was not
enough: the file exists before `open()` returns its handle (found in
review).
- The five-file spin loops, capped at 100 turns, use the same real-time
bound.
- `afterEach` calls `mock.restore()`, so a test that fails before its
`finally` cannot leak the `fetch` spy into the next test. The existing
per-test `mockRestore()` calls stay.

No assertion is weakened or removed.

## Validation (Linux, 16 cores)

| Check | Before | After |
| --- | --- | --- |
| `-t "bounded cancellation"` under 16-way parallel load | 5 of 400
failed, with the hosted signature (5 s timeout, then 2 calls and the
guard error) | **400 of 400** pass at 5f16f8b |
| Whole file under 16-way parallel load | 2 of 191 failed at an
intermediate head (progress/total false stall, found in review) | **208
of 208** pass (15/15 each) at 5f16f8b |
| Preload delaying `open()` by 50 ms | 4 failures, incl. read-error and
stalled-read timeouts with the guard error | 15/15 |
| Preload delaying `open()` by 200 ms | — | 15/15 |
| Preload delaying after `open()` creates the file | progress/total fail
| 15/15 |

`tsc -p packages/cli` (it includes `tests/`) and `biome check` pass.

Not verified: macOS. The lead runs the 200 on the Mac.

## Release signal

`test`: test-only.

Refs HACK-2250.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Kit <344842+roodboi@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant