Skip to content

feat(daemon): let /v1/projects wait for a newer snapshot (HACK-2218 R9) - #864

Open
roodboi wants to merge 6 commits into
nextfrom
feat/hack-2218-projects-long-poll
Open

roodboi wants to merge 6 commits into
nextfrom
feat/hack-2218-projects-long-poll

Conversation

@roodboi

@roodboi roodboi commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

taking R9

The desktop app polls hackd's GET /v1/projects to show project state. This lets it wait for a change instead.

sequenceDiagram
  participant App
  participant hackd
  App->>hackd: GET /v1/projects
  hackd-->>App: payload + state_token T1
  App->>hackd: GET /v1/projects?wait=T1
  Note over hackd: same token: wait for refreshes
  hackd->>hackd: refresh (Docker event / 30 s)
  hackd->>hackd: rebuild, token still T1: keep waiting
  hackd->>hackd: refresh, token T2
  hackd-->>App: payload + state_token T2
  App->>hackd: GET /v1/projects?wait=T2
  Note over hackd: no change by wait_timeout_ms: fresh build, same token
Loading
  • Marker. The design asked for wait=<generated_at>, but generated_at is stamped on every response, so it cannot mark a state. Responses now carry state_token, a 32-hex digest of the payload without generated_at, profiling, runtime_checked_at, runtime_last_ok_at and the elapsed time in Docker's container status ("Up 12 seconds (healthy)" is digested as "Up (healthy)"). Those change on every 30 s refresh with no state change; the status's health and exit code, container state and the R2 state/health fields still count.
  • Waiting. ?wait=<token> answers at once when the current token differs. Otherwise it rebuilds after each runtime refresh and answers when the token changes, or at wait_timeout_ms (default 25000, 1000–55000) with a fresh build. A client disconnect stops the wait. At most 16 requests wait; beyond that the answer is immediate. Malformed parameters return 400. Registry-only changes appear at the next refresh or at the timeout.
  • Idle timeout. Bun closes requests idle for 10 s by default, and server.timeout(request, n) does not raise that limit on Bun 1.4.4 (probed: requests were cut at ~12 s whatever value was passed). The daemon's internal server now sets idleTimeout to 70 s. Bun's types declare idleTimeout only for host/port servers, so a small helper adds it. The socket stays owner-only. The unsupported gateway route does not wait.
  • CLI shape. hack projects --json can be served from the daemon payload, so the CLI strips state_token to keep its published shape the same with and without a daemon.

Docs: waiting for changes, next.18 release notes, desktop DESIGN.md R9 row marked as existing.

Validation and gaps are in the ready note.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hack-docs Ready Ready Preview Nov 15, 2026 2:21pm UTC

Request Review

@linear-code

linear-code Bot commented Nov 15, 2026

Copy link
Copy Markdown

HACK-2218

roodboi and others added 2 commits November 15, 2026 09:31
hackd's projects response now carries state_token, a digest that
ignores per-response and refresh-timestamp fields and Docker's
humanized container status. `?wait=<token>&wait_timeout_ms=` answers
when the token changes after a runtime refresh, or at the timeout with
a fresh build. At most 16 requests wait; malformed parameters are 400.
The daemon socket's idle limit rises to 70 s because Bun's 10 s default
closes long-polls and server.timeout(request) cannot raise it. The CLI
strips state_token so `hack projects --json` keeps its published shape.
HACK-2218 R9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The waiter subscribed to refreshes only after building, so a refresh
completing during the build was missed and the request could wait out
its timeout with newer state cached. The cache now counts completed
refreshes; the resolver reads the count before each build and
nextRefresh returns at once when a newer refresh has completed.
Addresses the persistent review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi
roodboi marked this pull request as ready for review November 15, 2026 09:37
@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T14:23:01.912795Z 3323a32 Manual request
🔒 Security Review ✅ Completed 2026-11-15T14:25:10.666743Z 3323a32 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 7c1359fc9a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 7c1359fc9a

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

The fixed 32-hex test token in tests/projects-wait.test.ts matches
gitleaks' generic-api-key rule. It is a synthetic state digest, not a
credential; allow that exact line in that file, like the existing relay
test token. History is scanned in full, so the line itself stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: a50b4d739f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: a50b4d739f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Ready for lead: a50b4d7

HACK-2218 R9, based on next 34b5053.

hackd's GET /v1/projects carries state_token, and ?wait=<state_token>&wait_timeout_ms= waits for a change. The request answers when the token changes after a runtime refresh, or at the timeout (default 25 s, 1–55 s) with a fresh build. At most 16 requests wait; malformed parameters return 400; a client disconnect ends the wait. Registry-only changes appear at the next refresh (30 s) or at the timeout. The CLI strips state_token, so hack projects --json keeps its published shape. The unsupported gateway route does not wait.

Decisions taken (flagged):

  • Marker: the design said wait=<generated_at>, but generated_at is stamped per response. The token is a digest that ignores generated_at, profiling, runtime_checked_at, runtime_last_ok_at and Docker's humanized container status. Container state and the R2 state/health fields still count.
  • Idle limit: the daemon socket's idle limit rises from Bun's 10 s default to 70 s for every request on it. server.timeout(request, n) does not raise the server limit on Bun 1.4.4 (probed: requests were cut at ~12 s whatever value was passed). Bun's types declare idleTimeout only for host/port servers, so a small helper adds it. The socket stays owner-only.
  • Secret scan: the synthetic 32-hex test token matched gitleaks' generic-api-key rule, and CI scans full history, so .gitleaks.toml gets a path-plus-exact-line allowlist entry, the same shape as the existing relay-test entry for the same token.

Fixed during the loop (persistent reviewer P2): a refresh completing while the first payload was built could be missed, leaving the request waiting out its timeout with newer state cached. The cache now counts completed refreshes; the resolver reads the count before each build, and nextRefresh returns at once if a newer refresh has completed.

Ran locally (Linux, Bun 1.4.4, temp HOME, HACK_DAEMON_AUTOSTART_DISABLE=1, no Docker):

  • tests/projects-wait.test.ts 7, including the build-time refresh race.
  • tests/runtime-cache.test.ts 13, including nextRefresh generation, deadline and abort.
  • tests/daemon-projects-wait.test.ts 1. It starts a real foreground hackd in a private /tmp home; a 12 s wait returns 200 with the same token, a bad token gets 400, and projects --json served by that daemon has no state_token.
  • Each new test fails with its fix removed.
  • tests/machine-output-contract.test.ts 87, tests/sync-spawn-bound.test.ts 2, projects-observe 8, project-views 24, daemon-client 1, daemon-detach 1.
  • Root typecheck, check (including privacy), docs:prepare + docs:check.
  • bun run build, build:config-compiler, check:native-plan-cli: all 8 runner scripts pass.
  • Manual end-to-end through the public socket: a 40 s wait survived the 30 s periodic refresh with an unchanged token.
  • Local gitleaks full-history scan with CI's options: 0 findings.

Exact-head hosted CI (run 43700628817): every Linux job that ran passed (test shards 4–8 and fan-in, test focus, secret scan, Config compiler ubuntu, Desktop app Linux, docker-e2e fan-in, docs site). Candidate core, Docker e2e shards, linux-process-lifetime and runtime models/images were skipped by change scope. Desktop app (macOS): queued, not waited for. Optional Proof skipped. GitHub reports mergeable. computer pr review-status: CLEAR at a50b4d7.

Review: persistent Codex reviewer CLEAN at this head (round 3). Codex bot: no major issues and no security issues at a50b4d7.

Not verified: macOS (the 70 s idle limit on a Mac daemon, and the daemon test there), a real state change end to end (covered by unit tests with fake refreshes), and desktop adoption. No further pushes planned after this note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5c2ed8c2e2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/daemon/projects-wait.ts Resolved
@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 5c2ed8c2e2

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

The token dropped Docker's whole container status to ignore elapsed time,
so a branch container going from healthy to unhealthy (health is carried
only in that text for branches) could not wake a long-poll. Keep the
status's leading word and parenthesized parts ("Up (healthy)",
"Exited (0)") and drop only the elapsed time. Addresses the Codex review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 3323a32470

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 3323a32470

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Ready for lead: 3323a32

Replaces the note at a50b4d7. Since then:

Decisions from the previous note are unchanged: state_token instead of generated_at; the daemon socket's 70 s idle limit; the narrow gitleaks allowlist for the synthetic test token.

Ran locally on this head (Linux, Bun 1.4.4, temp HOME, HACK_DAEMON_AUTOSTART_DISABLE=1, no Docker): projects-wait 8; runtime-cache 13; daemon-projects-wait 1 (real foreground hackd, 12 s wait); tests/machine-output-contract.test.ts 87; tests/sync-spawn-bound.test.ts 2. On the merge, also projects-observe 8 and project-views 24. Root typecheck, check (including privacy), docs:prepare + docs:check. bun run build, build:config-compiler, check:native-plan-cli: all 8 runner scripts pass. Local gitleaks full-history scan with CI's options: 0 findings.

Exact-head hosted CI (run 43723848698): all 16 active checks passed, including every Linux job (test shards 4–8 and fan-in, test focus, secret scan, Config compiler ubuntu, Desktop app Linux, docker-e2e fan-in, docs site). Candidate core, Docker e2e shards, linux-process-lifetime and runtime models/images were skipped by change scope. Desktop app (macOS): queued, not waited for. Optional Proof skipped. GitHub reports mergeable. computer pr review-status: CLEAR at 3323a32.

Review: persistent Codex reviewer CLEAN at this head (round 5). Codex bot: no major issues and no security issues at 3323a32.

Not verified: macOS (the 70 s idle limit and the daemon test on a Mac), a real container state change end to end, and desktop adoption. No further pushes planned after this note.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant