Skip to content

feat(projects): share backend-aware service state across CLI and daemon - #831

Merged
roodboi merged 36 commits into
nextfrom
feat/hack-2218-project-state
Nov 15, 2026
Merged

roodboi merged 36 commits into
nextfrom
feat/hack-2218-project-state

Conversation

@roodboi

@roodboi roodboi commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

HACK-2218 R2: full and summary CLI project rows and hackd /v1/projects share backend-aware state, readiness/completion, health, exit code and start time. Native projects use authenticated observers; missing evidence stays unknown. Completed declared jobs do not degrade running peers; inactive absent profiles are excluded.

Depends on #835, ready head 58605b6. Merge #835 first. Current next and that prerequisite are merged normally. Plain projects JSON keeps its published shape; observe listing uses the prerequisite schema envelope. Only R2's appendix row is edited.

Default --json --observe makes no engine/provider contact and bypasses daemon/registration. --observe --runtime explicitly opts into runtime reads. Changed and terminated cached Docker IDs are re-inspected on event refreshes; unchanged running IDs stay query-free and omit fresh health/start evidence. Native concurrency is four. Inspect fields exclude env values, commands, error text and health logs.

Shared state projection

Validation: 165 focused tests pass in separate isolated processes, including state/profile/health, native observations, runtime cache, full/summary projection, non-contact observe and machine-output contracts. CLI build, typecheck, root check/privacy, documentation checks and regenerated CLI reference pass. Exact-head persistent review and hosted CI pending. The prerequisite passed desktop core on Linux/macOS; R2 changes no desktop code.

Live Docker probe: task-owned containers with and without health checks retain state and mounted-volume ownership; disposable-volume verification succeeds. The inspect template safely handles absent health keys. The cache-prune scenario explicitly disables its health check to guard that path; its full hosted rerun is pending.

Limits: no live hackd, native provider or real Mac state-tool run locally. Compose-format projects follow the observing process backend selection, including hackd environment; Compose observations do not establish native state. Native timestamps remain null, and native rows describe the base instance. Lead owns Mac qualification and merge.

@vercel

vercel Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hack-docs Ready Ready Preview Nov 15, 2026 6:12am UTC

Request Review

@linear-code

linear-code Bot commented Nov 14, 2026

Copy link
Copy Markdown

HACK-2218

@roodboi

roodboi commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T06:20:47.726149Z 5bf815c Manual request
🔒 Security Review ✅ Completed 2026-11-15T06:23:19.781356Z 5bf815c Manual request

Security findings

Advisory findings (1)

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bbdfe7d306

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/lib/project-observation.ts Resolved
@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: bbdfe7d306

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@roodboi

roodboi commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 811bc74619

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/lib/compose-service-readiness.ts Resolved
@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 811bc74619

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@roodboi

roodboi commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 1dd47a614f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 1dd47a614f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@roodboi

roodboi commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: eeb8def781

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: eeb8def781

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

…ontract

# Conflicts:
#	scripts/check-native-acquisition-plan-cli.ts
#	scripts/check-native-endpoint-plan-cli.ts
#	scripts/check-native-env-plan-cli.ts
#	scripts/check-native-host-plan-cli.ts
#	scripts/check-native-process-plan-cli.ts
#	scripts/check-native-routing-plan-cli.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Codex Security Review · Automatically triggered

Here are some automated security review suggestions for this pull request.

Reviewed commit: 6e3509cac0

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread src/daemon/runtime-cache.ts Resolved
The four-read limit applied per request, so parallel /v1/projects calls
to hackd could multiply native status subprocesses. Native observation
now takes a slot from one process-wide limit of four, and concurrent
reads of the same project share one read. Settled results are not
cached. Addresses the Codex security review on this PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 36fba169bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 36fba169bf

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Ready for lead: 36fba16

Took over R2 from the previous owner (account out of capacity).

Merge: this branch already contained an older #835 head, so I made a normal merge of #835's current head 2058a97 (fc15ac17 → 6e3509c). That head contains next e47681e in full. The merge had no conflicts. Recommended default, flagged: #831 stays stacked on #835, so merge #835 first. If you want next merged without the newer #835, say so.

Fix 36fba16 (Codex security review P2 on src/daemon/runtime-cache.ts): the four-read native observation limit applied per request, so parallel /v1/projects calls to hackd could multiply native status subprocesses. Native reads now take a slot from one process-wide limit of four (NATIVE_OBSERVATION_LIMIT in src/lib/project-native-observation.ts). Concurrent reads of the same project with the same reader share one read, and settled results are not cached. New tests: 13 concurrent observations peak at exactly 4; three concurrent reads of one project make one call, and a later read calls again. Both fail with the limit or sharing removed. docs/reference/project-state.md is updated. Not done (said on the thread, which is answered and resolved): per-request cancellation and filtering before observation. An abandoned request can hold at most the four shared slots until each read's existing timeout.

Ran locally (Linux, Bun 1.4.4, temp HOME, HACK_DAEMON_AUTOSTART_DISABLE=1), each file in its own process: project-native-observation 10, project-views 24, runtime-cache 12, projects-observe 8, project-state 6, runtime-projects 8, compose-service-activation 1, compose-service-readiness 2, runtime-service-state 2, machine-output-contract 86, machine-output-success-shapes 10, all passing. Root typecheck and check; docs:prepare + docs:check; CLI reference regenerates unchanged; bun run build, build:config-compiler, check:native-plan-cli: all 8 relocated acceptance scripts pass, both after the merge and after the fix.

Exact-head hosted CI (run 43669730137): every Linux job passed: test shards 3–8, all 4 Docker e2e shards and fan-in, linux-process-lifetime, Candidate core (ubuntu), Config compiler (ubuntu x64 and arm), Desktop app (Linux), runtime state models and images, docs site, secret scan. Already passed on macOS: Desktop app. macOS test shards 1/8 and 2/8, Candidate core (macOS) and Config compiler (macOS): queued, not waited for. Optional Proof skipped. GitHub reports mergeable. computer pr review-status: CLEAR at 36fba16.

Review: new persistent Codex reviewer (gpt-6.1-sol, codex-dio) CLEAN at this head (round 2; round 1 was clean on the merge). Codex bot: no major issues and no security issues at 36fba16.

Not verified: live hackd, native provider and Mac state-tool runs; the Mac batch is the gate. No further pushes planned after this note.

roodboi and others added 4 commits November 15, 2026 06:04
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…migration

`env migrate --json` (#797) is an envelope without `data.schema`;
`usage --json --by-service` (#852) is a schema-tagged variant, so its
schema id moves into MACHINE_SCHEMAS. Flag variants whose base command
keeps another shape now get their own reference row instead of a
hard-coded list.

The outputs this branch migrates first shipped in 5.0.0-next.17, so the
reference page no longer calls them unpublished and the next.18 release
notes describe the change for scripts.

BREAKING CHANGE: version, activity, projects --observe, global status
--observe and global install|trust --plan JSON move into the
{ok,data:{schema}} / {ok:false,error} envelope.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…74aaf) into feat/hack-2218-project-state

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 5bf815c743

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 5bf815c743

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@roodboi

roodboi commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Ready for lead: 5bf815c

Normal merge of #835's head db01be6, which contains next bf74aaf. No conflicts; git show --remerge-diff has no resolution hunks. This branch's own commits are unchanged since 36fba16 (process-wide native observation limit). Replaces the note at 36fba16. Stacked on #835, so #835 merges first.

Ran locally (Linux, Bun 1.4.4, temp HOME, HACK_DAEMON_AUTOSTART_DISABLE=1), each file in its own process, all passing: project-native-observation 10, project-views 24, runtime-cache 12, projects-observe 8, project-state 6, runtime-projects 8, compose-service-activation 1, compose-service-readiness 2, runtime-service-state 2, tests/machine-output-contract.test.ts 87, success-shapes 10, tests/sync-spawn-bound.test.ts 2, usage-by-service-cli 5. Root typecheck and check; docs:prepare + docs:check; CLI reference regenerates unchanged; bun run build, build:config-compiler, check:native-plan-cli: all 8 runner scripts pass.

Exact-head hosted CI (run 43684367835): 21 checks passed so far, with no failures. Outstanding on Linux at the time of this note: docker-e2e shards 2/4 and 4/4 (in progress). I'll comment here if either fails. macOS Candidate core, Config compiler, Desktop app and test shards 1/8, 2/8, 3/8: queued, not waited for. Optional Proof skipped. GitHub reports mergeable. computer pr review-status: CLEAR at 5bf815c.

Review: persistent Codex reviewer CLEAN at this head (round 3). It confirmed no interaction with #797's migration guards, #852's usage changes or #857's spawn lint, and that plain projects --json keeps its shape. Codex bot: no major issues and no security issues at 5bf815c.

Not verified: live hackd, native provider and Mac state-tool runs; the Mac batch is the gate. Known gap from the security fix (unchanged): no per-request cancellation, so an abandoned request can hold at most the four shared slots until each read times out. No further pushes planned after this note.

@roodboi
roodboi merged commit 345dc0d into next Nov 15, 2026
31 checks passed
@roodboi
roodboi deleted the feat/hack-2218-project-state branch November 15, 2026 07:35
roodboi added a commit that referenced this pull request Nov 15, 2026
…econcile

Brings in next at 345dc0d, including the shared JSON envelope rule and command inventory (#835) and shared service state (#831).

Conflict: only the generated plugins/hack/skills/hack-diagnose/references/error-codes.md (content revision line). Regenerated with bun run generate:agent-plugins: next's rows plus E_DAEMON_SERVICE.

Adapted to #835: hack daemon start|stop|restart|install --json are recorded as schema-envelope in the JSON inventory, and their schemas are registered in MACHINE_SCHEMAS. The envelope was already #835's shape (exactly ok and data, schema inside data, E_DAEMON_SERVICE errors with detail), so only the schema id changes from hack.daemon-<command>/v1 to the registry's dotted form hack.daemon-<command>.v1. The post-update parser uses the same constant. Regenerated docs/reference/machine-readable-output.md; the CLI reference had no diff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
roodboi added a commit that referenced this pull request Nov 15, 2026
Resolve conflicts with #831 shared service state: project-views keeps both
import sets (observation fields come from next's serializer); projects.ts
keeps the autostart row field and drops the now-redundant observe spread;
runtime-cache test takes next's file plus this branch's autostart polling
test; getting-started keeps both paragraphs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
roodboi added a commit that referenced this pull request Nov 15, 2026
# Conflicts:
#	apps/desktop/core/src/source.rs
roodboi added a commit that referenced this pull request Nov 15, 2026
… setup

# Conflicts:
#	apps/desktop/core/tests/dump_state.rs
roodboi added a commit that referenced this pull request Nov 15, 2026
# Conflicts:
#	apps/desktop/core/src/runner.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant