Skip to content

Add a compose-only node log sidecar - #309

Merged
samgbafa merged 10 commits into
mainfrom
skgbafa/tc-959-n0-sidecar
Oct 17, 2026
Merged

samgbafa merged 10 commits into
mainfrom
skgbafa/tc-959-n0-sidecar

Conversation

@samgbafa

@samgbafa samgbafa commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Tracking: https://linear.app/tinycloud-labs/issue/TC-959

Compose-only GELF → Vector shipping on the unchanged production node 1.20.6-dstack / e18578f and unchanged ingress digests. No Rust/app-image or deployment-workflow edits. Canonical contract: https://github.com/samgbafa/tinycloud-dev/pull/131.

  • Default errors_slow: valid request status ≥400 OR node latency_us >1,000,000 / ingress request_time_seconds >1.0. Equality is excluded unless status ≥400. Explicit all_requests remains available. No sampling/throttle.
  • All valid node-labelled JSON WARN/ERROR records remain eligible, including dependencies; fixed codes/counts only, unknown targets unclassified, no target/message/error content. Malformed/oversized/plaintext records remain explicit safety/transport limitations. No fabricated IDs or panic event.
  • Native evidence: 32 JavaScript tests; 60 Vector cases per profile; native UDP→HTTP 5 positive controls, zero console canary hits and safe wire output. Profile/diagnostic boundary failed 13 cases before implementation and passes afterward.
  • Hosted acceptance PASSED at 44eeb20a1461ef66b5af492e3be4c42be2728b67: https://github.com/TinyCloudLabs/tinycloud-node/actions/runs/38963872225. Exact rendered-compose/runtime parity; unpaced fast-503 markers 100000–100499: 500 emitted/500 unique captured, zero missing/duplicates. Healthy-only source/gate/admitted/sink/capture deltas are all 500. All six injected diagnostic occurrences, 30 paused-sidecar health requests/resumption, injection isolation and privacy controls passed. Capture: 516 checked, zero invalid/incomplete/canary hits. Effective UDP buffer 2 MiB (kernel max 1 MiB); healthy socket-drop delta zero. Cleanup and always-cleanup both leave owner-labelled counts 0/0/0. Synthetic bootstrap substitutions do not claim dstack/TLS deployment proof.
  • Historical local burst FAILED: 203 source /203 gate /202 allowlist /201 HTTP sink /201 capture TOTAL across controls+burst; not a 500/500 result or established root cause. All local resources cleaned; no further local Docker.

Keep Draft pending Amos-owned independent exact-head Astra approval and remaining required checks. Hosted success is not #310 compose-binding or deployment authorization. Outage delivery remains BEST-EFFORT, not absolutely lossless. No merge/dispatch/deploy/secret read. The runbook records eligible-volume math, raw-log fallback loss, compose hash, pre-N0 rollback/privacy decision and the separate reviewed-workflow prerequisite; no unreviewed dispatch command is published.

@samgbafa

Copy link
Copy Markdown
Contributor Author

APPROVE: see TC-959

@samgbafa

Copy link
Copy Markdown
Contributor Author

APPROVE: see TC-959

samgbafa added a commit that referenced this pull request Oct 17, 2026
Template hash 92641ed5… over #309 e6d96c8 (raw b0cd8248…), whose
only change is CLIENT_MAX_BODY_SIZE 32m -> 95m. Pre-N0 template and
captured rollback artifacts are unchanged.

Refs TC-959
@samgbafa

Copy link
Copy Markdown
Contributor Author

Head ade9e04 has the same tree (525c5ef4) as the approved head 2f595cf (the revert of e6d96c8); the approval carries over unchanged. Raw docker-compose.dstack-postgres.yaml sha256 ca110b0b…, as reviewed. See TC-959.

@samgbafa
samgbafa marked this pull request as ready for review October 17, 2026 03:54
samgbafa added a commit that referenced this pull request Oct 17, 2026
Raise CLIENT_MAX_BODY_SIZE 32m -> 95m in the root compose (#309's
approved ca110b0b… becomes b0cd8248…, 2 bytes) and lock the release
template over those bytes: 92641ed5…. #309's approved pre-cap bytes no
longer satisfy the template. Pre-N0 template and captured rollback
artifacts are unchanged.

Refs TC-959, TC-1012
@samgbafa
samgbafa merged commit 199a9e3 into main Oct 17, 2026
21 checks passed
samgbafa added a commit that referenced this pull request Oct 17, 2026
Lock #309's compose as a template (raw bytes with the single node digest
normalized) so the main release and the sidecar ship in one dispatch.
Only a recorded released image (labels, vX.Y.Z tag and crate version
agree) may fill the pin; same-image recovery and the captured pre-N0
rollback are unchanged.

Refs TC-959
samgbafa added a commit that referenced this pull request Oct 17, 2026
Raise CLIENT_MAX_BODY_SIZE 32m -> 95m in the root compose (#309's
approved ca110b0b… becomes b0cd8248…, 2 bytes) and lock the release
template over those bytes: 92641ed5…. #309's approved pre-cap bytes no
longer satisfy the template. Pre-N0 template and captured rollback
artifacts are unchanged.

Refs TC-959, TC-1012
samgbafa added a commit that referenced this pull request Oct 17, 2026
* test(TC-959): use pre-N0 template fixture in recovery tests

Future-digest and normal-build recovery cases ran against the repository's
current compose. Since #309 that file is the reviewed N0 compose, which the
workflow intentionally refuses to pair with any image other than the reviewed
1.20.6 digest. Run these cases against a frozen copy of the pre-N0 template
(sha256 28302d67...) instead, and assert the N0 compose refuses a normal build
image as it already refuses a future digest.

* test(TC-959): derive pre-N0 template from the rollback artifact

Same bytes (sha256 28302d67...) as the frozen copy, without adding a fixture
file; matches the derivation used by #315.
samgbafa added a commit that referenced this pull request Oct 17, 2026
samgbafa added a commit that referenced this pull request Oct 17, 2026
…346)

* TC-959 log sidecar: pass validated N2 correlation fields through on node_request

request_id (UUID v1-8), request_id_replaced (bool), account_hash and space_hash
(64 lowercase hex) are copied only when exactly well-formed; anything else is
dropped and the request still ships. Ingress rows, diagnostics and db_hash are
unchanged. Regenerates the embedded Vector config and counts the widest valid
node request in the byte budget.

* TC-959 #346 review: re-pin n0 compose hash, allow correlation keys in assertCapture, anchor tests, request_id provenance note

* docs(TC-959): the N0 raw lock changed in #346; 6563d51b supersedes #309's ca110b0b
samgbafa added a commit that referenced this pull request Oct 18, 2026
Lock #309's compose as a template (raw bytes with the single node digest
normalized) so the main release and the sidecar ship in one dispatch.
Only a recorded released image (labels, vX.Y.Z tag and crate version
agree) may fill the pin; same-image recovery and the captured pre-N0
rollback are unchanged.

Refs TC-959
samgbafa added a commit that referenced this pull request Oct 18, 2026
Raise CLIENT_MAX_BODY_SIZE 32m -> 95m in the root compose (#309's
approved ca110b0b… becomes b0cd8248…, 2 bytes) and lock the release
template over those bytes: 92641ed5…. #309's approved pre-cap bytes no
longer satisfy the template. Pre-N0 template and captured rollback
artifacts are unchanged.

Refs TC-959, TC-1012
samgbafa added a commit that referenced this pull request Oct 18, 2026
…te (#315)

* ci: promote a released main image with the reviewed N0 compose template

Lock #309's compose as a template (raw bytes with the single node digest
normalized) so the main release and the sidecar ship in one dispatch.
Only a recorded released image (labels, vX.Y.Z tag and crate version
agree) may fill the pin; same-image recovery and the captured pre-N0
rollback are unchanged.

Refs TC-959

* ci: carry the 95m ingress body cap in the N0 release template

Raise CLIENT_MAX_BODY_SIZE 32m -> 95m in the root compose (#309's
approved ca110b0b… becomes b0cd8248…, 2 bytes) and lock the release
template over those bytes: 92641ed5…. #309's approved pre-cap bytes no
longer satisfy the template. Pre-N0 template and captured rollback
artifacts are unchanged.

Refs TC-959, TC-1012

* ci: refuse release downgrades under the N0 template and old same-image recovery

The promoted revision must strictly descend from the running production
revision; 12c same-image recovery refuses an image older than running.
The captured pre-N0 rollback stays the intentional downgrade route.

Refs TC-959
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant