Skip to content

feat(jobs): end-to-end encrypted jobs (DIP-0032); v0.3.3 - #215

Merged
ralyodio merged 8 commits into
masterfrom
feat/e2e-jobs
Nov 9, 2026
Merged

ralyodio merged 8 commits into
masterfrom
feat/e2e-jobs

Conversation

@ralyodio

@ralyodio ralyodio commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

Job offers went to the gossip topic c0mpute/jobs/<workload> with the whole job inline. Every peer and relay on the topic could read every prompt, transcript and URL. Inputs and results were plaintext chunks, and anyone could fetch them by the hash that the offer and the receipt published. This PR makes every job end-to-end encrypted between the buyer and the one worker that wins. The design is in dips/0032-e2e-encrypted-jobs.md.

What changes on the wire

  • Offer: no spec. It carries a public spec (a per-workload allowlist of small scalars such as model, engine, voice, width/height, frame range and base_model, plus power-of-two size buckets), a fresh buyer X25519 key, and a salted commitment as spec_hash. required_capabilities gains c0mpute:e2e:v1, so pre-e2e workers never bid.
  • Bid: carries a fresh worker X25519 key, signed by the worker's libp2p ed25519 identity. The buyer checks the signature against the bidder's peer id before it seals anything.
  • Accept: carries no spec. The winner fetches the sealed spec over request-response (c0mpute.job.payload.v1). The buyer serves it only to the noise-authenticated winner, and only while the auction is open.
  • Crypto: X25519, then HKDF-SHA256, then AES-256-GCM. Keys and associated data are bound (length-prefixed) to job id, workload, both peer ids and both keys. The winner checks the commitment and that the public spec matches the sealed spec. It then re-runs the workload's own filter on the whole spec before running the job; on any failure it sends a Failed receipt.
  • Results: sealed to the buyer before they are stored. This covers every runner, render frames, text adapters and answers, gameserver results and inference. gateway_util::fetch_result / fetch_result_to_file open them with the auction's result key. Receipts name the ciphertext hash.
  • Inputs: STT audio, img2vid images, OCR documents, .blend files, training corpora and chat adapters are stored sealed, each under its own key, and the key rides inside the sealed spec. The winner decrypts inputs only in memory or in the job dir. A sealed .blend stays sealed in the cache. A sealed adapter is never put in the adapter cache. spec_ref (large specs stored as plaintext chunks) is no longer used, because sealed specs travel point to point.
  • Confidential option: a with_confidential(true) auction requires c0mpute:confidential. The offer carries an attest nonce. The report data binds the TEE to the sealed-to key. AttestationVerifier / AttestationProvider hooks fail closed. A concrete SEV-SNP/TDX producer and verifier are deferred.
  • Compatibility: workers still serve plaintext offers for this one release. C0MPUTE_REJECT_PLAINTEXT_OFFERS=1 refuses them now.

What the winning worker still sees

The plaintext of the job it computes on. For sensitive work, use private mode or the confidential requirement. Observers still see metadata: the public spec, price, timing, bidders, sizes, and text-train progress events.

Tests

  • Crypto unit tests: keys derive the same on both sides; a wrong worker cannot open; tampering, truncation, a moved job, a wrong commitment and low-order keys are all rejected; streamed files; public views carry no SECRET for any workload.
  • Dispatch: an e2e bid refuses plaintext and downgraded accepts.
  • tests/e2e_jobs_e2e.rs: a buyer, two workers and a passive gossip subscriber, across 15 workloads. The passive node sees no plaintext in any offer, bid, accept or receipt. The loser is refused the sealed spec. Result and input chunks fetched by the passive node are ciphertext. The buyer opens the results.
  • Existing auction, private, large result/input, ocr, render, text train (incl. replicas), workflow, inference, video, gameserver, temporal and crowdgpt e2e tests all pass. ads_e2e fails on dev2 only because the hosting disk floor refuses replicas at 46 GiB free; that is environmental and unrelated. Gossip tests need C0MPUTE_DISABLE_MDNS=1 on dev2's busy LAN, the same as before.

🤖 Generated with Claude Code

ralyodio and others added 3 commits November 9, 2026 12:07
Job offers went to gossip topics with the whole job inline, so every peer
and relay on c0mpute/jobs/<workload> read every prompt, transcript and URL,
and inputs and results were plaintext chunks anyone could fetch by the hash
the offer and receipt published.

- Offers carry only a public spec (per-workload allowlist of small scalars
  plus size buckets), a fresh buyer X25519 key and a salted commitment as
  spec_hash; buyers require c0mpute:e2e:v1 so pre-e2e workers never bid.
- Bids carry a fresh worker X25519 key signed by the worker's libp2p
  ed25519 identity; buyers verify it against the bidder's peer id.
- X25519 -> HKDF-SHA256 -> AES-256-GCM, keys and associated data bound to
  job id, workload, both peers and both keys. The accept carries no spec;
  the winner fetches the sealed spec over request-response
  (c0mpute.job.payload.v1), served only to the authenticated winner. The
  winner checks the commitment and the public spec, re-runs the workload's
  filter on the whole spec, then runs it.
- Results are sealed to the buyer before they are stored (results, render
  frames, text adapters and answers); buyers open them in fetch_result.
- Inputs (STT audio, img2vid images, OCR documents, .blend files, training
  corpora, chat adapters) are stored sealed under per-input keys carried in
  the sealed spec; spec_ref (large specs as plaintext chunks) is no longer
  used by buyers.
- Every offer filter judges the public spec at bid time.
- confidential requirement: c0mpute:confidential tag, attest nonce, report
  data binding the TEE to the sealed-to key, AttestationVerifier and
  AttestationProvider hooks, failing closed. SNP/TDX evidence is deferred.
- Workers still serve plaintext offers for one release
  (C0MPUTE_REJECT_PLAINTEXT_OFFERS=1 refuses them now).

Tests: crypto unit tests (wrong worker, tamper, truncation, moved job,
commitment, public views per workload), dispatch downgrade test, and
e2e_jobs_e2e: buyer, two workers and a passive gossip subscriber across 15
workloads; the passive node sees no plaintext, the loser is refused the
sealed spec, result and input chunks are ciphertext, the buyer opens them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

ThreatCrush Security Scan

13 finding(s)

HIGH/CRITICAL: 1 | MEDIUM: 8 | LOW: 4

Severity Rule Location
HIGH js-ssrf-outbound-request apps/web/public/sw.js:24
MEDIUM js-unescaped-html-sink apps/web/src/app/blog/[slug]/page.tsx:53
MEDIUM js-unescaped-html-sink apps/web/src/app/layout.tsx:89
MEDIUM sh-remote-script-execution scripts/dev-setup.sh:25
MEDIUM sh-eval-expansion scripts/dev-setup.sh:35
MEDIUM sh-remote-script-execution scripts/install.sh:181
MEDIUM sh-remote-script-execution scripts/install.sh:206
MEDIUM sh-remote-script-execution scripts/install.sh:399
MEDIUM sh-remote-script-execution scripts/install.sh:441
LOW py-ssrf-outbound-request examples/business-strategist-agent/chat.py:37
LOW py-ssrf-outbound-request examples/business-strategist-agent/train.py:54
LOW secret-generic-credential node/crates/c0mpute-core/src/webhook.rs:1171
LOW py-path-traversal node/crates/c0mpute-core/tests/fixtures/fake-infernet-text:29

Snippets are redacted; ThreatCrush never prints matched credential material.

ralyodio and others added 2 commits November 9, 2026 12:10
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v0.3.1 went to #216 meanwhile. Drops jobs_gateway::internal, unused since
specs no longer go through the bridge's store.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ralyodio ralyodio changed the title feat(jobs): end-to-end encrypted jobs (DIP-0032); v0.3.1 feat(jobs): end-to-end encrypted jobs (DIP-0032); v0.3.2 Nov 9, 2026
@ralyodio ralyodio changed the title feat(jobs): end-to-end encrypted jobs (DIP-0032); v0.3.2 feat(jobs): end-to-end encrypted jobs (DIP-0032); v0.3.3 Nov 9, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ralyodio
ralyodio merged commit 8e88d2c into master Nov 9, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant