Skip to content

feat(nzb-web): idempotent queue admission via Idempotency-Key - #149

Merged
thedancingdeveloper merged 3 commits into
mainfrom
feat/idempotent-queue-admission
Sep 27, 2026
Merged

thedancingdeveloper merged 3 commits into
mainfrom
feat/idempotent-queue-admission

Conversation

@thedancingdeveloper

Copy link
Copy Markdown
Collaborator

Summary

Ports the idempotent queue-admission feature from MrVampy/rustnzb (fe7c35d): a durable, at-most-once admission path for queue adds, so a caller that retries an ambiguous POST /api/queue/add cannot create a duplicate job. Main had no equivalent.

What it adds

  • queue_admissions table (DB migration v11) keyed by idempotency key, binding it to a payload digest and job id. It intentionally has no foreign key to queue/history, so it stays authoritative after a job moves or bounded history is pruned.
  • Idempotency-Key header handling (apps/rustnzb/src/admissions.rs): parsed/validated, path-safe, ≤128 bytes, with a SHA-256 payload digest; plus GET /api/queue/admissions/{key} to resolve one admission and its current engine location (queue / history / unobserved).
  • DB layer: Database::queue_admit (transactional admission bind + queue insert) and queue_admission_observe.
  • QueueManager: add_job_idempotent and queue_admission_observe, with add_job refactored so the shared activation path is activate_admitted_job (no double insert).
  • Conflict semantics: new NzbError::AdmissionConflict → HTTP 409 admission_conflict when a key is reused with a different payload. A keyed add must carry exactly one NZB (multi-payload uploads 400).

Port notes

  • Migration renumbered v9 → v11: main already uses v9 (retry_data) and v10 (damage_ledger).
  • The fork's flake.nix/flake.lock and workspace version bumps are excluded.
  • Digests use hex::encode because main is on sha2 0.11, whose output no longer implements LowerHex (identical lowercase hex output).
  • The integration test now completes first-run auth setup and sends a bearer token, since main gates /api behind auth middleware (the fork's test predated it).

Tests

  • DB unit: queue_admission_replays_exact_payload_and_rejects_conflict, queue_admission_survives_reopen_and_outlives_queue_observation.
  • admissions unit: key validation + digest binding.
  • End-to-end idempotent_admission suite: exact replay returns one job; conflicting payload 409s (error_kind: admission_conflict); keyed multi-payload upload 400s; observation endpoint reflects queue state.
  • Full suites pass: nzb-core (130), nzb-web (104+), rustnzb (all); cargo clippy --workspace --all-targets + cargo fmt --check clean.

Original author: @MrVampy (credited via Co-Authored-By; ported by hand due to migration collision and rewritten queue/handler code on main).

🤖 Generated with Claude Code

Add a durable, at-most-once admission path for queue adds so a caller that
retries an ambiguous `POST /api/queue/add` cannot create a duplicate job.

- New `queue_admissions` table (DB migration v11) keyed by idempotency key,
  binding it to a payload digest and job id. It intentionally has no foreign
  key to queue/history, so it stays authoritative after a job moves or bounded
  history is pruned.
- `Idempotency-Key` request header (parsed/validated, path-safe, <=128 bytes)
  and SHA-256 payload digest (`apps/rustnzb/src/admissions.rs`), plus
  `GET /api/queue/admissions/{key}` to resolve one admission and its current
  engine location (queue / history / unobserved).
- `Database::queue_admit` (transactional bind + queue insert) and
  `queue_admission_observe`; `QueueManager::add_job_idempotent` /
  `queue_admission_observe`, with `add_job` refactored so the shared activation
  path is `activate_admitted_job`.
- New `NzbError::AdmissionConflict` -> HTTP 409 `admission_conflict` when a key
  is reused with a different payload. A keyed add must carry exactly one NZB.

Ported from MrVampy/rustnzb (fe7c35d), adapted to main: migration renumbered
v9 -> v11 (main's v9/v10 are retry_data / damage_ledger); the fork's
flake.nix / workspace version bumps are excluded; digests use `hex::encode`
because main is on sha2 0.11 (whose output no longer implements LowerHex);
and the integration test now completes first-run auth setup and sends a bearer
token, since main gates `/api` behind auth middleware.

Tests: db `queue_admission_*` unit tests (replay/conflict, reopen/outlive);
`admissions` key + digest unit tests; and an end-to-end
`idempotent_admission` suite (exact replay returns one job, conflicting
payload 409s, keyed multi-payload upload 400s). `cargo test` across nzb-core,
nzb-web and rustnzb all pass; clippy/fmt clean.

Co-Authored-By: MrVampy <4302946+MrVampy@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
thedancingdeveloper and others added 2 commits September 27, 2026 11:11
The desktop app depends on `rustnzb`/`nzb-web` by path; the new dependencies
added by this change must be reflected in desktop/src-tauri/Cargo.lock so the
`desktop` CI job (built with --locked) accepts it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-admission

# Conflicts:
#	apps/rustnzb/src/server.rs
@thedancingdeveloper
thedancingdeveloper merged commit ddbb4fa into main Sep 27, 2026
11 checks passed
@thedancingdeveloper
thedancingdeveloper deleted the feat/idempotent-queue-admission branch September 27, 2026 22:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant