feat(nzb-web): idempotent queue admission via Idempotency-Key - #149
Merged
Merged
Conversation
Add a durable, at-most-once admission path for queue adds so a caller that
retries an ambiguous `POST /api/queue/add` cannot create a duplicate job.
- New `queue_admissions` table (DB migration v11) keyed by idempotency key,
binding it to a payload digest and job id. It intentionally has no foreign
key to queue/history, so it stays authoritative after a job moves or bounded
history is pruned.
- `Idempotency-Key` request header (parsed/validated, path-safe, <=128 bytes)
and SHA-256 payload digest (`apps/rustnzb/src/admissions.rs`), plus
`GET /api/queue/admissions/{key}` to resolve one admission and its current
engine location (queue / history / unobserved).
- `Database::queue_admit` (transactional bind + queue insert) and
`queue_admission_observe`; `QueueManager::add_job_idempotent` /
`queue_admission_observe`, with `add_job` refactored so the shared activation
path is `activate_admitted_job`.
- New `NzbError::AdmissionConflict` -> HTTP 409 `admission_conflict` when a key
is reused with a different payload. A keyed add must carry exactly one NZB.
Ported from MrVampy/rustnzb (fe7c35d), adapted to main: migration renumbered
v9 -> v11 (main's v9/v10 are retry_data / damage_ledger); the fork's
flake.nix / workspace version bumps are excluded; digests use `hex::encode`
because main is on sha2 0.11 (whose output no longer implements LowerHex);
and the integration test now completes first-run auth setup and sends a bearer
token, since main gates `/api` behind auth middleware.
Tests: db `queue_admission_*` unit tests (replay/conflict, reopen/outlive);
`admissions` key + digest unit tests; and an end-to-end
`idempotent_admission` suite (exact replay returns one job, conflicting
payload 409s, keyed multi-payload upload 400s). `cargo test` across nzb-core,
nzb-web and rustnzb all pass; clippy/fmt clean.
Co-Authored-By: MrVampy <4302946+MrVampy@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The desktop app depends on `rustnzb`/`nzb-web` by path; the new dependencies added by this change must be reflected in desktop/src-tauri/Cargo.lock so the `desktop` CI job (built with --locked) accepts it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…-admission # Conflicts: # apps/rustnzb/src/server.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the idempotent queue-admission feature from
MrVampy/rustnzb(fe7c35d): a durable, at-most-once admission path for queue adds, so a caller that retries an ambiguousPOST /api/queue/addcannot create a duplicate job. Main had no equivalent.What it adds
queue_admissionstable (DB migration v11) keyed by idempotency key, binding it to a payload digest and job id. It intentionally has no foreign key to queue/history, so it stays authoritative after a job moves or bounded history is pruned.Idempotency-Keyheader handling (apps/rustnzb/src/admissions.rs): parsed/validated, path-safe, ≤128 bytes, with a SHA-256 payload digest; plusGET /api/queue/admissions/{key}to resolve one admission and its current engine location (queue/history/unobserved).Database::queue_admit(transactional admission bind + queue insert) andqueue_admission_observe.add_job_idempotentandqueue_admission_observe, withadd_jobrefactored so the shared activation path isactivate_admitted_job(no double insert).NzbError::AdmissionConflict→ HTTP 409admission_conflictwhen a key is reused with a different payload. A keyed add must carry exactly one NZB (multi-payload uploads 400).Port notes
retry_data) and v10 (damage_ledger).flake.nix/flake.lockand workspace version bumps are excluded.hex::encodebecause main is on sha2 0.11, whose output no longer implementsLowerHex(identical lowercase hex output)./apibehind auth middleware (the fork's test predated it).Tests
queue_admission_replays_exact_payload_and_rejects_conflict,queue_admission_survives_reopen_and_outlives_queue_observation.admissionsunit: key validation + digest binding.idempotent_admissionsuite: exact replay returns one job; conflicting payload 409s (error_kind: admission_conflict); keyed multi-payload upload 400s; observation endpoint reflects queue state.cargo clippy --workspace --all-targets+cargo fmt --checkclean.Original author: @MrVampy (credited via
Co-Authored-By; ported by hand due to migration collision and rewritten queue/handler code on main).🤖 Generated with Claude Code