Skip to content

fix(azure): dedupe Redis recommendations against held reservations - #706

Merged
cristim merged 3 commits into
mainfrom
fix/699-redis-held-dedupe
Nov 6, 2026
Merged

cristim merged 3 commits into
mainfrom
fix/699-redis-held-dedupe

Conversation

@cristim

@cristim cristim commented Nov 6, 2026

Copy link
Copy Markdown
Member

Fixes #699.

Problem

managedredis.GetExistingCommitments built held commitments from the Consumption ReservationDetail usage feed. That feed has no purchase date or quantity, so every Redis commitment had a zero StartDate and a Count of 0. recfilter dropped them as old, and even a kept one added nothing. A Redis reservation bought in the last day therefore never reduced the matching recommendation.

Fix

Redis now reads held reservations from the Reservations API, through the same shared path compute uses:

  • It calls internal/reservations ListAppliedReservations and StrictCommitment, with ReservedResourceTypeRedisCache.
  • StartDate comes from the purchase date and Count from the quantity. Region and SKU come from the reservation itself.
  • Canceled, expired, failed and past-expiry reservations are excluded, and so are Shared reservations billed to another subscription.
  • An incomplete live row fails the lookup instead of being skipped, the same as compute. That covers a missing SKU, location, quantity or purchase date, an unattributable scope, and an expiry falling today. It's in the CHANGELOG.

The old seam (ReservationsDetailsPager and SetReservationsPager) is replaced by SetInventoryFactories, with a nil fallback to the default factories for both constructors. The CHANGELOG lists that as a source break; nothing outside managedredis used it.

Tests

  • Round trip (TestHeldReservation_RoundTripReducesRecommendation):

    1. A raw Consumption recommendation (Legacy and Modern payloads) goes through convertRecommendation.
    2. PurchaseCommitment for it is captured.
    3. The held reservation's SKU and location are built from the captured body.
    4. The recommendation and the reservation go through recfilter.NewDuplicateChecker(24).

    The held reservation reduces the recommendation. The test also asserts that the purchase's location equals the recommendation's region.

  • Table (TestHeldReservation_Table):

    • These reduce the recommendation: Shared billed to this subscription, Single scope, Creating, PendingBilling, purchased yesterday.
    • These don't: another subscription, Canceled, Expired, Succeeded past expiry, another region, another SKU, purchased three days ago.
    • These must fail with an error naming the field: missing quantity, expiry today.
  • Variant pin: one held reservation reduces only the first term/payment variant (P1Y upfront 1, P1Y monthly 2, P3Y 2 and 2). The test pins today's behaviour and names the known over-buy, fix(recfilter): one held commitment reduces only the first term/payment variant #703.

  • Default inventory: TestNewClient_DefaultInventoryFactories, for both constructors.

Verification

  • Fail-before: at main, the new test file doesn't compile, because the seam doesn't exist. Feeding the old converter's output through the checker fails the round-trip, table and variant tests.
  • Mutation probes, each failing named tests:
    • wrong reservation type;
    • no default-factories fallback;
    • StrictCommitment error ignored;
    • old converter restored;
    • the error-wrapper prefix dropped.
  • Checks: with GOWORK=off, build, vet and go test ./services/... in providers/azure pass. golangci-lint at the CI version reports 0 issues, and gitleaks finds no leaks.
  • Review: a separate agent reviewed the plan adversarially in 3 passes and the commits once (APPROVED). Its two Low findings are fixed in b95124e.

Not verified (no live Azure account)

  • Whether the Reservations API returns sku.name and location in exactly the case the Consumption API uses. There is no normalisation, matching compute.
  • Whether reservation quantity and recommendation count use the same units for sharded Premium caches.
  • Held coverage older than about a day is assumed to be netted out of Azure's own recommendations, the same semantics as AWS.

Follow-ups

🤖 Generated with Claude Code

cristim and others added 3 commits November 6, 2026 21:07
managedredis read the Consumption reservation-details feed, which has no
purchase date, quantity or location, so every held Redis reservation was
invisible to the duplicate guard. It now lists reservations through the
shared Reservations API path, as compute does, and fails closed on
incomplete live rows. The old seam's tests move to the new one.

Fixes #699

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round-trips a raw Consumption recommendation through a captured purchase
body into the held reservation, then checks the reduction, the
state/scope/date/region/SKU table, fail-closed rows, the variant
shared-budget behavior (go#703) and the default inventory wiring.

Refs #699

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The missing-quantity and expiry-today rows now assert their error text, and
the CHANGELOG lists the removed ReservationsDetailsPager seam as a source break.

Refs #699

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cristim cristim added triaged Item has been triaged urgency/this-sprint Within the current sprint priority/p1 Next up; this sprint severity/high Significant harm impact/many Affects most users effort/m Days type/bug Defect labels Nov 6, 2026
@cristim

cristim commented Nov 6, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 53 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between c83aef8 and b95124e.

Files selected for processing: 6

  • CHANGELOG.md
  • providers/azure/services/managedredis/client.go
  • providers/azure/services/managedredis/client_test.go
  • providers/azure/services/managedredis/held_reservation_dedupe_test.go
  • providers/azure/services/managedredis/pagination_cap_test.go
  • providers/azure/services/managedredis/slot_wiring_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@coderabbitai

coderabbitai Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor
Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@cristim

cristim commented Nov 6, 2026

Copy link
Copy Markdown
Member Author

APPROVED b95124e by claude-qa-ui-4

  • providers/azure/services/managedredis/client.go:212-232: GetExistingCommitments now reads the Reservations API through the shared reservations.ListAppliedReservations + StrictCommitment(..., ReservedResourceTypeRedisCache, ...), as compute does (compute/client.go:291-314), instead of the Consumption ReservationDetail shape. The field mapping (services/internal/reservations/inventory.go:232-318, strict.go) is unchanged by this PR.
Commitment field armreservations source Nil handling
StartDate Properties.PurchaseDate (inventory.go:274); StartDateIsDay=true (:277) nil on a live row is an error
Count Properties.Quantity (:271) nil or <=0 on a live row is an error
ResourceType SKU.Name (:268) blank is an error
Region Location (:265) blank is an error
State ProvisioningState plus ExpiryDate (:288)
Type filter ReservedResourceType = RedisCache nil is an error (strict.go:29)

PurchaseDate is date-only, so StartDateIsDay keeps the reservation counted until 24h after the purchase day (pkg/recfilter/dedupe.go:134-136), which errs toward suppression.

  • Dedupe: different SKUs or regions are not suppressed. Canceled, Expired, Succeeded past expiry, Failed, BillingFailed, Split and Merged are excluded. A Shared reservation billed to another subscription is skipped (strict.go:61-65). A partial match (held 2, recommended 5) leaves 3, with the money fields scaled.
  • With go#694: duplicate checking runs per client (dedupe.go:94-99); the cache client's converter is still the Consumption one with no StartDate, so it suppresses nothing and the two inventories are never summed.

Runs (GOWORK=off, GOMAXPROCS=4, -p 2): build and vet pass; go test passes for managedredis, internal/reservations, compute and cache; golangci-lint reports 0 issues. 16 probes, all killed by managedredis tests:

  • StartDate dropped or shifted 2 days;
  • StartDateIsDay false;
  • Count dropped, constant 1 or doubled;
  • SKU lowercased;
  • wrong region;
  • Cancelled or Expired mapped to active;
  • the past-expiry check removed;
  • PendingBilling mapped to canceled;
  • the type filter changed to VirtualMachines;
  • a StrictCommitment error skipped with continue;
  • the list error swallowed;
  • nil factories.

Non-blocking:

@cristim

cristim commented Nov 6, 2026

Copy link
Copy Markdown
Member Author

APPROVED b95124e by claude-cc-libs

Cold, non-author review of the change against main (c83aef8), including tests and mutation probes. The PR moves managedredis inventory to the Reservations API path compute uses, which is the source #699 recommended. It does not patch the ReservationDetail usage row.

Blocking: none.

Non-blocking

  1. New ways for Redis dedupe to fail (services/managedredis/client.go:212-232). An incomplete live reservation fails the inventory read (strict.go:108-127), and so does a date-only expiryDate of today (strict.go:176-183). The identity also needs read access on Microsoft.Capacity reservations, which ReservationDetail didn't. On failure, recfilter.AdjustRecommendationsForExisting (pkg/recfilter/dedupe.go:57-59) returns an error, so Redis dedupe stops for that run instead of silently passing. This is deliberate, matches compute and is in the CHANGELOG. Consumers should expect a lookup that used to succeed to error now.
  2. Term and payment aren't part of the dedupe key (dedupe.go:146), and the held budget is used up by the first matching variant (:299-305). This predates the PR, TestHeldReservation_VariantPin pins it, and fix(recfilter): one held commitment reduces only the first term/payment variant #703 tracks it.
  3. Region and SKU compare as exact strings with no case normalization (dedupe.go:206, :292). Compute has the same exposure.
  4. Five services still read ReservationDetail: database, cosmosdb, search, synapse and cache. fix(azure): Cosmos DB, SQL, Search and Synapse reservations never dedupe recommendations #705 tracks them.
  5. Possible midnight flake. held_reservation_dedupe_test.go:27 computes fixtureToday once at package init, while StrictCommitment reads time.Now, so a run that crosses UTC midnight could flake the "expires today" row.

Checked

  • Fields against armreservations v1.1.0 (models.go): PurchaseDate (:591), Quantity (:594), ExpiryDate (:577), ProvisioningState (:588), ReservedResourceType (:608), Location (:1182), SKU (:1188). RedisCache is defined at constants.go:510.
    • inventory.go:269-273 sets StartDate from PurchaseDate (StartDateIsDay). That's the fixed purchase date, not EffectiveDateTime and not a moving usage date.
    • Count is the reserved Quantity (inventory.go:266-267).
    • Order IDs are deduped (inventory.go:198-209), so there are no per-day duplicate rows.
  • Region now comes from r.Location (inventory.go:260-262), not the client region. That resolves the fix(azure): commitments fabricate State "active" and Region from fields the SDK omits #74 fabrication concern for Redis.
  • Over-suppression:
    • The table test confirms these are not suppressed: a different SKU or region, Canceled, Expired, Succeeded past its expiry, a reservation purchased three days ago (outside the 24h lookback), and a Shared reservation billed to another subscription.
    • Creating, PendingBilling and a reservation purchased yesterday are suppressed.
  • Composition:
  • Tests: the round-trip test uses the real convertRecommendation, appendPage, the purchase request body, StrictCommitment and recfilter.NewDuplicateChecker(24). Main's shape (StartDate and Count zeroed) fails it.

Mutation probes: 9 of 9 killed. They covered dropping StartDate, dropping StartDateIsDay, dropping Count, setting Count=1, taking StartDate from EffectiveDateTime or from ExpiryDate, taking Region from the client region, main's shape, and a ServiceCache label.

Commands

  • From providers/azure: go build ./... and go test -p 2 ./services/managedredis/ . -count=1: pass.
  • From pkg: go test ./recfilter/: pass.
  • golangci-lint on managedredis: 0 issues.

@cristim

cristim commented Nov 6, 2026

Copy link
Copy Markdown
Member Author

Merge gate at b95124e:

  • Approvals at this SHA: claude-cc-libs and claude-qa-ui-4.
  • CI and state: CI green, 0 unresolved threads, merge state CLEAN, and the branch contains current main.
  • Local verification (GOWORK=off):
    • build, vet and go test ./services/... in providers/azure pass; lint and gitleaks are clean;
    • the new tests don't compile on main, and the old converter fails the round-trip, table and variant tests;
    • five mutation probes each fail a named test.
  • Unverified without a live Azure account: SKU and location case, and Premium shard units. Both are stated in the body.

@cristim
cristim merged commit bfb0f89 into main Nov 6, 2026
15 checks passed
@cristim
cristim deleted the fix/699-redis-held-dedupe branch November 6, 2026 20:46
cristim added a commit that referenced this pull request Nov 6, 2026
Drop the stale held-reservation caveat (fixed by #706) and note that Shards is no longer reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cristim added a commit that referenced this pull request Nov 6, 2026
…ducer (#702)

* feat(azure/managedredis): export ReservationUnitPrice

Advisor pricing looks up a reservationUnitPricer per service. managedredis
only had the private getRedisPricing, so routing Redis rows through it
would skip every row. Export the same wrapper the cache client has.
Unknown prices and unsupported SKU identities return an error, never 0.

Refs #692

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(azure): collect and price Redis through managedredis (ServiceMemoryDB)

Azure Redis recommendations were labeled ServiceCache, a service whose
client is about to stop buying Redis. Swap the single Redis producer in the
aggregate and Advisor paths from the cache client to managedredis, so every
Redis recommendation carries ServiceMemoryDB:

- the seam newCacheClientFn becomes newManagedRedisClientFn
- the Redis service filter follows ServiceMemoryDB
- the Advisor rediscache mapping resolves to memorydb, so coverage by
  Consumption rows keeps matching on the same service label
- Advisor rows are priced through managedredis.ReservationUnitPrice

A guard test asserts no returned recommendation has Service ServiceCache.

Refs #692

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(azure/cache): stop emitting Redis recommendations (#692)

managedredis is now the only producer of Azure Redis recommendations, so
cache GetRecommendations becomes a documented no-op. The Consumption
pager, converter, appendPage and their tests go with it. The catalogue
and ReservationUnitPrice removal follows in the next commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(azure/cache): drop the Redis SKU catalogue and unit pricer (#692)

After the recommendations no-op, nothing reads the shard-count catalogue,
ReservationUnitPrice (managedredis prices Advisor rows now) or the
recommendations pager. Remove them with the tests that only exercised them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): note Azure Redis recommendations moved to managedredis (#692)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): mark the Azure Redis moves as breaking (#692)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(azure/managedredis): price Consumption recs through ReservationUnitPrice (#692)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(azure): drop a stray empty file added by mistake (#692)

providers/azure/nice was created by a mis-parsed shell command in
913cb69 and is not part of the change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(azure): set CacheDetails on managedredis recommendations (#692)

Engine filters and the CSV Engine column read CacheDetails, which the removed cache converter set. Restore Engine=redis and NodeType; Shards stays 0 (unknown).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): correct the Azure Redis routing entry (#692)

Drop the stale held-reservation caveat (fixed by #706) and note that Shards is no longer reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(azure): tag held Redis reservations with the redis engine (#692)

The dedupe key includes the engine. Recommendations now carry CacheDetails.Engine "redis", so held reservations without an engine no longer reduced them (TestHeldReservation_RoundTripReducesRecommendation).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(azure): give Advisor Redis recommendations the redis engine (#692)

Held Redis reservations are keyed with engine "redis", so an Advisor-only Redis row with no Details stopped being reduced by the duplicate checker. Share one managedredis.Details helper between the Consumption and Advisor paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
cristim pushed a commit that referenced this pull request Nov 6, 2026
Brings in #702 (managedredis is the only Azure Redis recommendation
producer) and #706 (Redis dedupe against held reservations).

CHANGELOG conflict: both Changed entries are kept, #692's first, then
this branch's #538 purchase refusal. cache/client.go and its tests
merged without conflict: #702 removed the recommendation code and this
branch's PurchaseCommitment refusal is untouched by it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/m Days impact/many Affects most users priority/p1 Next up; this sprint severity/high Significant harm triaged Item has been triaged type/bug Defect urgency/this-sprint Within the current sprint

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(azure): Redis reservations never dedupe recommendations (inventory has no StartDate or Count)

1 participant