Repository navigation
fix(ladder): convert existing coverage to on-demand-equivalent dollars and never claim legacy tranches - #1238
Conversation
…s and never claim legacy tranches The AWS provider reports LayerState.ExistingUSDPerHour and ExpiringUSDPerHour in commitment dollars (SavingsPlan.Commitment for the SP layers, amortized cost for the RI layer), but Allocate compares them with an on-demand-equivalent target. The planner passed them through unconverted, so existing coverage was understated by the commitment rate and the gap overstated: a 2.5 commitment $/hr EC2 Instance SP at rate 0.5 was netted as 2.5 OD instead of 5. executeLadderRun now converts each layer with od = commitment / rate, using the GetCommitmentRates rate the run already fetched (option 2 on the issue). Existing rounds up and expiring down, so net coverage is never understated. A layer with non-zero existing or expiring commitment and no rate fails the run instead of being netted 1:1. ladder_runs.existing_usd_hr and the plan JSON existing are now on-demand-equivalent. Per the maintainer's note on the issue, ClaimDueLadderTranches now claims only amount_basis = 'commitment' rows. Legacy NULL-basis and on_demand_equivalent rows stay netted until the overdue sweep cancels them, so an executor can never buy their OD amount as commitment dollars. Closes LeanerCloud#946 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Independent review 2 (claude-fleet, cold): APPROVE 6af6413 Findings (none blocking on their own):
Attacked:
Verified locally, with GOTOOLCHAIN=go1.26.10 and no cloud credentials, under the heavy lock:
Reviewed by an independent agent with no author context (second reviewer). Review only: nothing was pushed, and the GitHub review state is unchanged. |
|
VERDICT: APPROVE 6af6413 Merge check: merge-tree vs origin/main 2ded648 is clean (tree 9f4c82b0). go build ./... and go vet -tags integration on config/server/purchase/api pass on the merged tree. Findings:
Attacked:
@IAmJSD CI on this head is red only from the old merge base (govulncheck on go1.26.9, fixed on main by #1236). Please merge or rebase onto current |
|
Merge-order note from review: when this PR and #1199 are both on one tree, |
Brings in main's CI fixes (LeanerCloud#1236, LeanerCloud#1252) and CI changes since, so this PR gets a fresh CI run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: REQUEST_CHANGES Blocking:
Non-blocking:
Attacked:
Body-stale:
Reviewer: rB21. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths. |
…p test After merging main, TestLadderTrancheAmount_RoundTripsProviderPrecisionContract (LeanerCloud#1199) claimed 0 of its 19 rows: it seeded NULL-basis tranches, which this branch makes unclaimable. Seed commitment-basis rows at rate 1 through claimableTranche instead, so the claim path still reads every amount back. The assertions are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rate cloud-commitments-go only returns a discount rate for EC2 Instance SPs, so accounts holding Compute SPs or Convertible RIs (and EC2 Instance SP holders whenever Cost Explorer returns no rate) cannot have their existing coverage converted to on-demand-equivalent dollars. Failing those runs made every tick Errored and stopped ladder planning for most AWS customers. Owner decision D3: persist the run as a Hold that buys nothing and says why. ladderLayerStatesOnDemand now reports every layer that holds commitment but has no rate, and executeLadderRun persists a Hold-only run with plan reason_code "missing_discount_rate" and one Hold per layer whose rationale starts "Can't size this yet because". Allocate never runs, so no rate is guessed or netted 1:1; existing and gap stay NULL, which also keeps the amend ceiling from allowing raises against that run. The run counts as planned and logs one WARN line, not an error. Invalid rates and negative or non-finite amounts still fail the run. Target-met and other Allocate Holds are unchanged and carry no reason_code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Covers executeLadderRun end to end for each AWS layer kind holding coverage with no rate (Compute SP, Convertible RI, EC2 Instance SP): the run is persisted with no tranches, NULL existing and gap, reason_code missing_discount_rate and a "Can't size this yet" Hold naming the layer. A genuinely met target must still produce the ordinary "target already met" Hold with no reason_code, and bad amounts on a rate-less layer must still fail the run. The first three fail on the previous fail-loud head. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ladder timeline already shows each action's rationale, so the UI tells a
missing-rate Hold ("Can't size this yet because ...") from a target-met Hold
("target already met: ...") without a frontend change. Projecting the plan's
reason_code onto /api/ladder/runs gives API clients the same distinction in
machine-readable form. Runs without one omit the field.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both reviews flagged the reflowed comment line at handler_ladder.go:284 as over 100 columns. Comment only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Updated to owner decision D3. New head: What changed since 4fe4f58:
Checks: build and vet (plain, integration and e2e) are clean. golangci-lint, with and without integration tags, reports 0 issues, and gocyclo is clean. |
|
Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE Blocking:
Non-blocking:
Attacked:
Body-stale:
Reviewer: rC1. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths. |
|
Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE Blocking:
Non-blocking:
Attacked:
Body-stale:
Reviewer: rC4. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths. |
|
Local verification at |
|
Merged by IAmJSD's agent at IAmJSD's request (IAmJSD is acting owner while the maintainers are away), under the bar IAmJSD set: 2 approving independent review(s) on |
Summary
The ladder compared an on-demand-equivalent (OD) target with existing Savings Plan coverage measured in commitment dollars. This PR converts existing and expiring coverage to OD dollars before
Allocate. Following the maintainer's note on the issue, it also makes legacy NULL-basis andon_demand_equivalenttranches impossible to claim.When existing coverage cannot be converted because the provider returned no discount rate for its layer, the run is persisted as a Hold that buys nothing and says why (
reason_code: missing_discount_rate). An earlier revision failed these runs instead; see "Owner decision" below.Closes #946
Owner decision (D3, 2026-10-26)
The first revision made any run with existing coverage and no rate Errored on every tick. Both reviews (cristim, rB21) pointed out that this stops ladder planning for most AWS customers. cloud-commitments-go only returns a rate for EC2 Instance SPs, so accounts holding Compute SPs or Convertible RIs had no rate. EC2 Instance SP holders had none either whenever Cost Explorer had no recommendation (
ErrNoSPCommitmentRate, an empty rate map).The acting owner (IAmJSD) decided to replace fail-loud with a persisted Hold-only run that states the reason. The allocator still never gets a guessed rate: no 1:1 netting and no estimate.
Units
LayerState.ExistingUSDPerHourandExpiringUSDPerHourare in this unit:SavingsPlan.Commitmentfor the SP layers and amortized RI cost for the RI layer.ris in (0, 1] and means commitment dollars per OD dollar.od = commitment / randcommitment = od * r. This is the same rule go'sallocate_units.gouses for tranches.Root cause
executeLadderRunpassedGetLayerStatesstraight toAllocateandassembleLadderPlan. go'sLayerStatedoc says callers must convert these values to OD first. Existing coverage was therefore understated by a factor ofrand the gap overstated. Example: a 2.5 commitmentFix
internal/server/ladder_existing_units.go(new):ladderLayerStatesOnDemand(states, rates)returns a converted copy of the states and never mutates the provider's map.executeLadderRunalready fetched fromGetCommitmentRatesfor the config's term and payment option. This is option 2 on the issue. DescribeSavingsPlans does not expose each plan's own rate, so plans bought at a different discount are approximated by today's rate.big.Rat). When the quotient is not a float64, existing rounds up and expiring rounds down. Net coverage (existing - expiring) is therefore never understated, which errs toward under-buying.Allocatestill rejects it.planMissingRateHold): when any layer is missing a rate,executeLadderRunpersists a run through the same lockedPlanAndSaveLadderRunpath, andAllocateis never called.planned, no tranche rows andtotal_hourly_commit = 0."reason_code": "missing_discount_rate"and oneholdaction per layer. Each action's rationale names the layer, its commitment amounts and the term and payment option, and starts with "Can't size this yet because". Example:Can't size this yet because layer compute-sp holds existing coverage (2.5 commitment USD/hr, 0.5 expiring) and the provider returned no discount rate for it at 1yr no-upfront, so that coverage cannot be converted to on-demand-equivalent dollars. Nothing is bought this run: the rate is never estimated or assumed 1:1. The ladder sizes the gap once a rate is available.target_usd_hris reported.existing_usd_hrandgap_usd_hrstay NULL (unmeasured, never 0). A NULL existing snapshot makes the amend ceiling refuse every raise against this run, so a stale commitment-dollarexistingfrom an older run can no longer keep the ceiling loose.ladder_run: config X: persisted Hold-only run Y (reason_code=missing_discount_rate): ...) and counts asPlannedinLadderRunResult, notErrored.Allocate, is skipped for these runs. Reshapes are display-only today.AllocateHold) is unchanged. It has noreason_code, and its rationale is go'starget already met: ....GET /api/ladder/runsnow returns the plan'sreason_codeon each run (omitted when empty).executeLadderRunconverts right after the rate fetch. The converted states feedAllocateand the plan, soladder_runs.existing_usd_hrand plan JSONexistingare now OD. That matches go'sLadderPlan.ExistingUSDPerHourdoc and the amend ceiling (LEAST(target, baseline*coverage) - existing), which already assumed OD. The ceiling therefore gets stricter, never looser.ClaimDueLadderTranches(selectDueLadderTranchesTx) now selects onlyamount_basis = 'commitment'rows. The validatedladder_tranche_amount_provenanceconstraint guarantees those rows have acommitment_rate.on_demand_equivalentrows are never claimed. They stay netted while withinLadderFireGrace. After that, the existing overdue sweep cancels them and the next run re-plans their gap in commitment dollars ("cancelled or re-planned", with no migration).LIMIT ... FOR UPDATE SKIP LOCKED. Older legacy rows therefore cannot use up a claim's limit and starve a due commitment row.Visible behavior changes
existing_usd_hron new sized runs is OD. Historicalladder_runsrows keep their commitment-dollar snapshot because no rate was stored to backfill from. If the latest run predates this change, the amend ceiling uses that older, smaller existing value until the next run writes a new one. A missing-rate Hold writes NULL, which refuses raises./api/ladder/runsgains an optionalreason_codefield.Verification
All evidence comes from unit, fake-capability and local-PostgreSQL tests. Nothing ran against a real AWS account. Tests ran with
AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true.Missing-rate Hold, fail-before (the new
ladder_missing_rate_hold_test.gorun on the previous fail-loud code, merged with main at 14e41c5):TestExecuteLadderRun_MissingDiscountRatePersistsHoldWithReason/{compute-sp,convertible-ri,ec2-instance-sp}FAIL with an error ("a missing rate is a Hold, not an Errored run").TestExecuteLadderRun_MissingDiscountRateNamesEveryLayerFAILS with an error.TestExecuteLadderRun_InvalidCoverageWithoutRateIsErroredFAILS: the old code reported "no commitment rate" instead of rejecting the negative amount.TestExecuteLadderRun_TargetMetHoldKeepsOrdinaryReasonpasses before and after. It is the guard that a met target still gives the ordinary Hold.Pass-after (head 9b19af3):
total_hourly_commit0, NULL existing and gap, and target 10;reason_codeismissing_discount_rate;holdnames the layer, its amounts and1yr no-upfront, starts "Can't size this yet because", and does not say "already met".reason_code, and its rationale startstarget already met:.TestLadderLayerStatesOnDemand_ReportsEveryLayerWithoutARatereturns no partial map...._RejectsUnconvertibleCoveragenow also covers negative or NaN amounts on a rate-less layer.TestLadderTimelineRuns_ProjectsPlanReasonCode(PostgreSQL) checks that/api/ladder/runsreturnsreason_codefor the missing-rate run and omits it for the target-met run.Unit conversion and claim filter (from the first revision; still pass):
origin/mainc8bdd46: theexecuteLadderRunexisting/expiring tests showed existing 2.5 (want 5), gap 7.5 (want 5), commit 3.75 (want 2.5), and gap 8 (want 6).TestClaimDueLadderTranches_NeverClaimsOnDemandOrUnknownBasisRowsclaimed the NULL row and fired the NULL and OD rows.TestLadderLayerStatesOnDemand_PinsCommitmentToOnDemandConversion(5 @ 0.5 → 10, 3 @ 0.25 → 12, 7 @ 1 → 7),..._RoundsTowardUnderBuying,..._DoesNotMutateInputand..._EmptyLayersNeedNoRate.Merged with main (including #1199):
TestLadderTrancheAmount_RoundTripsProviderPrecisionContractclaimed 0 of 19 rows on the merged tree, because its NULL-basis fixture is no longer claimable. It now seeds throughclaimableTranchewith the assertions unchanged. It fails before that change ("[]" should have 19 item(s), but has 0) and passes after.Checks on 9b19af3:
go build ./...,go vet ./...,go vet -tags integration ./...andgo vet -tags e2eintests/e2e: clean.--build-tags integration: 0 issues.gocyclo -over 10on the changed non-test files: silent.go test -race -short -count=1on api, config, purchase and server: ok.go test -race -tags integration -count=1on config, api, purchase and server (whole packages, testcontainers PostgreSQL via Docker): ok, with no skips (config 333s, api 351s, purchase 138s, server 166s). The full test runs were on 8d8619b. 9b19af3 only rewraps a comment; build, vet, lint and-race -shorton server were re-run on it.scripts/test-git-secrets-allowlist.sh: 183 PASS, 0 FAIL.Out of scope / follow-ups
sumSPHourlyCostsums EC2 Instance SPs from every region, while the EC2 Instance SP rate is regional. This is pre-existing and noted here only.ladder_runs.existing_usd_hrrows are not backfilled.LadderAmountFloathas no production caller now that fix(ladder): floor tranche amounts to the stored scale instead of rounding them up #1199 is in (cristim's note). It is left for a separate cleanup.Frontend
No change was needed. The timeline already shows run rationales and renders a NULL existing as "unavailable", so a missing-rate Hold reads "Can't size this yet because ..." and a met target reads "target already met: ...". New sized runs now match the chart's OD base label. Only runs from before this PR show a commitment-dollar base.
🤖 Generated with Claude Code