Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.10.17
0.10.18
5 changes: 4 additions & 1 deletion internal/resources/provenance_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,10 @@ func TestBuildEnvSpecStampsUserProvenance(t *testing.T) {
t.Errorf("RESOURCE_LIMITS = %q, want cpu=6,memory=24Gi", env["RESOURCE_LIMITS"])
}
if env["RESOURCE_REQUESTS"] != env["RESOURCE_LIMITS"] {
t.Error("requests and limits diverged — Guaranteed QoS is the chart contract")
t.Error("requests and limits diverged — BuildEnvSpec writes them equal so a" +
" CPU training pod is Guaranteed. (Not \"the chart contract\", as this" +
" message said until backend#2872: the chart's derive path writes no cpu" +
" limit at all since backend#2418.)")
}
}

Expand Down
14 changes: 11 additions & 3 deletions internal/resources/resources.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,14 @@
// Grounding (verified against tracebloc/client + client-runtime, 2026-07):
// - The machine's capacity is the sum of Ready nodes' Status.Allocatable
// (the installer path is single-node, so this is normally one node).
// - A training run's ceiling is the jobs-manager env RESOURCE_LIMITS
// ("cpu=2,memory=8Gi", requests==limits for Guaranteed QoS). This is the
// - A training run's ceiling is the jobs-manager env RESOURCE_LIMITS. This
// package writes it equal to RESOURCE_REQUESTS on both dimensions, so a CPU
// training pod comes out Guaranteed. Two caveats this comment used to elide
// (backend#2872): the built-in fallback is the contract floor
// cpu=1,memory=2Gi since backend#2254, not the "cpu=2,memory=8Gi" named
// here before; and a GPU pod is BestEffort whatever this writes, because
// client-runtime's GPU path sets only nvidia.com/gpu and ephemeral-storage
// and neither counts toward QoS (backend#2871). This is the
// exact value client-runtime's jobs_manager.py stamps on spawned jobs and
// the same value `cluster doctor`'s checkNodeFit already parses — so the two
// read it identically (di#358 lesson: a reader must mirror the writer).
Expand Down Expand Up @@ -135,7 +141,9 @@ func MachineCapacity(nodes []corev1.Node) Machine {

// ParseTraining reads the per-run ceiling from a jobs-manager env map. It
// prefers RESOURCE_LIMITS (the true ceiling) and falls back to RESOURCE_REQUESTS
// (requests==limits by chart contract), then to the chart default so an older
// (which this package writes equal to the limits; NOT a chart-wide contract --
// the chart's derive path writes no cpu limit at all since backend#2418,
// backend#2872), then to the chart default so an older
// chart without the literal env still reports the effective size. GPU is read
// from GPU_LIMITS, then GPU_REQUESTS.
func ParseTraining(env map[string]string) Training {
Expand Down
11 changes: 9 additions & 2 deletions internal/resources/set.go
Original file line number Diff line number Diff line change
Expand Up @@ -194,8 +194,15 @@ func MemFloorText() string { return "2 GiB" }
const NoGPUEnvValue = ""

// BuildEnvSpec renders a per-run ceiling into the exact chart env the run is
// stamped with. RESOURCE_REQUESTS == RESOURCE_LIMITS (Guaranteed QoS, the chart
// contract), both "cpu=X,memory=Y".
// stamped with. RESOURCE_REQUESTS == RESOURCE_LIMITS, both "cpu=X,memory=Y" --
// which yields a Guaranteed CPU training pod.
//
// NOT "the chart contract", as this said until backend#2872. The chart's own
// DERIVE path (DERIVE_JOB_ENVELOPE) writes no cpu limit at all since
// backend#2418, so requests == limits is this package's choice for the explicit
// envelope, not a chart-wide invariant -- and calling it a contract is what let
// the claim survive after #2418 falsified it elsewhere. A GPU pod is BestEffort
// regardless of what is written here (backend#2871).
//
// Every dimension is ALWAYS written — never omitted. The apply uses
// `helm upgrade --reset-then-reuse-values`, which re-applies the release's
Expand Down
Loading