deficienta: publish episodes, fix the strazi asymmetry, enforce the contract - #3
Conversation
…ontract
CMTEB publishes three states - Oprire, Deficienta (pressure/temperature below
spec) and normal. The headline `days` counts only oprire+ACC, which is correct
and deliberate. But deficienta was being flattened to a bare day count, and on
streets not even that.
Measured on the current published bundle: 15,747 deficienta PT-days in 2025
against a 26,559-day headline (59%), rising to 73% year-to-date in 2026, with
~85% of them falling on days no outage touched.
These three changes MUST land together: validate's new key constants require
the fields publish now emits, so landing validate alone would FAIL the next
nightly on all ~11,700 entity-years and block the release.
1. Publish deficienta episodes (PT only - streets have no episode array).
The display record is hoisted above the severity branch and shared, so
oprire and deficienta cannot drift apart. Three additive keys:
episodes_deficienta / episodes_count_deficienta / est_hours_deficienta.
Strictly additive: the deficienta path still `continue`s before pt_days,
city_eps, pt_epn and pt_hours, so `days` and everything derived from it are
untouched. Proven by test, which asserts est_hours stays 24.0 and
episodes_count stays 1 while the deficienta counterparts fill.
2. Emit days_deficienta on the strazi ndjson year object. `defi` was already
computed there as the emission gate and thrown away, so street-years shipped
deficienta RUNS with no counter to reconcile against - while ARTIFACTS.md:91
claims the union invariant holds "for every PT-year and street-year".
Verified against the live bundle: absent on all 6,670 street-years.
3. Enforce the contract in validate.py, which mentioned deficienta exactly once
(in a docstring) and asserted nothing:
runs_days_union FAIL - the ARTIFACTS.md:90-91 guarantee
ndjson_year_keys FAIL - a missing key renders NaN on the site
deficienta_reconciliation FAIL - exact biconditionals by construction
rankings_row_keys FAIL - exact contract key set per row
deficienta_non_vacuous WARN - heuristic; FAIL only if EVERY
namespace-year is zero, since a quiet year is
plausible upstream and blocking the release on a
CMTEB editorial change would be wrong
The union check unions day-of-year numbers rather than summing run lengths:
classes legitimately overlap, and summing would FAIL on correct data.
The ranking key check is a PRE-PASS, before the existing `break`s. Those stop
at the first bad row, so a key regression further down would never be seen;
it also fixes a latent crash where a malformed row raised KeyError outside
any try and killed validate with a traceback instead of a FAIL.
Verified against the live published bundle before enforcing: PT 5,054
entity-years and street 6,670 entity-years, ZERO union mismatches on either.
The invariant is real, so these checks will not produce false FAILs.
Detail strings carry only slugs, years and integers - never cause_raw - so
untrusted CMTEB text cannot forge PASS/FAIL lines in the CI log (SEC049).
tests/test_publish_shapes.py now imports the key sets from validate instead of
redeclaring them, so the CI-visible check and the db-gated check cannot drift.
Its fixtures were also non-vacuous-ified: strazi ranking was [] and both ndjson
year objects were {"days": 3}, so several of these checks would have passed
without asserting anything.
45 passed, 15 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Reviewed the full diff against CLAUDE.md's contract rules. Verified: the deficiență aggregation in publish.py is genuinely additive — epd is hoisted but the deficiență branch still continues before touching pt_days/city_eps/pt_epn/pt_hours, and the new parallel pt_eps_defi/pt_epn_defi/pt_hours_defi maps never feed the headline structures. The strazi days_deficienta fix correctly reuses the already-computed defi gate variable (pipeline/publish.py:798). validate.py's new checks are sound: _run_day_set unions day-of-year numbers rather than summing run lengths (correctly handles legitimate avarie/programat overlap), the rankings key pre-pass runs before the existing breaks so it also fixes a latent unguarded KeyError crash, and deficienta_non_vacuous is correctly scoped to WARN unless every namespace-year is simultaneously zero. ARTIFACTS.md is updated in the same commit as the emitting code and its validate assertion, per the repo's additive-contract rule. Tests cover the new checks plus an end-to-end publish.build() proof that headline fields stay blind to deficiență episodes.
CMTEB publishes three states —
Oprire,Deficiență(pressure/temperature below spec) and normal. The headlinedayscounts only oprire+ACC, which is correct and deliberate. But deficiency was being flattened to a bare day count, and on streets not even that.Measured on the current published bundle: 15,747 deficiency PT-days in 2025 against a 26,559-day headline (59%), rising to 73% year-to-date in 2026, with ~85% falling on days no outage touched.
These three must land together
validate's new key constants require the fieldspublishnow emits. Landing validate alone would FAIL the next nightly on all ~11,700 entity-years and block the release.1. Publish deficiency episodes (PT only — streets have no episode array)
The display record is hoisted above the severity branch and shared, so oprire and deficiency cannot drift apart. Three additive keys:
episodes_deficienta,episodes_count_deficienta,est_hours_deficienta.Strictly additive. The deficiency path still
continues beforept_days,city_eps,pt_epnandpt_hours.test_deficienta_episodes_published_without_touching_headlineproves it end-to-end throughpublish.build(): with one oprire and one deficiency episode,days == 2,episodes_count == 1andest_hours == 24.0whiledays_deficienta == 3and the deficiency counterparts fill.2. Emit
days_deficientaon the strazi ndjson year objectdefiwas already computed there as the emission gate and thrown away. So street-years shipped deficiency runs with no counter to reconcile them against — whileARTIFACTS.md:91claims the union invariant holds "for every PT-year and street-year". Verified against the live bundle: absent on all 6,670 street-years.3. Enforce the contract in
validate.pyIt mentioned deficiency exactly once, in a docstring, and asserted nothing.
runs_days_unionndjson_year_keysNaNon the sitedeficienta_reconciliationrankings_row_keysdeficienta_non_vacuousdeficienta_non_vacuousFAILs only if every namespace-year is zero. A single quiet year is plausible upstream, and blocking the nightly release on a CMTEB editorial change would invert the design intent.Two subtleties worth reviewing:
breaks. Those stop at the first bad row, so a key regression further down would never be seen. It also fixes a latent crash: a malformed row previously raisedKeyErroroutside anytry, killing validate with a traceback instead of a FAIL.Verified against real data before enforcing
Ran the new invariant over the live published bundle:
The invariant is genuinely true in production, so these checks will not produce false FAILs. And end-to-end (new publish → new validate) reports 0 FAIL with all five new checks PASS.
Notes
cause_raw, so untrusted CMTEB text cannot forge PASS/FAIL lines in the CI log (SEC049).tests/test_publish_shapes.pynow imports the key sets fromvalidateinstead of redeclaring them, so the CI-visible check and the db-gated check cannot drift apart.strazi-2025.jsonwas[]and both ndjson year objects were{"days": 3}, so several of these checks would have passed while asserting nothing.45 passed, 15 skipped.🤖 Generated with Claude Code