backlog(#347): a PHI-at-rest assertion that can pass for the wrong reason

[wshallwshall]

One file, docs only: docs/BACKLOG.md +1 item. No code, no test, no CI change.

The defect

tests/test_store_encryption.py:95 asserts raw.startswith(MARKER_PREFIX) and "DOE" not in raw — three characters of a 76-character body — as the proof that a patient surname is unreadable at rest. The instrument is wrong in both directions:

• it fails when encryption worked perfectly — the value is encrypted under make_cipher(generate_key()), a fresh random key every run, so the base64 body is fresh random text and base64's alphabet contains D, O and E; and
• it would pass on a weak encoding that merely happened to avoid those three characters.

The second half is the PHI defect. The flake is what made someone look; it is not what is wrong.

The rate — per CI run, not per assertion

Surfaced on PR #142, job 91502517146, leg test (windows-2022, py3.14). (L-k+1)/64^k at the observed ~146-char ciphertext gives p ≈ 5.5e-4 for a 3-char literal. But there are two such assertions (:95, :303) and this repo runs three OS legs (ubuntu + windows-2022 + windows-2025, verified against ci.yml):

Scope	Rate
one assertion, one leg	1 in 1,820
either assertion, one leg	1 in 910
either assertion, one full CI run	1 in 303

Both over-estimate caveats are stated in the item rather than buried: L comes from one measured value, and the mfenc:v1:<hex-fingerprint>: prefix is not base64, so the effective window count is lower.

Fix scope — neither "just one" nor "all of them"

Fix: :95, :303 (3 chars, 5.5e-4 each) and test_content_search.py:123 (JANE, 4 chars, ~9e-6). Leave alone: ~13 sites at ≥6 chars (≤2e-9) — same pattern, but not a defect at any rate that will ever be observed, and churning correct assertions costs review attention for no risk reduction. The pattern-propagation risk is answered instead by writing the rule into the existing convention comment at :49–58:

a substring assertion against ciphertext is safe iff the token is ≥ 6 characters.

test_off_by_default_stores_plaintext does not share the shape (deterministic equality), and neither do the many "DOE" not in … assertions elsewhere that check safe_text-scrubbed plaintext rather than ciphertext.

Why file rather than fix

The fix direction is the maintainer's call — decoded-bytes assertion, non-recoverability, or full-plaintext absence (assert ADT not in raw, deterministic because the fixture contains | and \r). Widening or deleting the substring check would drop the PHI-at-rest property it reaches for, which is the failure mode this repo already tracks.

Provenance

Diagnosed independently by two sessions and reconciled. The instrument-first framing, the ≥6 rule and the leave-the-rest-alone scoping come from the review by the session that owns PR #142; the fourth site (content_search.py:123) and the per-CI-run multiplier came back the other way. The item also records a confirmation by prediction: #142's re-run returned 25 passed with the prediction written beforehand, so a green re-run confirms a chance collision rather than resetting the question.

Coordination

• Number allocated via alloc.ps1; claimed FILED ONLY, NOT BUILT — the fix is unclaimed and free for any session to take.
• backlog_status_check.py passes: 269 items, each declaring exactly one status.
• Appends at EOF after #343, where #345/#346 also land. One file, so it rebases trivially in either direction.
• Auto-merge intentionally not armed — owner's call.

🤖 Generated with Claude Code

[wshallwshall]

Revised after review by three sessions (89b1d1ee)

Rates, recomputed exactly (two methods that agree — Fraction and -expm1(N*log1p(-x))):

Scope	Rate
one assertion, one leg	1 in 1,821
either assertion, one leg	1 in 911
either assertion, one full CI run	1 in 304

A float64 trap now recorded in the item, because it occurred inside the analysis of this very defect. The 14-char row was first written down as p = 0 (unreachable). It is 7.44e-24. The naive expression 1-(1-64**-14)**144 underflows to exactly 0.0 — 1 - 64**-14 is not representable in float64 and rounds to 1.0 — so it reports certainty, silently, with no warning, in a column of otherwise plausible values. In an item arguing that token length is the discriminator, at the one row where length was extreme enough to break the arithmetic. Whoever fixes these tests will re-derive these rates, so the trap and the idiom that survives it are written down.

The count, now stated with its basis. Three different numbers were quoted before anyone checked. Re-derived from the working tree: 7 lines / 8 clauses against at-rest ciphertext in test_store_encryption.py (:95, :232, :233, :251, :303×2, :304, :1062), of which 2 are unsafe — both DOE. Explicit exclusions are listed so it is not re-litigated: :905–908 / :927–928 assert against caplog/exception text, not ciphertext; :56 / :532 / :625 are full-plaintext; :512 is a fixed-nonce blob and : is not in the base64 alphabet.

#344 citation kept, with the discriminator its owner supplied, which is the sentence that stops the two being folded:

#344 is a fixed bound meeting variable latency; #347 is a deterministic property tested by a probabilistic proxy. This one would fire at exactly the same rate on an infinitely fast machine.

#346 added as the closer sibling — an assertion that passes for a reason unrelated to the property it claims to test. This one passes because random base64 usually lacks a 3-character run; #346's would have passed because nothing walks the imports.

Verification of this filing's own instruments

• Every probability computed by two independent methods that agree.
• Audit counts re-derived from the tree rather than quoted.
• backlog_status_check.py falsified against this item — a deliberately doubled banner makes it fail at BACKLOG.md:8429 naming #347, so its green is evidence it can see the item. The first probe attempt silently no-op'd on a cp1252 decode and "passed" — the same shape this item is about.
• ADR 0158 is referenced without a link: I guessed its filename from its title, checked, and was wrong; the real file exists only on PR docs(adr): land ADR 0158 — the silent-controls taxonomy the ledger already pointed at #145's branch, not on main.

Rebase order

PR #148 (#346) is armed and will land first. Both items append at EOF after #343 and are independent — take both, in either order; backlog_status_check.py should then report 270 items.