diff --git a/docs/ASVS-ASSESSMENT-METHOD.md b/docs/ASVS-ASSESSMENT-METHOD.md index 13dc557..7c0e2bd 100644 --- a/docs/ASVS-ASSESSMENT-METHOD.md +++ b/docs/ASVS-ASSESSMENT-METHOD.md @@ -62,7 +62,7 @@ These are the real disputes. They are here so the next assessor reaches the same |---|---|---| | **5.4.3** | `fail` | Rule 3. A scan *hook* exists but its only shipped implementation is `_no_scan`, and there is **no configuration key at all** — an operator must **author** the scanner. Supplying a control is not configuring one. | | **15.2.5** | `partial` | Rule 5, **not** rule 3. `[sandbox].mode` ships `off`, but `subprocess` mode is real and was verified by executing it. A working control that ships off. | -| **11.7.1** | `fail` | Rule 3. The read-out is all-`None` off Linux and the only refusing branch keys on a **TOML declaration**, never on the measured property. Zero percent of the verb is satisfied by shipped code. | +| **11.7.1** | `na` | **Rule 1 — the hardest call in this table, and it moved.** The verb is *"full memory encryption is in use"*: a property of the **CPU, firmware and hypervisor**, not of the three software artifacts in §2. Outside the declared scope, so rule 1 fires before rule 3 is ever reached. **The objection this has to answer, because it is a good one:** ADR 0152's rungs 1–2 *do* ship in-engine, so the engine is not silent on this cell. But that code **reports on and gates against** the platform property — it never provides it. Reporting is not implementing (§2's first guard). ⚠️ Previously scored `fail` under rule 3 and cited here as the worked example of one. That reading was not wrong on its own terms; it answered *"does code implement the verb"* without first asking *"is the verb's subject in scope"*, and rule 1 runs first. **This cell has moved four times in eighteen days — it is CLOSED by owner decision (2026-08-02); do not re-derive it.** ⛔ It buys **no** Level 3 claim: see §2.1. | | **3.7.3** | `fail` | Rule 3. One off-site navigation, a bare 303, no interstitial and no cancel. `oidc_enabled=False` removes the **trigger**, not a control. | | **10.5.5** | `na` | Rule 1. The requirement is conditional — *"**when using** OIDC back-channel logout"* — and the precondition is false and unreachable by configuration. **Building it would create applicability.** | | **12.2.2** | `na` | Rule 1. No external-facing services on the declared scope. *Also a scoping error worth remembering: this row spent months scoring 12.3.1's verb.* | @@ -86,6 +86,86 @@ Per ASVS's guidance, stated as what **is** included rather than what is excluded **Included:** the MessageFoundry engine, the web console, and the IDE extension, assessed as source, at a named commit, against **all 345 ASVS 5.0.0 requirements** at **Level 3**. +**The boundary that phrase implies, stated explicitly because it decides verdicts.** The subject of +this assessment is those three **software artifacts**. It is not the host, the hypervisor, the CPU, the +firmware, or the network the operator deploys onto. Where a requirement's **verb** names a property of +that substrate rather than of the software, the requirement is outside the declared scope and takes +`na` under rule 1 — with the rationale written, as always. + +**This is ASVS's own principle, not a local invention.** Verbatim from `0x03-What-is-the-ASVS.md` at the +`v5.0.0` tag: + +> "Conversely, ASVS generally excludes requirements that are not directly relevant to the application or +> **where configuration is outside the application's responsibility**. For example, DNS issues are +> typically managed by a separate team or function." + +⚠️ **Do not over-read that, and do not reach for the fork clause to do this job.** The same chapter says +organizations are *"strongly encouraged to create an organization- or domain-specific fork that adjusts +requirements"* — but its worked examples of omission are **technology-not-used** (*"omitting irrelevant +sections (e.g., GraphQL, WebSockets, SOAP, if unused)"*), which is the functionality-based shape, and +**forking changes what you are claiming conformance TO**: your tailored ASVS, not stock ASVS 5.0. This +project does **not** fork. It applies rule 1 against a positively-declared scope, which is narrower, +cheaper to defend, and leaves the conformance target unchanged. + +Two guards on that boundary, because it is exactly the kind of clause that grows to swallow +inconvenient cells: + +* **Shipping code that *reports on* a platform property is not the same as *providing* it, and neither + direction is decisive on its own.** A cell does not become in-scope merely because the engine + observes the substrate; nor does it leave scope merely because the substrate is involved. Ask what + the **verb** requires to be true, and of what. +* ⛔ **This boundary does not shrink the conformance claim's denominator.** A requirement excluded here + is still a requirement OWASP assigns to Level 3. See §2.1. + +### 2.1 What an out-of-scope cell does NOT buy + +**It does not preserve an unqualified Level 3 claim.** Two facts, both from ASVS 5.0.0 itself: + +* **4.0's clause that an organization excluding requirements "may still claim full ASVS compliance" + was DROPPED in 5.0.** The 5.0 text says only that non-applicability must be noted in the report. + There is no longer any standard text saying a documented exclusion preserves a compliance claim, and + a rationale that cites the older wording is citing a superseded standard. +* **OWASP does not certify anyone — but it does retain normative authority over the requirement SET.** + It assigns each requirement to a level. So a Level 3 claim that silently omits a requirement OWASP + places at Level 3 is non-conformant **on OWASP's own terms**, regardless of how well-argued the + exclusion is. + +**Therefore:** scoping a cell out is a statement about *what was assessed*, never a statement that the +level was achieved anyway. Any published attestation must say which requirements were excluded, or say +something weaker than "verified at Level 3". Writing `na` in the record and "Level 3 verified" in a +brochure is the failure mode this section exists to prevent. + +### 2.2 A count movement is not a posture movement — read the counts accordingly + +**The single most misreadable thing this method produces is a change in the bucket totals.** Counts move +for four different reasons, and only one of them means the software got better: + +| The count moved because… | Did the posture improve? | What actually happened | +|---|---|---| +| A control was **built or turned on by default** | **Yes** | The verb is now satisfied by shipped code | +| A cell was **read for the first time** (`unverified` → anything) | **No** | The survey advanced. A cell moving `unverified` → `pass` is a *discovery*, not an improvement | +| A **scope boundary was stated** (→ `na`) | **No** | The requirement left the denominator. Identical code, smaller question | +| A **rule was applied more carefully** (re-grade in either direction) | **No** | The assessment got more accurate. Some of these move *down* | + +**The worked example, and it is recent.** On 2026-08-02 the fail count went **3 → 2** when 11.7.1 was +scoped out under rule 1. **Zero lines of engine code changed.** A reader comparing only the fail counts +across those two days would conclude a defect was fixed; nothing was. The rationale is on the cell and +the boundary is in §2, but neither is visible to someone reading a total. + +**Two obligations follow, and they are cheap:** + +1. **Never report a bucket total as a trend without naming which of the four causes moved it.** "Fails + went from 3 to 2" is not a finding. "One cell was scoped out; no code changed" is. +2. **When a count improves, state what would have had to happen for it to mean an improvement, and + whether that happened.** This is the same discipline as a negative control: a number that can only + go one way is not measuring anything. + +⚠️ **This cuts against us more often than for us, which is why it is written down.** The survey is +incomplete, so most future movement will come from cause 2 — cells being read for the first time — and +the aggregate will get *worse* before it gets better as `unverified` cells resolve into real verdicts. +**That is the survey working, not a regression**, and it should be reported as such rather than +defended against. + **The configuration assessed** — one posture, not a matrix: > On-premises single hospital · private network, never internet-facing · SQL Server store · operator diff --git a/docs/Secure_Development_Standards.md b/docs/Secure_Development_Standards.md index 5658f06..9302bed 100644 --- a/docs/Secure_Development_Standards.md +++ b/docs/Secure_Development_Standards.md @@ -364,7 +364,16 @@ The project maintains a current evidence set so any claim is backed: - **Per-project applicability profile** (Appendix A and onward). - A **claims register** recording each published claim, its wording, and the evidence behind it. -**Attestation posture.** The software is self-attested as NIST SSDF–aligned, tested per NIST SP 800-115, verified against OWASP ASVS 5.0 Level 3, and built to support HIPAA-compliant deployment (controls mapped to NIST SP 800-66 Rev. 2). Third-party validation of the SSDF attestation and the ASVS 5.0 Level 3 assessment raises the weight of these claims. **Attestations are published with releases** so adopters can rely on them; each adopter still performs its own deployment risk assessment (§7.3). None of these is a NIST certificate; displayable certificates (SOC 2, ISO 27001, HITRUST) are a separate, organization-level track. +**Attestation posture.** The software is self-attested as NIST SSDF–aligned, tested per NIST SP 800-115, **assessed against** OWASP ASVS 5.0 **using Level 3 as the target** — an assessment **in progress**, not a completed verification — and built to support HIPAA-compliant deployment (controls mapped to NIST SP 800-66 Rev. 2). Third-party validation of the SSDF attestation and of the ASVS assessment raises the weight of these claims. + +> **The ASVS wording above changed on 2026-08-02, and the previous wording was wrong.** It read *"verified against OWASP ASVS 5.0 Level 3."* Three facts make that unsupportable, and they are stated here rather than quietly corrected because the claim was **published**: +> 1. **The survey is incomplete.** A minority of the 345 requirements have been read against the ASVS text at a known commit; the remainder are recorded as *unverified*, which is explicitly **not** a pass. "Verified" asserted a completed verification that had not occurred. +> 2. **Open requirements exist**, including at Level 2 — so the claim was not rescuable by narrowing it to a lower level. +> 3. **At least one Level 3 requirement is scoped out** as a hosting-platform property outside the assessed software. Under ASVS 5.0 that does **not** preserve a Level 3 claim: 4.0's "may still claim full ASVS compliance" clause was **dropped** in 5.0, and OWASP retains normative authority over which requirements sit at which level. A Level 3 claim omitting a Level 3 requirement is non-conformant on OWASP's own terms. +> +> **No accredited Level 3 pathway exists to appeal to.** OWASP certifies no vendor or software and states that any trust mark claiming ASVS compliance is not officially endorsed by it; the one ASVS-based accreditation scheme accredits testing *firms*, and its published scope is Levels 1–2. So "self-attested" is the only honest register available here, and this section now uses it. +> +> Current status is held in the project's private assessment record, which is the count of record; **no figure is restated here**, deliberately, so this page cannot go stale against it. **Attestations are published with releases** so adopters can rely on them; each adopter still performs its own deployment risk assessment (§7.3). None of these is a NIST certificate; displayable certificates (SOC 2, ISO 27001, HITRUST) are a separate, organization-level track. ---