docs(asvs): correct a published attestation that was not true, and declare the scope boundary Rule 1 depends on - #157
Merged
Merged
Conversation
… attestation that was not true
Two public documents, one cause: the assessment method never wrote down where the
software ends and the substrate begins, so a platform-property requirement had no
principled home and churned between verdicts -- and separately, a published
attestation claimed more than the record supports.
ASVS-ASSESSMENT-METHOD.md
* Section 2 now states the boundary the "assessed as source" phrase already implied:
the subject is three SOFTWARE ARTIFACTS, not the host, hypervisor, CPU, firmware
or network. Where a requirement's VERB names a property of that substrate, rule 1
applies. This is ASVS's own principle and is now quoted verbatim from 0x03 at the
v5.0.0 tag rather than paraphrased -- "excludes requirements ... where
configuration is outside the application's responsibility", with the DNS example.
* Two guards, because a scope clause grows to swallow inconvenient cells: reporting
on a platform property is not providing it AND is not disqualifying either -- ask
what the verb requires, and of what.
* Explicitly declines the fork route. 0x03 does say organizations are "strongly
encouraged" to fork, but its omission examples are technology-not-used
(GraphQL/WebSockets/SOAP) and forking changes what you claim conformance TO.
Rule 1 against a declared scope is narrower and leaves the target unchanged.
* New section 2.1: what an out-of-scope cell does NOT buy. 4.0's "may still claim
full ASVS compliance" clause was DROPPED in 5.0, and OWASP retains normative
authority over which requirements sit at which level -- so an L3 claim omitting an
L3 requirement is non-conformant on OWASP's own terms.
* 11.7.1 moves from the worked example of a rule-3 fail to the worked example of a
rule-1 na, with the objection it has to answer stated rather than omitted.
Secure_Development_Standards.md
* "verified against OWASP ASVS 5.0 Level 3" -> "assessed against ASVS 5.0 using
Level 3 as the target -- an assessment IN PROGRESS". The old wording was
unsupportable three ways: the survey is incomplete and unread cells are explicitly
not passes; open requirements exist including at L2; and an L3 requirement is
scoped out, which under 5.0 does not preserve an L3 claim.
* The correction is stated in the document rather than made quietly, because the
claim was PUBLISHED. No figure is restated here, deliberately, so this page cannot
go stale against the record.
* Notes that no accredited L3 pathway exists to appeal to: OWASP certifies nobody,
and the one ASVS-based scheme accredits testing FIRMS at L1-L2.
…the four causes When 11.7.1 was scoped out under rule 1, the fail count went 3 -> 2 with ZERO lines of engine code changed. A reader comparing only the totals across those two days concludes a defect was fixed. Nothing was. The rationale sits on the cell and the boundary sits in section 2, but neither is visible to someone reading a number. New section 2.2 names the four causes a bucket total moves, of which exactly one is an improvement: a control shipped on by default (real); a cell read for the first time (the survey advanced -- unverified -> pass is a DISCOVERY, not a fix); a scope boundary stated (identical code, smaller denominator); a rule applied more carefully (more accurate, and these move DOWN too). Two obligations, both cheap: never report a total as a trend without naming which cause moved it, and when a count improves, state what would have had to happen for it to MEAN improvement and whether that happened -- the negative-control discipline applied to a metric. Written down because it cuts AGAINST us more often than for us. The survey is incomplete, so most future movement is cause 2, and the aggregate will get worse before it gets better as unverified cells resolve into real verdicts. That is the survey working, and it should be reported as such rather than defended against. Sits directly under 2.1 (an out-of-scope cell buys no Level 3 claim), because the two failure modes are the same one seen from opposite ends: 2.1 stops a scoped-out cell becoming a conformance claim, 2.2 stops it becoming a trend line. Prompted by the ASVS-11.7.1-summary session, whose objection -- that the rule-3 observations about the code stay literally true even after the verdict moves -- is what made the gap visible.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split out of #156 at the owner''s direction so the urgent correction lands without waiting on review of a bundled
featcommit. Two cherry-picked commits, docs only, no code.The problem, live on
mainright nowdocs/Secure_Development_Standards.md:367publishes:The ASVS survey is at 124/345 examined (35.9%), 221 cells
unverified. "Verified against ASVS 5.0 Level 3" is not supportable, and it is a published security attestation in a public repository — the only item in this thread with an audience outside the repo.The fix
The live sentence becomes "assessed against OWASP ASVS 5.0 using Level 3 as the target — an assessment in progress, not a completed verification."
The old wording is quoted, not erased, in a dated note that records why it was unsupportable: the survey is incomplete; open requirements exist including at Level 2, so narrowing the level would not have rescued it; and at least one Level 3 requirement is scoped out as a hosting-platform property, which under ASVS 5.0 does not preserve a Level 3 claim. Correcting a published claim by silently editing it would itself be a defect.
Why the scope boundary rides along
The rubric''s Rule 1 is "Does the requirement apply to this product on the declared scope (§2)?" — it delegates applicability to §2.
main''s §2 declares no platform-substrate boundary, so Rule 1 could not producenafor a CPU/firmware property. These commits add that boundary (§2 substrate text, §2.1 what an out-of-scope cell does not buy, §2.2 a count movement is not a posture movement), and update the 11.7.1 row tonaunder Rule 1 accordingly — matching the owner''s ruling.Declare the boundary, then apply the rule. Landing the verdict without the boundary would leave the public rubric justifying
failwhile the register saysna.Verification
verified against OWASPsurvives once, inside the correction blockquote quoting the old wording as wrong — not as a live claim. The live sentence at :367 is corrected.main: 2 files, 91 insertions, 2 deletions. No code, nofeat, no ADR/PHI/CONFIGURATION edits — those stay in docs(asvs): correct a false published attestation, declare the scope boundary, and gate a closed cell against silent re-scoring #156 for review.naunder Rule 1.Not included
The
featgate (scorecard.py+61, tests +83) and the ADR 0152 / PHI.md / CONFIGURATION.md edits remain in #156, unreviewed and unarmed.🤖 Generated with Claude Code