Add conformance/negative/: vectors, failure codes, and runner (closes the suite gap for the enforcement layer) - #30
Open
avp9-nexus wants to merge 1 commit into
Conversation
…AI-Security-Project#29, GenAI-Security-Project#19) Implements the negative conformance proposal of GenAI-Security-Project#29 and begins closing the conformance-suite gap of GenAI-Security-Project#19. - 18 framework-agnostic vectors across the six categories of GenAI-Security-Project#29 (12 must-reject, 6 positive controls) - a proposed failure code enumeration whose backbone separates REJECT (measured and found wrong) from UNMEASURABLE (could not be measured; must still refuse, under a distinct code) - runner.py fails on verdict, code, missing reason substring, or any category lacking its positive control - reference_adapter.py: a minimal passing implementation, so the suite is executable end to end out of the box - category 3 covers expiry only; revocation vectors are a declared gap, per the contributing implementation's documented scope Each failure code was produced by a real refusal in a production enforcement layer before it was named here. Signed-off-by: avp9-nexus <avp9pro@gmail.com>
avp9-nexus
requested review from
GangGreenTemperTatum,
afogel,
almogbhl,
bar-capsule,
fewdisc,
mamicidal,
rocklambros,
sclintonowasp and
stefanoamorelli
as code owners
August 17, 2026 20:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the proposal of #29 and begins closing the conformance-suite gap of #19.
What is in here:
Category 3 (expired or revoked mandates) is covered for expiry only. Revocation vectors are a declared gap: the contributing implementation has no revocation mechanism, by documented decision, and vectors for a path never exercised in production would be design fiction. Contributions from implementations that exercise revocation are the way to close it.
Every failure code in the enumeration was produced by a real refusal in a production enforcement layer before it was named here. Vector wording is deliberately implementation-neutral; naming, directory layout and the enumeration itself are all open to review.
Type of change
(Nothing under specification/ or docs/spec/ is touched; the new directory sits at conformance/negative/.)
Checklist
git commit -s(required by the DCO)uv run mkdocs build --strictpasses (not run locally; conformance/ sits outside the docs tree, happy to add the check if maintainers want it wired into CI)Security
(It adds test vectors and a runner; it changes no normative text and no schema.)