docs(badges): add BADGE-CRITERIA-SPEC — step 1 of #446, criteria only - #734
Merged
Conversation
#446 asks for the readiness-badge work in a strict order: "Do not build a cross-project evaluation engine first. In order: 1. Write the criteria/axes spec 2. A self-assessment badge for one repo 3. Only then generalise." This is step 1 and nothing else. It defines what a readiness badge asserts, what evidence must back the assertion, how confidence is expressed, what withdraws the badge, and how a self-assessment becomes verified. It builds no engine, evaluates no repository, and renders no badge. The document sits across the four existing CRG-family axes — CRG (components), TRG (toolchain), ARG (adoption surface), FRG (typing-discipline formalisation) — all of which already share the ordinal scale X(0) F(1) E(2) D(3) C(4) B(5) A(6). It adds no new scale and no composite score: aggregation is worst-of, never averaging, which is what keeps a project-level badge honest under CRG Principle 1 ("assess components, not projects"). A mean would let one A-grade component conceal an X. Drafted 2026-08-28 against commit 4e74daa; reviewed and corrected today against 516bd72, 38 commits later. The review checked cited *content*, not just cited paths, on the principle that a file can survive while the field a citation rests on is renamed away. All twelve honesty rules kept their grounding; the audit is recorded in the Provenance section. Four corrections were applied to the draft before landing: - It referenced a machine-readable twin and a worked example as though both shipped alongside it. Neither does. The twin is held behind the .a2ml -> .deed conversion, which another agent owns, so it is not born needing a rename. The worked example is step 2 and waits on the pilot-subject confirm still open as item 5 of #709. - It listed the CRG version number as unresolved, citing v2.0 and v2.2 without choosing. Resolved: three files say v2.0 and only templates/CRG-PROFILE-TEMPLATE.adoc says v2.2, for which no spec exists. That is a template defect, now filed as #733. This document cites v2.0. - It asserted "the owner moved the pilot to standards". That was false. #446 puts three options on the table and none is ruled. The correction is recorded in the text rather than quietly deleted, because the working filename of the step-2 draft could otherwise be misread as a ruling. - Q4 (a2ml dialect) is reclassified from an open BC question to DEFERRED. It only ever arose because a twin was expected to land here; it now belongs to whoever owns the .deed conversion. BC takes no position. Lands as DRAFT v0.1. Q1 (one badge or four), Q2 (who may issue `verified`) and Q3 (whether the measured floor is too harsh at launch) need owner rulings, and are flagged on #446 rather than opened as a new decisions issue, so that they batch with the others in #709. Touches no .a2ml or .deed file, no A2ML manifest content, and no K9 contract. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
|
Warning Review limit reachedNext included review available in 40 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Team Run ID: 📒 Files selected for processing (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands the first of the three steps #446 lays out, and deliberately stops there.
This is step 1 only. It builds no engine, evaluates no repository, and renders no badge.
What it is
docs/BADGE-CRITERIA-SPEC.adoc— a criteria specification sitting across the four axes that already exist (CRG components, TRG toolchain, ARG adoption surface, FRG typing-discipline formalisation), all of which already share the ordinal scaleX(0) F(1) E(2) D(3) C(4) B(5) A(6).It defines what a badge asserts, what evidence must back it, how confidence is expressed, what withdraws it, and how a self-assessment becomes verified.
It adds no new scale and no composite score. Aggregation is worst-of, never averaging — the rule that keeps a project-level badge honest under CRG Principle 1, "assess components, not projects". A mean would let one A-grade component conceal an X, which is the exact failure
crg-overclaim-detector.a2mlexists to catch.Provenance, and why the review was not just a path check
Drafted 2026-08-28 against
4e74daad. Reviewed today against516bd724— 38 commits later.Path existence is a weak check: a file can survive while the field a citation rests on is renamed away. So the review verified cited content.
scorecard.schema.jsonevidence,check,aspirational,systemall presentscripts/build-scorecards.shrun_verify()at :184, invoked :436crg-overclaim-detector.a2mlhonest D > dishonest B.machine_readable/STATE.a2ml@scannerglob unchanged*-readiness-grades/*.adocARG <= TRG,ARG-A requires FRG >= BunchangedNo honesty rule lost its grounding. One discrepancy surfaced — see below.
Four corrections applied before landing
badge-criteria.a2mland a worked example as though both shipped with it. Neither does. The machine-readable twin is held behind the.a2ml→.deedconversion, which another agent owns — landing it now would only mean it is born needing a rename. The worked example is step 2.COMPONENT-READINESS-GRADES.adoc:3, its.a2ml:11,crg-overclaim-detector.a2ml:242); onlytemplates/CRG-PROFILE-TEMPLATE.adoc:17,25says v2.2, and no v2.2 spec exists in this repository. That is a template defect, not a spec ambiguity → docs(crg): CRG-PROFILE-TEMPLATE claims conformance to v2.2, but the spec declares v2.0 #733. This document cites v2.0. (Worth noting the shape: the template overclaims the spec version it conforms to — the same failure CRG's own overclaim detector polices, one level up.)standards". That is not true. Proposal: an honest, adoptable readiness/insight badge framework — grade our own work, and challenge the evaluators #446 offers three unruled options (pilot =boj-server, pilot =standards, defer) and item 5 of Owner decision batch 2026-08-31: 7 decisions from the triage wave, one reply resolves all #709 still holds the confirm. The correction is written into the text rather than quietly deleted, because the step-2 draft's working filename (PILOT-standards.adoc) could otherwise be misread as a ruling..deedconversion. BC takes no position and needs no ruling on it.Also updated: the Glama badge was removed from both
boj-serverREADMEs on 08-31 via hyperpolymath/boj-server#324, after this draft was written. The draft said that question was untouched; it is now partly closed. Listing links,glama.jsonanddocs/glama/remain, pending the same #709 item.⚖️ Owner rulings needed — Q1–Q3
Flagged on #446 rather than opened as a new decisions issue, so they batch with #709:
verified? Draft allowsestate-auditif the checks ran in CI. The alternative reserves it for external parties, which means the estate has no verified badges for the foreseeable future.floor = X, because almost nothing has per-grade executable checks yet. Accurate, but a badge readingCRG X–Con every repo may be noise rather than signal. Option: gate rendering onfloor >= D.Lands as DRAFT v0.1, unratified, so none of these blocks the merge.
Verification
asciidoctor --failure-level=WARN→ rc 0. (The defaultFATALhides dropped table cells; the document has 10 tables, so WARN is load-bearing here. The Provenance table was independently cell-counted: 3 header + 15 body, none dropped.)<<xref>>constructs — the auto-generated-section-id WARN trap that bit feat(gates): document manifest depth addressing; add the free prefix-vs-depth check #731 cannot occur.docs/SEAMS-SPEC.adocsibling convention; SPDXCC-BY-SA-4.0, as required for docs.docs/*-SPEC.adocis not indexed by the registry (confirmed againsthypatia-rules/registry-staleness.a2mlandscripts/registry-readiness.sh), so this needs no registry entry and will not trip "Registry + topology in sync".Explicitly not touched
.a2ml/.deedextensions ·.deedgrammar · A2ML manifest contents · K9 contracts and their validators.🤖 Generated with Claude Code