Skip to content

♻️ refactor(forge): extract the conformance suite into an importable package with an explicit observation surface (E10-S01) - #98

Merged
konih merged 2 commits into
mainfrom
lane/e10-s01-conformance-suite
Aug 25, 2026
Merged

♻️ refactor(forge): extract the conformance suite into an importable package with an explicit observation surface (E10-S01)#98
konih merged 2 commits into
mainfrom
lane/e10-s01-conformance-suite

Conversation

@konih

@konih konih commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Implements E10-S01 ([autonomous], depends on S00 which merged as e2e7691). Decision row: D-165.

The problem

internal/forge/conformance was 1,155 lines across four _test.go files. Go cannot
import _test.go, so the only way to conformance-test a second adapter was to COPY
the cases — which guarantees drift, and is why D-084's github-deferred rows were
unflippable.

What landed

Case bodies now live in ordinary Go. An adapter supplies a Factory and calls
RunSuite; nothing is copied.

  • suite.goFactory, Case, Cases(), RunSuite
  • observe.go — the Observer / Fixture surfaces (REQ-E10-S01-04)
  • portcount.go — port-level write counting
  • replay.go, fixtures.go, cases_replay.go, ids.go — moved case + fixture bodies
  • catalog.go — strict-decoding catalog loader (REQ-E10-S01-03)
  • tb.go — the TB seam that makes "can this case fail?" answerable

The five exported test names are unchanged. The DoD forbids renaming a case, and
exitgate_test.go's l1CatalogTests pins these exact names — renaming them would
have silently unhooked the E7 exit gate.

What the extraction found

The two SHA-guard cases had never run against GitLab, despite their catalog rows
saying forge: gitlab. The httptest harness served discussions and notes only. It now
serves MR/branch/approve/merge, so they do.

A conditional assertion that could not fail. The "summary updated in place" check
was guarded behind if ff, ok := f.(*fake.Forge); ok — false on GitLab, so on that
backend it proved nothing while still reporting PASS. It is now unconditional.

Merge ATTEMPTS were never distinguished from merges PERFORMED. Both SHA-guard cases
asserted only len(f.Merges) == 0, which is equally true of a backend that skipped the
pre-check and let the atomic CAS refuse. Split, the two cases now disagree — 0 attempts
when the pre-check fails closed before any mutation, 1 when the CAS is reached and
refuses — which is the guarantee each was supposed to prove.

Counting is done by a port-level decorator, not per adapter: GitLab's MergeCAS
re-reads heads and refuses client-side, so a moved head produces ZERO merge HTTP
requests. A harness counting PUTs would report 0 attempts where the port was called once.

Three gates keep it honest

A conformance suite that has stopped proving its property still reports PASS, so:

  1. TestCatalogMatchesExecutedCases — the catalog is compared against OBSERVED
    EXECUTION (RunSuite returns what it dispatched), not against a name list. A gate
    keyed on test-function names stays green for a case unhooked from the runner — a
    predicate over TEXT standing in for a STRUCTURAL property, the species behind D-164.
  2. TestEveryCaseCanFail — every case must go red against a sabotaged backend, on
    every adapter, with a positive control that it still passes against a conforming one.
  3. TestEveryObservationIsLoadBearing — corrupts each value a case reads, ONE AT A
    TIME, and requires each to flip the verdict.

Gate 3 found two real weaknesses in code that had already passed my own review:
a NotesUpdated() > 0 check no corruption could flip (now an exact count), and a
blind spot in my own mutation operator (a uniform offset is invisible to a before/after
delta — corruption is now applied to the first read only).

Scope held deliberately

Factory returns forge.Forge, not forge.RunPort. RunPort is E10-S02's
deliverable and S02 is LGTM-gated; declaring it in this autonomous lane would
smuggle an LGTM-gated core-contract change past backlog.md:557. S02 depends on S01
precisely so the port change lands against an executable suite.

Cases take pins from Fixture.Pins() rather than literals, because GitLab synthesises
its merge-result digest — a literal digest is why those cases were fake-only.
Collapsing that digest is E10-S03.

Verification

  • task check EXIT=0 (all 21 stages), coverage 91.1% (floor 91%)
  • Assertions went from 9 test functions to 103 passing subtests; all five
    catalogued cases now run on both adapters
  • Mutation controls run and confirmed red: case unhooked from dispatch → catalog gate
    reds; merge attempts collapsed onto performed → REQ-04 proof reds on both backends;
    summary re-created instead of updated → rerun-idempotence reds

@konih
konih merged commit 65f1393 into main Aug 25, 2026
7 checks passed
@konih
konih deleted the lane/e10-s01-conformance-suite branch August 25, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant