dp#51 deferrals + dp#60: meta opener walk, qagent returns, typed claim recovery, truncation guards, render fences (v7.51.0) - #65
Open
SSFSKIM wants to merge 41 commits into
Open
dp#51 deferrals + dp#60: meta opener walk, qagent returns, typed claim recovery, truncation guards, render fences (v7.51.0)#65SSFSKIM wants to merge 41 commits into
SSFSKIM wants to merge 41 commits into
Conversation
…09s, name-inference rung, fail-closed renderer)
…ammar, lift-first ordering, pr sixth field)
…p truncating marker-quoting bodies (#60)
…-write trigger); validate before external writes
board-body.sh's gh half carried the proven rightmost walk inline; it now
calls B.meta_match. board-migrate-gh.sh still inlined the PRE-FIX
META_RE.sub("") strip, keeping #60's truncation alive on the one-shot
migration path — it now strips through B.strip_meta.
Adds a migration-path regression drill: a linked issue whose prose quotes
a marker-shaped example keeps its prose (RED against the parent commit),
and a legacy note carrying an opening marker dies loudly rather than
minting a second marker inside the real block.
Review finding I1: write_body stripped twice — B.strip_meta(body) fed B.render_body, which strips again internally. Harmless while the first strip was leftmost (it had already eaten everything from the first marker on); once that strip became correct, the second one landed on prose. A marker-shaped example ENDING the base still satisfies META_RE's \s*$, so it was deleted as if it were the block — a #60-class truncation surviving on the one path where the write is one-shot. Splits compose_body(base, meta) out of render_body: prose the caller has already stripped, plus a rendered block, stripping nothing. render_body is now compose_body(strip_meta(body), meta), so every other caller is byte-identical. The migration composes through it. Drill grows the tail-example variant (RED against d62127b, one FAIL) and pins the refusal's own message rather than only its exit code. Also corrects the board-body.sh comment's causal clause: the offset invariant is why the splice must supply the separator, not why it normalizes.
…e inference, pr re-supply gh-mode review-dispatch never stamped a role, and board-answer's unrecorded-pre-park fallback had no QAGENT arm, so an answered reviewer park landed in in-progress — where the sweep's stale-reviewer arm retires the resumed worker out from under it. - _spawn_reviewer stamps role: QAGENT into the registry meta after the bind succeeds, via the file's own _stamp_meta helper (non-fatal). - board-answer's fallback is a three-way (ARCHITECT / QAGENT / else), with a legacy rung that infers QAGENT from a review-pr-* / review-epic-* registry name so the fix is not upgrade-gated. - the QAGENT arm re-supplies the ticket's recorded pr: as --pr, which is what board-transition's in-review gate requires; with no pr: meta it demotes to in-progress and warns rather than dying on the answer path. The probe heredoc moves into a function: bash 3.2 re-scans an inline "$(...)" at expansion time with a matcher that ignores the heredoc, so the body's prose apostrophes made the next double-quoted string with a space fail to parse. Reproducible against the parent commit; bash -n never saw it because -n does not expand.
The QAGENT stamp sat in _spawn_reviewer's shared tail, so the API claim
path ran it one line after board-bind had written the run bearer at a
forced 0600. _stamp_meta rewrote the file at the umask default, and the
api path's own stamp — which preserves whatever mode it finds — then made
the widened 0644 permanent, leaving the bearer world-readable at rest.
- gate the stamp on [ -z "${CLAIM_JOURNAL:-}" ] (gh mode only; the api
path stamps role itself, in the write that owns the bearer's mode)
- give _stamp_meta board-bind write_meta's discipline: 0600 when a
run_bearer is present, else the mode os.stat finds; unlink the stale
tmp, O_EXCL, chmod. This also closes the hole for the helper's existing
callers (retired_from on an api meta, closure_package).
Review follow-ups: restate the _probe_binding comment to the measured
mechanism (apostrophe parity in bash 3.2's command-substitution matcher,
not spaced strings), and drop a new board-answer assertion that was green
at base and so proved nothing.
…p uncharged, faults count (#51)
…r tick; reconcile honors suppression
…or one ticket are one attempt
…er; suites pin critical bindings non-empty
…the empty-INTEGRATION_REF render
…ences, roster and boundary checks
…d missing-placeholder check
…osmetics) Every id the A2 integration drills assert on ended what it was printed in, and `t`/`nt` match with `grep -qF`, which has no anchor: `owner=1` is a substring of `owner=12`, so a drill could pass — or an `nt` fail — on a run or ticket that merely prefixed the one it named. Closed once per emitter rather than per assertion: `owner_line` and `eol` move into drill-lib.sh (the four identical local copies go away), `row()` brackets its scalar fields the way it already bracketed its lists, and the lane's pick gets a `drew()` emitter. JSON ids are closed with the next token (`"ticketId":$T1,`), the live `grep -q` in test-escalation's poll with `[,}]`, and env-dump lines with `eol`'s terminator. The transcript-diff walk now records its argv twice — as executed and as written, `%T` unsubstituted — and transcript-compare.py compares the unsubstituted form. The two walks number their own tickets and only agreed by coincidence on a scratch board where both were #1; the alternative, a digit normalizer, would have erased step 6's deliberate literal 4242.
…re id wants Review I-1: the STRICT argv comparison is a capture-integrity check, not a behavioral fence — both walks iterate one shared STEPS array, so the argv is the drill's own input and no gh/api divergence is constructible. That was equally true before this task (the executed argv differed only by the ticket id, which is a way to fail falsely, not a way to catch drift), so the header, the walk's comment and the comparator's docstring now say so and hand the fence to the exit status, where it actually lives. M-1: the gh walk's id is asserted against the stub board that holds it — title and all — instead of against itself, mirroring the API half's ticket_state read. M-2: test-escalation's skip line closed with eol. M-3: the resume-feed membership assertion closed the same way, rather than a bare id against a one-per-line list. M-4: the executed argv is marked forensic. M-5 needs no change (the env-dump line is the only surface that carries BOARD_RUN_ID, and it is fully covered).
…ack: legacy nested markers + quadratic rescan)
The rightmost walk landed in Task 1 to beat the QUOTED shape (#60), where prose documents the block and a marker-shaped example sits above the real one. It regressed the mirror shape: a pre-grammar client could store a meta VALUE carrying a verbatim `<!-- board:meta`, so the real block's interior holds a second opener. Rightmost anchors on that nested one, and strip_meta cuts INSIDE the block — half of it is left behind as prose, and every rewrite re-emits it. The old leftmost code mis-parsed that body too, but it cut at the correct byte boundary. Neither end wins by position, so the interior decides. Candidates are the leftmost match's opener plus every later line-start opener that still precedes the (unique) closer; walking left to right, a candidate is real iff every line between it and the next candidate is a legal `key: value` block line. A quoted example's `-->`, a blank line or any prose disqualifies it; the last candidate wins by default. The nested block's forged keys still parse — that content is legacy corruption and unrecoverable — but the BOUNDARY is not, so the new pins are the strip boundary and rewrite stability: strip_meta lands on the outer opener, a rewrite leaves the prose byte-stable and collapses the body to one clean block, and that block is a fixed point. This also removes the quadratic rescan: the walk re-ran the end-anchored regex once per opener, 3.4s on a 4000-marker body, paid per issue by snapshot() against a server that accepts 1MB bodies. Two regex scans and linear slicing now — 0.004s on the same body, pinned by a loose-bound smoke.
…back: quoted legacy-nested shape)
dc89641 asked only whether the lines between two ADJACENT candidates were block-legal. Convergence review found the hole (spec v1.2.4): prose that QUOTES a legacy-nested example has nothing but that example's own `note:` entry between its outer and nested opener, so the gap reads legal and the quoted opener wins — parse leaks the example's keys and strip truncates the body from the example onward, losing every line after it. Since every META_RE match runs to the same closer, the interior that decides a candidate is its WHOLE interior: legal iff every line from its opener to that closer is a known-key `key: value` or a line-start nested marker. A blank line, prose, or an intermediate `-->` cannot sit in a block, so the answer is the FIRST opener standing after the LAST such line — one classification pass over the span, no per-candidate rescan, so the linear cost stands (0.004s on the 4000-marker body). Shape B still resolves to the outer opener; the quoted shapes, plain and nested, resolve to the real block. When no opener clears the last illegal line the block is noncanonical — an unknown key, a comment, hand spacing — and the last candidate wins, which is the old rightmost behavior and what board-body.sh's raw splice carries through untouched; that path is now pinned too. The degenerate-opener guard survives as the `pos + head <= close` room check, and META_OPENER_RE is gone — the pass finds its own openers.
…egment Two boundary regressions in c71a867's selector, both found by convergence review. The interior pass split on `\n` while parse_meta reads the block with str.splitlines(), which also honours U+2028, bare CR, \v, \f, \x1c-\x1e and \x85. A quoted example folded on any of those hides its own `-->` and the prose behind it inside one line that reads like a legal `note:` entry, so the quoted opener wins and the next meta write truncates the body. The pass now cuts on LINE_SEP_RE — the same boundaries, offsets kept — and the (f) separator roster gains a mirror in (j): (f) stops a value forging a key, (j) stops one hiding a disqualifying line. The fallback took the LAST candidate, which reopened shape B whenever a legacy block carried a nested marker AND an unknown key: the unknown key is not block-legal, so it fenced off every candidate, and opens[-1] is the nested opener — strip left the outer header behind as prose again. The fallback belongs to the segment the illegal line landed in, so each candidate now records the fence standing when it was seen and the fallback takes that segment's FIRST opener. A noncanonical block with no nesting still resolves to its own opener, so the splice path is unchanged. Shapes A, B and C, the degenerate trailing opener, and the linear cost all hold (4000-marker body at 0.006s).
The opener rule was hand-corrected three times, and twice the correction broke a shape the PREVIOUS formulation had handled — dc89641's adjacent-gap rule got the mixed legacy block right, and c71a867's whole-interior rule regressed it. Every case pin in the meta-grammar section is therefore a record of what someone thought to look for, not evidence of coverage. Three rounds of hand-reasoning is enough of a signal to stop hand-reasoning. The fuzz composes bodies from a component grammar — prose lines including colon-bearing and known-key-looking ones, quoted examples at column 0 and indented, with and without closers, quoting a legacy-nested block, folded onto one line by any of the eleven splitlines() separators — around exactly one intended trailing block, canonical or legacy-nested or noncanonical. The intended opener, prose and block bytes are recorded at generation time, and four properties are checked per body: the opener chosen, the strip boundary, rewrite idempotence, and the verbatim block bytes board-body.sh splices. It has teeth where the case pins do not: at the default seed and size it reports 2004, 182 and 954 divergences against the three superseded implementations, and zero against this one. 140k bodies across seven seeds are clean. Costs ~0.3s. Two regions are excluded and documented in the header, both because the bytes genuinely do not determine the answer: the v1.2.4 all-legal-no-closer ambiguity, and a block whose illegal interior line precedes its own nested marker. BOARD_SCRIPTS=<dir> points it at another implementation, which is how a candidate rewrite of the rule should be judged before it lands.
…k, qagent returns, typed claim recovery, truncation guards, render fences, drill anchoring)
… ledger+reset reach phase 4; indented openers; validate before labels)
board-answer's fail-open demotion for a recognized QAGENT return re-created the stranding it was meant to avoid: after the demote-and-resume the ticket sits in-progress, review-dispatch's stale-reviewer arm retires the resumed reviewer once idle, and the ticket is left in a lane with no PR binding and nobody bound. A reviewer-lane ticket with no pr: meta is an anomaly (the in-review entry gate stamps it), and anomalies pause rather than guess. The relay is now REFUSED: nonzero exit, the ticket stays parked at needs-human, and the message names the missing meta and the recovery (restore pr: in the body's board:meta block, then re-run --posted). The answers comment lands before the refusal, so nothing is lost. The IMPLEMENT/no-role fallback is unchanged. Spec v1.2.5 (1). The demotion pins are rewritten to pin the refusal.
Two halves of one invariant that stopped at the phase boundary. The ledger of tickets this tick already attempted a recovery for travelled no further than phase_resume, so after a replay FAULT left a ticket unowned an ordinary lane claim could pick that same ticket seconds later — a second attempt inside the tick the ledger exists to hold to one. It now rides to both dispatchers as BOARD_RESUMED_LEDGER, the way BOARD_SUPPRESS_DIR does, and a claim that yields a tick-ledgered ticket is released head-of-line exactly as a suppressed one is; the next tick serves it if still unowned. And only _resume_one reset the failed-cycle count, so a ticket the DISPATCHER recovered kept its stale .attempts-<tid> and a much later, unrelated fault escalated early. A successful bind now clears it. Spec v1.2.5 (2).
META_RE's opener is unanchored — it matches after leading spaces mid-line — but meta_match admitted a candidate only where the WHOLE line equalled the opener. A body with a column-zero QUOTED example and an INDENTED real trailing block therefore had no candidate for its real opener, fell back to the example, and the next meta write deleted the prose between them (the panel reproduced it). The candidate test is now lstrip() == the opener, with the offset pointing at the '<' so every byte consumer keeps the indent on the prose side. Indented quoted examples stay excluded by interior legality: their own closer line carries no colon, so it fences. The fuzzer grammar gains indented real blocks and is the acceptance: 711 divergences against the column-zero rule, 0 after. Header counts for the superseded rules re-derived under the extended grammar. Spec v1.2.5 (3).
…l write apply_state validates the merged meta ahead of its own label write, but board-transition.sh writes labels of its own first — ensure_labels, then the surface re-match's create+add — so a note the grammar refuses still tore the ticket: surface labels persisted, transition failed, and nothing rolls them back. The ticket then reads as a lane member it never entered. The merge-and-validate step is now check_meta_write, called by apply_state at its own top and by board-transition.sh ahead of every label call with the same extra it will hand apply_state — one implementation, two call sites. Spec v1.2.5 (4).
…t survives a crash Convergence review on the F2 fix, two holes in its mechanics. The ledger recorded only the tickets RECONCILIATION replayed for, so a ticket served by the ordinary needing-resume feed whose recovery then faulted or released was never fenced: unowned, spent, and claimable by phase 4 in the same tick. The feed loop now writes the ledger before the attempt, exactly as the replay arm does; a recovery that succeeds owns the ticket, so the record costs it nothing. And the failed-cycle reset was an unjournaled post-bind step: a dispatcher that died between the durable journal mark and the rm left a delivered recovery beside a stale count, and reconciliation skips completed journals forever. The reset now runs one line AHEAD of the marker write, which puts the whole remaining window inside the crash reconciliation already sees — the `repaired` arm (bind landed, marker lost) — and that arm clears the count too, carrying the journal's ticket for the purpose. Blanket-clearing on every completed journal was rejected: those records stand indefinitely, so a later unrelated fault would have its ladder reset out from under it. Spec v1.2.5 (2).
… acked delivery Second convergence round on the repaired arm. The seal (spawn_completed) is what makes a journal invisible to every later pass, so anything left until after it happens once or never — and the reset sat in the shell action that runs after the classifier already wrote the seal. A crash or a failed reset in that window left a durable recovery beside a stale counter with nothing able to revisit it: the same early-escalation bug, one window later. The reset now runs in the classifier immediately BEFORE the seal write, so the two are ordered the way the dispatcher orders them; both steps are idempotent, and a crash between them leaves the journal open for the next pass to redo both. The suppression directory reaches the classifier through a new _claim_suppress_dir callback, which replaces the _claim_attempts_clear one. And a bound journal whose control dir holds no ack was falling through to repaired whenever its writer was still alive — a peer between its bind and the ack it waits on. That delivery is not durable: if the ack never lands the stranded arm retires the worker and releases the run, so clearing the ladder there erases it for a delivery about to be undone. It now classifies as in-flight and is left entirely alone, journal and counter both. Spec v1.2.5 (2).
The reset ahead of the seal swallowed every OSError, so a removal that genuinely failed — a read-only or unmounted registry, a permission change — still sealed spawn_completed=true on top of a counter that was still standing. A sealed journal is invisible to every later pass, which is the early-escalation failure one door further along. FileNotFoundError alone is benign: no file is a finished reset. Any other removal error now skips the seal and reports the errno, leaving the journal open so the next pass retries the idempotent pair. Spec v1.2.5 (2).
os.remove answers ENOENT both when the counter is already gone (benign) and when the directory holding it cannot be seen at all — an operator BOARD_SUPPRESS_DIR whose volume unmounted, a path that moved. Sealing on the second reading loses the way swallowing EACCES did: the mount returns carrying the stale count and the journal that would have cleared it is closed forever. Reachability is what separates them, not the directory's own existence: the sweep creates that directory the first time it counts anything, so on a registry that has never had a failed cycle it legitimately does not exist — and reading THAT as a fault would refuse to seal every repair on every healthy fleet, forever. So the benign test is the directory or its parent being present; both gone is a path this process cannot see, where absence proves nothing and the seal is skipped. Pinned both ways: an unreachable path leaves the journal open, and a registry with no suppression directory repairs normally. Spec v1.2.5 (2).
Owner
Author
|
Post-open review round (user-invoked Five convergence rounds followed, hardening the reconcile-side reset: crash-durable (reset lands before the journal seal), ack-gated (a live-but-unacked handoff keeps its ladder), and honest about ENOENT (a missing counter ≠ a missing registry; the fixer correctly rejected a bare existence probe that would have broken every healthy fleet). Round 5's residual — a mountpoint-preserved unmount that remounts with a stale counter — is accepted, not fixed: inexpressible in default config, worst case is a liftable premature escalation, and both misclassification directions degrade gracefully. Spec at v1.2.5. All suites green at e6314d4: board-scripts, all board-api suites, both dispatch-claim suites, reviewing-prs, fuzz |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
The tail of the dp#51/dp#60 program: the five non-consumption deferrals recorded on #51 after PR #61 shipped, plus #60, the live gh-mode meta-truncation bug found during that work. Six bounded items, spec-driven (
docs/doperpowers/specs/2026-08-12-dp51-deferrals-dp60-design.md, v1.0 → v1.2.4 across two pre-implementation codex reviews and four flow-backs), executed subagent-driven over 11 plan tasks.Closes #60. Closes #51.
What shipped
meta_match/clean_metain_board.py: candidates are line-start openers; a candidate wins iff its whole interior (to the final closer) is block-legal underparse_meta's own line model (all 11splitlines()separators); segment-aware fallback preserves pre-grammar blocks for the raw splice. Values are normalized at the write (clean_meta), validated before any external write so a refusal can no longer tear a transition (label moved, meta dead).board-body.shandboard-migrate-gh.shride the shared helper — the migration was the last one-shot truncation path. The rule took four formulations to stabilize; the round was ended by a committed property fuzzer (140k generated legacy bodies, 7 seeds, zero divergences, self-validated by catching 2004/182/954 known divergences in the three superseded implementations — 0.3s per 5000 bodies, the judge for any future revision of this rule).in-review—role: QAGENTstamped at reviewer spawn (with a legacy rung inferring the role from the deterministicreview-pr-*/review-epic-*worker name), and the ticket's ownpr:meta re-supplied to the transition so the--prgate is satisfied on its own terms. Fix-wave catch:_stamp_metanow preserves file modes (a bookkeeping stamp was republishing 0600 run bearers at 0644).nonce-consumed/stale-resumemean the journal is obsolete: dropped uncharged. Real faults charge the existing 3-cycle ladder. One recovery attempt per ticket per tick (tick ledger, read by both the feed and the replay arm), phase order now lift → reconcile → feed, reconcile honors suppression — closing both double-charge and unliftable-suppression modes measured in review./ticketsread can no longer lift a suppression (absent row = unknown, mirroring the write-side "an empty read is not a state" doctrine), and the outward half is pinned as arkho#9:/queue/decisions+/ticketsstay unbounded until a paged envelope exists.Verification
test-board-scripts.sh501, board-api 12/12, reviewing-prs three suites (dispatch 308, entrypoint, parity 94), full runner PASSED with the 8 integration drills against the live service,lint-shell.shclean.Env note for re-runners: the integration tier needs
ARKHO_DIR=/Users/new/Developer/GitHub/arkho— thearkho-a1-board-servicecheckout is a stale branch and produces 32 false failures.Version: 7.51.0 (main's 7.50.1 merged in).