You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat: adaptable-code rules in prompts, pipeline and regen gate - #12261
PR 2 of the adaptable-code rollout (design, "Rollout" item 2; PR 1 was feat(core): adaptability checker for Python implementations #12244): the Data block convention in the generator and in all 15 library prompts, a Python self-check (Step 3c) in generation and repair, a record-only smoke run and predecessor render in impl-generate.yml and impl-repair.yml, a record-only adaptability check in impl-review.yml, and the identity-based adapt route plus the adapt regression in the regen gate.
Score-neutral: the review prompt and the rubric are unchanged, every new step is record-only (never a raw exit 1), and the review-free path stays off behind the repository variable ADAPT_REVIEW_FREE.
Checks are Python only (the 8 Python libraries). R, Julia and JavaScript get the convention in their first-generation prompts and keep today's regeneration mindset; every record carries language, so their checkers plug in later without format changes (the "all 15 libraries" goal).
Plan
Design: docs/concepts/adaptable-code.md ("The convention", "Other libraries", "The checks", "Pipeline changes", "The adapt route", "Rollout" item 2). Four parts were built against one interface contract and merged here with --no-ff; the merge commits keep each part's history.
A, prompts (eb3d074fa)
New prompts/adaptable-code.md holds the convention once: the six zones, the # Data/# Plot markers and # role: grammar, the plot-code rules, the data shapes, the bar-grouped example, and the R, Julia and JavaScript forms. plot-generator.md and Step 1 of impl-generate-claude.md reference it.
plot-generator.md: the template, Structure and Data Generation Strategy follow the six zones; Data storytelling gets rule-chosen callout targets; bbox_inches='tight' is gone. default-style-guide.md: "list all eight hues and use the first n", no bbox_inches='tight'.
All 15 library prompts get a "Data block" section; the [:N] palette slices are gone from matplotlib and seaborn; Chart.js no longer recycles colours with i % length.
impl-generate-claude.md: Step 3c (python -m core.smoke_run selfcheck, at most three attempts, never blocking), the commit line Adaptability self-check: pass|fail (...), attempt N, the five regen-mindset amendments phrased "For Python (LANGUAGE=python)", a "Converting a Python file" bullet that aims at an identical render. impl-repair-claude.md: self-check lines are real defects; Step 5b reruns the self-check.
B, generation side (416065feb, 3425ac00b)
core/smoke_run.py (run, pair, selfcheck): one process per perturbed table and theme, process groups killed on timeout, errors mapped back to the original line, a stale-label hook for matplotlib and seaborn, and pair, which renders the new source and then the predecessor in one environment and compares the raw PNGs.
core/perturbation.py: missing_number (NaN in every float64 column, binding) and missing_nullable (<NA> in Int64, boolean and string columns, advisory).
impl-generate.yml "Adaptability smoke run (record-only)" and the private adapt/ staging in "Upload to GCS Staging"; impl-repair.yml reruns the smoke run and replaces or removes adapt/new-smoke.json.
C, review side (4b710e2c0)
impl-review.yml "Adaptability check (record-only)": the workflow-ref checker mirror on the PR file and, on a regeneration, its predecessor; automation/scripts/adapt_record.py builds /tmp/anyplot-adapt-{new,prev}.json; regen_gate.py adapt-route before the review; the review is skipped only when ADAPT_REVIEW_FREE is true and every condition holds.
Review-free wiring, the regen:adapt count label, carried metadata (review.carried), and the staging download narrowed to plot-* so adapt/ never reaches the blind reviewer. impl-merge.yml promotes only plot-*. regen_writeback.py removes review.carried on a write-back.
D, gate (0438a60a4)
adapt_assess with the contract's 20 ordered conditions; adapt_identity replaces the keeps no_visible_improvement, no_defect_improvement and below_tolerance; --review-skipped merges a counted route before the score checks; adapt_regression turns any merge into keep. The record gains an adapt block (RECORD_VERSION stays 1), the outputs and notice gain adapt_counted and adapt_why. The gate stays stdlib-only and reads identity from impl-generate's pair.json.
Integration (db865896b)
impl-review.yml gets one unconditional "Set up Python" step (python-version-file: .python-version) before the first regen-tools call; part C's mirror-only step is gone and every tool runs on plain python3. Since chore: move to Python 3.14, with the sandbox fixes it needs #12247 the gate, review_provenance.py and the mirror use PEP 758 except A, B:, and the runner image's Python is not pinned.
The Step 3c log line gains smoke_why, so the adapt record counts a skipped first attempt as a pass only for a shape skip, as selfcheck's own exit 0 does.
Docs: the pipeline diagram and a table of the adaptability steps (docs/workflows/overview.md), adapt_record.py (automation/scripts/README.md), core/smoke_run.py (docs/reference/repository.md).
Review fixes (05ec4a82b, 602dd40c5, 4fd7f7c57)
Copilot: an adapt record without valid static and smoke blocks is record_invalid (fail closed); the Chart.js and ECharts Data block examples run as written; the "ninth group fails loudly" rule names the idioms it holds for (lookup by position or group) in prompts/adaptable-code.md and prompts/default-style-guide.md.
Gemini and part D's own note: the route needs a pair.json rendered from the very sources the records checked (code_sha256 on both sides), else the new reason token pair_stale.
Second opinion (Gemini 4 Argon + Fable, 1e75be40a..a423e7b69)
The gate rejects a record whose status does not follow from its static and smoke blocks (record_invalid), and pair.json must name its library.
A predecessor whose static check passed against a new source whose static check fails is an adapt regression even without a fresh predecessor smoke run.
decide --adapt-route-held, the record field adapt.route_pre_review, the output adapt_overruled and the label regen:adapt-overruled make the review's overrules of the route countable (O1).
The post-score re-dispatch also covers the review-free path, with the stored score.
adapt/ is staged with gsutil rsync -r so new/ and prev/ keep their names.
execution_env drops GITHUB_*, ACTIONS_*, RUNNER_* and GH_* before running an implementation.
An empty perturbation set has its own token no_perturbations:<shape>, read the same way by selfcheck and the record.
One palette name, IMPRINT_PALETTE, in the convention and its example; the IMPRINT alias is stated once.
Known gap, unchanged: the early verdict label lands before "Update metadata and implementation header"; a metadata push failure after it is left to the watchdog (pre-existing on the normal path).
Test plan
uv run ruff check . and uv run ruff format --check .: clean.
uv run --extra typecheck --extra agents mypy api core agents: no issues in 151 source files.
uv run pytest tests/unit tests/integration -n 4: 13158 passed, 3 skipped (seaborn is not installed locally). That is 443 more tests than main: test_smoke_run.py 34, test_adapt_generate_steps.py 36, test_adapt_review_steps.py 85, test_adapt_record.py 66, plus additions to test_regen_gate.py, test_perturbation.py, test_regen_writeback.py and test_prompts.py.
uv run python -m tools.changelog check --base origin/main: passes.
actionlint 1.7.12 with shellcheck 0.11.0 on the four changed workflows: 87 findings on main and on this branch, the same set (only line offsets in impl-generate's upload step moved).
End to end on one real pair outside the repository: bar-grouped matplotlib, a predecessor with a three-hue palette (static rule 5 fails) against the converted fixture. Part B's run, pair and selfcheck fed part C's mirror and adapt_record.py, which fed part D's adapt-route and decide --review-skipped: route=true, merge adapt_identity, identity 0.0000 in both themes, a valid gate record. The new side's smoke run passed all 7 binding tables; missing_nullable failed as advisory (float() argument ... 'NAType' in ax.bar).
After the merge: the first Python regeneration's adapt_smoke, adapt_check, adapt_route, adapt_review and regen_gate ... adapt_* notices, and the adapt/ folder of its regen-pair artifact.
Known gaps
Workflow changes have no branch run: claude-code-action refuses to run on a branch whose PR changes its own workflows, so the new steps are first exercised on main. Locally they are covered by unit tests that run the exact step scripts against throwaway repositories with fake gsutil and gh, under bash -eo pipefail.
First real runs. The MUI X pilot of 8 forced regenerations is the first real run of the new prompts, and the lead watches it. MUI X is JavaScript and regen:forced, so it exercises the convention text, not the Python steps; the first Python regeneration after the merge is the first run of the smoke run, the record-only check and the adapt route. The daily regeneration stays paused until this merges ("Rollout" step 3).
Not verified until a real run:gsutil rm -r on a missing prefix, rsync -r of adapt/ to staging, cp -r back in impl-review, and whether bucket-level IAM keeps adapt/ private; impl-merge's non-recursive cp "${STAGING}/plot-*"; the runtime of the Chrome-driven libraries (bokeh, plotly) inside the 20-minute step timeout (estimate 1 to 3 minutes); the seaborn, plotly, altair, plotnine, bokeh, pygal and letsplot paths against the runner (matplotlib ran locally); setup-python with .python-version in impl-review; vars.ADAPT_REVIEW_FREE resolution; the review-free path end to end.
Runner exposure. The smoke executions run model-written code again after "Create library metadata file" has the workflow token in .git/config, the same exposure as the generator's own Step 3 renders. The runner is not an isolation boundary; the protections are the workflow-ref copy of core/, the new-then-predecessor render order, and the owner-controlled switch.
Missing-value coverage. The loader cast turns integer columns into Int64, so an integer-valued sample (the design's own example) gets no binding NaN table, only the advisory <NA> one.
Open decisions for the owner
O1, review-free switch. It stays off. The evidence is the route-held PRs. regen:adapt marks a merge on which the adapt route held before and through the review. regen:adapt-overruled marks a PR whose route held before the review and whose review floor then turned it off, on a merge or a keep; its gate record has adapt.route_pre_review: true. Turn the switch on (gh variable set ADAPT_REVIEW_FREE --body true) only when the overruled share stays near zero over at least 50 route-held PRs:
gh pr list --state merged --label regen:adapt --limit 1000 --json number --jq length
gh pr list --state all --label regen:adapt-overruled --limit 1000 --json number --jq length
The alternative is an automatic flip once 50 route-held PRs show no overruled one.
O2, nullable integers.missing_nullable stays advisory: in part A's idiom run it failed 5 of the 8 Python libraries, and it fails the design's reference example. Either the convention allows .astype(float) on drawn nullable-integer columns, or the loader changes.
Review-retest arm. This PR edits prompts/default-style-guide.md (two code-only sentences) and adds a "Data block" section to every library prompt, and the reviewer reads both. CLAUDE.md asks for a candidate review-retest.yml arm on such edits; the design says none is needed, because no criterion, rubric line or review-prompt line changes. No arm was dispatched; dispatch one if you want the measurement before merging.
Local helpers./regen and /update list prompts/adaptable-code.md but have no Step 3c, so a local regeneration is not self-checked.
The regen gate now merges a conversion to the adaptable-code convention on
render identity (docs/concepts/adaptable-code.md, "The adapt route"), PR 2
contract section 5.
- adapt_assess(inp, review): one pure function for `adapt-route` (before the
review) and `decide`. The route counts when the new source is Python, both
adapt records are valid for the pair, the predecessor fails and the new
source passes, the staged render pair holds both themes at equal size within
IDENTITY_BOUND (0.5 %, equal to core.render_identity.DEFAULT_BOUND; the gate
applies its own bound whatever the pair names), the line bound
max(1.25 x prev, prev + 15) holds and the canvas gate passed; with a review
also a score above 0, no regression, no changed scenario, no added encoding,
a prev_checklist and no VQ/SC/DQ/CQ item below its re-score. The first
failing condition is the reason token (ADAPT_WHY, the contract's order).
- _judge turns no_visible_improvement, no_defect_improvement and
below_tolerance into merge/adapt_identity when the route counts; the
regression exit and the CQ-04 path (tolerance included) are unchanged, and a
carrier merge keeps code merge. _decide merges on the route before the score
checks with --review-skipped; otherwise an unreviewed pair keeps as
no_score. decide() turns any merge into keep/adapt_regression when both
records are valid and the predecessor passes while the new source fails.
- Record: `adapt` {version, prev, new, identity, counted, why, review},
validated by adapt_record_errors (known keys, tokens, finite fractions,
counted == (why == route), no lists); RECORD_VERSION stays 1. Outputs gain
adapt_counted/adapt_why; the notice gains them before reason= only when an
adapt flag is given, so older argument sets print the old line. The summary
gains one **Adapt route:** line.
- New subcommand `adapt-route` (always exits 0; any internal failure turns the
route off with record_invalid and a warning).
- The gate stays standard-library only and reads JSON, never PIL; a test runs
adapt-route and decide from a copy with `python -I`, and one pins the
imports.
Routine decisions (documented, not in the contract):
- A pair file that parses but lacks the identity structure, names another
library or has v != 1 is read as pair_missing ("does not parse as a pair").
- An unreadable record (bad JSON, over 1 MB, nesting too deep) is
record_invalid; an absent flag or a missing file is no_record.
- --review-skipped with a score present is treated as reviewed (the stricter
path, fail-safe).
- A review that ran without a valid review_regen.json is assessed as an empty
review, so the route is off with no_prev_checklist (or score_zero).
- _is_int and the new _real are TypeGuards; that also clears the seven mypy
errors regen_gate.py had on main, and `inp.score` in _judge is read through
a None-safe local.
Docs: the Regen gate section of docs/workflows/overview.md gains the route,
the regression, the record fields and the regen:adapt label row;
docs/concepts/adaptable-code.md gets its status line with the four contract
deviations; automation/scripts/README.md describes the route; the changelog
fragment covers all of PR 2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
…t merge
Part C of PR 2 (docs/concepts/adaptable-code.md, "Pipeline changes" and
"The adapt route"), against the PR 2 interface contract.
impl-review.yml:
- "Set up Python for the adaptability mirror" (py314, update-environment
false) and "Adaptability check (record-only)" (id adapt) after the canvas
gate: the workflow-ref checker mirror on the PR head and, on a
regeneration, on the predecessor from origin/main (written under its
library's file name, deleted after the check); the staged smoke runs from
staging/.../adapt/ under $RUNNER_TEMP; /tmp/anyplot-adapt-new.json and
/tmp/anyplot-adapt-prev.json; adapt_check notices; regen_gate.py
adapt-route; review_skipped as the step's last write. Never a raw exit 1.
- Review-free path behind vars.ADAPT_REVIEW_FREE (only the literal `true`):
the review step skips; Regen gate, Stage regen pair, early verdict,
metadata (merge only) and the verdict step run; a new "Add carried
quality score label" step; regen:adapt between regen:improved and
ai-approved when the gate counts the route (best effort).
- Metadata handoff: carried mode writes main's stored score and review back
unchanged with review.carried, plus the predecessor's impl_tags; the
header gets the carried score with the measured versions. A review that
ran drops any carried mark.
- The gate gets --adapt-new/--adapt-prev/--adapt-pair/--review-skipped.
- The staging download and impl-merge's promotion copy plot-* only, so
adapt/ never reaches the blind review or production.
automation/scripts/adapt_record.py (stdlib, joins the regen-tools copy):
builds a record per side, derives its status, prints the notice; its
code_sha256 is core.smoke_run's rule and its RULE_ORDER the checker's.
regen_writeback.py: a kept regeneration's write-back removes review.carried
(the re-score is a review that ran); verify-diff allows that removal only.
Decided and documented here: adapt-route's outputs go to a file of their
own and become the step's route/why (an unclear answer, or route=true
without why=route, reads as no route); the step uses the 3.14 interpreter
for adapt-route too; static.exit keeps the mirror's real exit code when it
wrote nothing; smoke error texts are capped at 300 characters and every
notice value is reduced to a token; selfcheck.first counts a skipped smoke
run as a pass, since the log line carries no skip reason.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
core/smoke_run.py runs a Python implementation on other tables, composing
PR 1's pure parts (core.adaptability, core.perturbation, core.canvas,
core.render_identity), with the CLI python -m core.smoke_run
{run,pair,selfcheck} of the PR 2 contract (sections 1.1 to 1.5):
- run: a sample probe executes the head and the # Data block once
(light theme) and pickles df with the column each role constant names;
every perturbed table then replaces the block, the column constants are
renamed through their role comments, and the whole file runs once per
theme in a fresh directory as Step 3 runs it (python -P {library}.py,
ANYPLOT_THEME, MPLBACKEND=Agg, PYTHONHASHSEED=0,
PYTHONDONTWRITEBYTECODE=1). A pass needs exit 0, plot-{theme}.png on the
canvas, and, for matplotlib and seaborn on `renamed`, no drawn text that
shows a retired sample category. Errors map back to the original
source's line. Writes the contract's JSON record; exit 0/1/2.
- pair: renders the new source, then the predecessor, unchanged in one
environment, copies the raw PNGs to DIR/new and DIR/prev and measures
render identity per theme.
- selfcheck: the generator's Step 3c, check_file plus run, with a JSON-lines
attempt log.
- code_sha256: the header-free source hash impl-review reimplements.
core/perturbation.py gains `advisory` on Perturbation and the two
missing-value tables for table shapes (contract 1.4, F2): missing_number
(NaN in every float64 column after the loader's dtype cast, binding) and
missing_nullable (<NA> in every Int64, boolean and string column, advisory
per open decision O2). Matrices get neither: the loader reads one flat
table, never a labeled square matrix. The docstring no longer claims there
is no nullable-dtype perturbation.
Decided and documented (contract section 10):
- Executions run in threads, 4 in parallel, 2 for bokeh and plotly (each
starts a headless Chrome); every execution gets its own process group,
killed on the timeout (120 s default) and after the run, and writes
stdout/stderr to files so a straggler holding a pipe cannot stall it.
- The stale-label hook is a runpy shim wrapping matplotlib.text.Text.draw,
so it sees text that was drawn, not every Text object a figure holds;
it skips the mandated title (anyplot.ai) and retired values that read as
prose, numbers or single characters, as the static checker does.
- Executions do not inherit PYTHONPATH, so a file runs as Step 3 runs it
even when the runner comes from a workflow-ref copy of core/.
- The loader cast reads an object column by its values (ints -> Int64,
datetime.date objects stay as they are), as the loader profiles values.
- The run record carries an additive top-level `error` (null unless the
run could not happen) next to the contract's fields.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
…and impl-repair
impl-generate gets "Adaptability smoke run (record-only)" (id adapt_smoke)
between "Create library metadata file" and "Process plot images", for
Python only, continue-on-error, 20-minute timeout (PR 2 contract 1.6):
- copies core/ from the workflow's own ref (git archive $GITHUB_SHA) and
runs it with PYTHONPATH pointing there, never the branch's core/;
- runs `core.smoke_run run` on the new source;
- on a regeneration (the file exists on origin/main, impl-review's test)
renders the new source and then the predecessor in this runner with
`pair` (both themes, python -P {library}.py with the theme env, as the
generator's Step 3), so the gate measures identity on raw renders before
pngquant, and runs the predecessor with --skip-if-static-fails;
- copies the generator's Step 3c log; every result lands in
$RUNNER_TEMP/adapt.
"Upload to GCS Staging" clears staging/.../adapt/ first, unconditionally,
then stages this run's artifacts privately (Cache-Control: no-store, no
AllUsers ACL, never promoted); a failed upload only warns. A predecessor
source left behind by a cut-off smoke step is never staged.
impl-repair gets the same step after "Process repaired image" (run only,
no pair: a gated regeneration never enters repair), and "Upload repaired
plot to GCS staging" replaces adapt/new-smoke.json or, without a new one
or on a failed upload, removes the stale one so impl-review reads missing.
No step gates on an exit code; no existing step name or id changes and
"Create library metadata file" is untouched. The new tests run the real
step scripts against a scratch origin with a stub runner in the workflow
ref, another one on the branch, and a stub gsutil, under bash -e and
bash -eo pipefail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
…library prompts
Part A of PR 2 of docs/concepts/adaptable-code.md (the rules). Prompts only;
the smoke runner, the workflow steps and the gate come from parts B to D.
- prompts/adaptable-code.md (new): the one copy of the convention that the
generator, the repair prompt and later the agent's adapter load: the six
zones, the marker and role-comment grammar, the plot-code rules (colors per
category, labels/ticks/limits/counts from df, computed statistics,
rule-chosen callout targets, missing values stay missing, data order), the
data shapes, the design's bar-grouped matplotlib example, and the R, Julia
and JavaScript forms.
- plot-generator.md: Output template (scatter-basic) and Structure rewritten
to the six zones; Data Generation Strategy gets the block rules and
block-form examples; Data storytelling gets rule-chosen callout targets;
bbox_inches='tight' removed from the template, Section Comments and Output
Files; the new file joins the allowed inputs and Testing.
- default-style-guide.md: Whitespace no longer recommends bbox_inches='tight';
Color Restraint gets "list all eight hues and use the first n".
- prompts/library/*.md (15): a "## Data block" section each, with the idiom of
"Other libraries": seaborn/plotnine/lets-plot data=df with the constants;
Altair escaped fields with type= and title=; plotly px and graph objects;
bokeh ColumnDataSource(df) with dodge; pygal .tolist() below the block;
ggplot2 tibble(!!X :=), .data[[X]] and labs(); Makie DataFrame(X => ...) and
df[!, X]; the five JavaScript libraries as records keyed by const role
constants, d[X], and window.ANYPLOT_DATA as the agent's later substitute.
Existing data-generating examples and skeletons follow the block form; the
[:N] palette slices in matplotlib.md and seaborn.md are gone; Chart.js no
longer recycles the palette with i % length. MUI X keeps its v9 content.
- impl-generate-claude.md: the convention joins Step 1 and Step 2; Step 3c
(Python only) runs `core.smoke_run selfcheck`, at most three attempts,
never blocking, with the optional `pair` measurement on a regeneration; the
commit body carries `Adaptability self-check: pass|fail (...), attempt N`;
the five regen-mindset amendments are phrased "For Python
(`LANGUAGE=python`)", plus a "Converting a Python file" bullet that aims at
render identity and an explicit note that R, Julia and JavaScript keep
today's regen mindset; Final Check items 7 and 8.
- impl-repair-claude.md: self-check lines are real defects like canvas
lines; the "Don't add code" sentence is amended for Python; Step 5b runs
the same self-check after the fix; the commit body carries its result.
- tests/unit/prompts/test_prompts.py: TestAdaptableCode pins the six zones,
runs both full examples through core.adaptability.check_source (both pass),
forbids recommending bbox_inches='tight', and pins every library's Data
block section, Step 3c's place and command, the five Python amendments and
the repair changes.
- prompts/README.md, docs/reference/repository.md, agentic/docs/project-guide.md,
agentic/commands/{regen,update}.md: the new file in their prompt lists (and
the stale "11 files: 9 Python" library count corrected).
Decided and documented here:
- The convention lives in a new prompts/adaptable-code.md (the contract's
recommended option), referenced from plot-generator.md and Step 1.
- The commit line's classes are the remaining non-advisory rule classes plus
the names of the remaining failed perturbations.
- Step 3c and Step 5b `cd "$GITHUB_WORKSPACE"` first, and the optional pair
command creates /tmp/anyplot-prev-src before `git show` writes into it.
- Before converting, the generator runs the static check alone
(`core.adaptability check`) on the previous file, so the self-check log
counts only attempts on the new file.
- plotnine needs `limits=` from df on the discrete axis and the fill scale:
it sorts string columns alphabetically, which would hand #009E73 to the
alphabetically first group; lets-plot keeps the data order by itself.
Verified: the Altair, plotnine, lets-plot, bokeh, plotly, seaborn, pygal and
matplotlib idioms were smoke-run outside the repo on PR 1's perturbed tables
(renamed columns with spaces, a unit and a dot; more/fewer categories; single
group; fewer/more rows; scaled) plus NaN in float64: every binding case passes.
The advisory <NA> case fails for matplotlib, plotly, plotnine, bokeh and
pygal, as the design expects for open decision O2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
…s, self-check reason
Integration of PR 2 of the adaptable-code rollout (parts A to D merged above).
- impl-review.yml: one unconditional "Set up Python" step (python-version-file
.python-version) right after "Checkout repository", before the first
regen-tools call, per the contract's addendum. regen_gate.py,
review_provenance.py and the checker mirror carry PEP 758 `except A, B:`
since #12247; the runner image's python3 is not pinned. Part C's
mirror-only step ("Set up Python for the adaptability mirror",
update-environment: false) is gone, and the adapt step calls plain python3
like every other step. The wiring test now checks that every regen-tools
call runs on that one interpreter.
- core/smoke_run.py selfcheck log line gains `smoke_why` (additive), and
adapt_record.selfcheck_block counts a skipped first attempt as a pass only
for a shape skip, as selfcheck's own exit 0 does. Before, a runner_error on
the first attempt read as a pass and inflated the first-attempt pass rate
the stop rules read.
- Docs: the Implementation pipeline diagram and a table of the adaptability
steps (the step sentences parts B and C sent), the pair artifact's adapt/
folder (docs/workflows/overview.md); adapt_record.py and the regen_writeback
`review.carried` handling (automation/scripts/README.md); core/smoke_run.py
and adapt_record.py (docs/reference/repository.md); the design doc's status
line no longer claims the runner's Python is 3.12.
- changelog.d/adaptable-code-rules.md: a Changed bullet for the pinned
Python and the plot-* promotion.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
This pull request rolls out the adaptable-code convention across plotting prompts, generation and repair workflows, review checks, and regeneration gating.
Changes:
Adds six-zone Data block guidance and library-specific examples across the prompt set.
…/smoke, prompt fixes
- regen_gate.adapt_record_problem: a record whose `static` or `smoke`
block is missing or carries an unknown status is `record_invalid`. The
derived status comes from those two blocks, so a record without them was
not built by the record step and must never count the route (fail closed,
as the design asks of any malformed record). The gate still reads the
derived status as it is and never recomputes it; D's test that pinned the
fail-open reading now pins the fail-closed one.
- prompts/library/chartjs.md: the Colors snippet named an undefined
`categories`; it now uses the Data block's `lines` and `quarters`.
- prompts/adaptable-code.md: "a ninth group fails loudly" holds for
`IMPRINT[i]` and a dict lookup by group; a library's color cycle repeats
colors instead, which the rule now says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
This new rule conflicts with the adaptable-code convention in prompts/adaptable-code.md:22: a library color cycle fed the full palette can repeat colors for a ninth group, so the code shown here does not make a ninth group fail loudly. Generators will receive contradictory instructions about the required behavior for 9+ groups. Please qualify the loud-failure claim to explicit index/lookup implementations and state that cycle-based libraries repeat (or require an explicit guard).
… in the style guide
- prompts/library/echarts.md: the Data block example now initializes the
chart before chart.setOption, so the fragment runs as written.
- prompts/default-style-guide.md: the loud failure on a ninth group holds
for a lookup by position or by group; a library's color cycle repeats
colors. Same wording as prompts/adaptable-code.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
…rces (pair_stale)
Gemini's first review (and part D's own hardening note): adapt_assess read
identity from pair.json without checking that the pair rendered the two
sources the adapt records checked. A pair from an earlier PR head, or from
another predecessor, measured other code and could still count the route.
The route now requires pair.new.code_sha256 and pair.prev.code_sha256 to
equal the new and prev records' code_sha256 (one rule: core.smoke_run's
code_sha256, which adapt_record.py reimplements). A mismatch or a missing
hash is the new reason token `pair_stale`, after `pair_error` and before
`size_changed`; it fails closed like every other malformed input. The end
to end chain on real artifacts (smoke_run pair -> adapt_record ->
adapt-route/decide) still routes, so both hash computations agree.
Tests: gate cases for a mismatched or missing hash on either side; part C's
real-gate step test now stages the pair as core.smoke_run writes it, plus a
case that refuses a pair of an earlier head and leaves the review running.
Docs: the route conditions in automation/scripts/README.md and
docs/workflows/overview.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
… library
A record whose status does not follow from its static and smoke statuses
is now malformed (record_invalid): a crafted `pass` over a smoke run that
never ran (static pass, smoke missing) routed with why=route before. The
gate keeps a stdlib copy of adapt_record.py's derive_status
(derived_adapt_status), and a unit test runs both over every combination.
pair_problem now requires the pair record's `library` key: core.smoke_run
pair always writes it, so a pair without one was not written there.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
The adapt regression held only when the predecessor's record was `pass`,
which needs a fresh prev-smoke.json. A predecessor whose static check
passes while its smoke run is missing, stale or skipped (status unknown),
against a new source whose static check fails, is a certain regression
without any smoke run: converted code was unconverted. adapt_assess now
sets `regression` in that case too, so any merge still becomes keep with
adapt_regression, and the reason and the PR-comment line say that the
regression rests on the static check alone.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
A PR whose adapt route held before the review and whose review then
turned it off is the evidence the owner needs before turning on
ADAPT_REVIEW_FREE, and it was invisible: regen:adapt goes only on PRs
where the route held through the review.
- regen_gate.py decide takes --adapt-route-held (impl-review passes it
when the adapt step's adapt-route said `true`). The record's adapt
block gains route_pre_review (true, false, or null without adapt
input; additive, RECORD_VERSION stays 1, a block without it still
validates), and decide writes adapt_overruled=true when the route held
before the review and adapt.why is one of the review-floor reasons
(ADAPT_REVIEW_WHY). The notice line and the PR comment's adapt line
name it.
- impl-review's early verdict step adds regen:adapt-overruled before the
verdict label, on a merge or a keep, with the same best-effort
three-attempt pattern as regen:adapt; the verdict comment says so.
- Docs: route-held PRs = regen:adapt + regen:adapt-overruled; the switch
is turned on only when the overruled share stays near zero over at
least 50 route-held PRs.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
…dict
"Re-dispatch review after a post-score failure" keyed on the review step,
which the review-free adapt path skips, so a failure of the carried
quality label or the early verdict there left the PR with no verdict
label and no retry. The step now also runs when review_skipped is true,
with the stored score as SCORE (as "Update metadata and implementation
header" takes it), and its comments say that no review ran instead of
"The review scored N/100".
Known gap, unchanged: on the normal path the early verdict step runs
before the metadata step, so a metadata failure after a verdict label
landed is left to the watchdog.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
…their names
"Upload to GCS Staging" copied `"$ADAPT"/*` with `cp -r` into an adapt/
prefix it had just removed. By gsutil's naming rule, copying a directory
into a destination subdirectory that does not exist yet drops the
directory's name, so new/plot-light.png and prev/plot-light.png could
both land as adapt/plot-light.png. `gsutil rsync -r "$ADAPT" .../adapt`
names every object by its path under $ADAPT; the warning fallback stays.
impl-repair is unchanged: it copies one file to an explicit object name
(adapt/new-smoke.json), which has no directory naming to lose. The fake
gsutil of test_adapt_generate_steps.py now mirrors `rsync -r SRC DST`,
and the test asserts the six object names, adapt/new/plot-light.png and
adapt/prev/plot-light.png among them.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
execution_env handed the runner's whole environment, minus PYTHONPATH,
to every probe, perturbation and pair render, all of which run
model-written code. It now also drops every variable whose name starts
with GITHUB_, ACTIONS_, RUNNER_ or GH_ (GITHUB_TOKEN, GH_TOKEN,
ACTIONS_RUNTIME_TOKEN, the cache and results URLs, the runner's paths)
and keeps everything a plot needs: PATH, HOME, DISPLAY, the locale and
the libraries' settings. This narrows what a smoke run exposes; the
runner is still no isolation boundary (the workspace's .git/config holds
the token, as the PR body's Known gaps say).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
…turb
core/smoke_run.py used `shape:<shape>` both for a shape without tables
and for a table shape whose sample gave perturbations() nothing, so a
`shape:table` skip read like a shape exclusion. The second case is now
`no_perturbations:<shape>`.
- selfcheck keeps exiting 0 for it (nothing the model can fix): it
accepts both prefixes, SELFCHECK_SKIP_PREFIXES.
- adapt_record.py keeps a copy of the prefixes (it is copied alone to
regen-tools and imports no core; a parity test ties the two), so the
record's first-attempt pass rate agrees with selfcheck's exit code.
- derive_status still maps the skip to unknown/smoke_skipped: only
shape:generated and shape:network are not applicable.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
prompts/adaptable-code.md named the palette constant `IMPRINT` (in its
bar-grouped example and the color rules), while the style guide and
every library prompt use `IMPRINT_PALETTE`. The convention, its example,
the generator's zone list and the style guide's "list all eight hues"
sentence (this PR's own, code identifiers only) now use
`IMPRINT_PALETTE`, and the convention states once that the checker also
accepts `IMPRINT` (core/adaptability.py PALETTE_NAMES). The example still
passes the checker (test_prompts.py runs it).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175hA62juZoajMcqKxWFGza
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
impl-generate.ymlandimpl-repair.yml, a record-only adaptability check inimpl-review.yml, and the identity-based adapt route plus the adapt regression in the regen gate.exit 1), and the review-free path stays off behind the repository variableADAPT_REVIEW_FREE.language, so their checkers plug in later without format changes (the "all 15 libraries" goal).Plan
Design:
docs/concepts/adaptable-code.md("The convention", "Other libraries", "The checks", "Pipeline changes", "The adapt route", "Rollout" item 2). Four parts were built against one interface contract and merged here with--no-ff; the merge commits keep each part's history.A, prompts (
eb3d074fa)prompts/adaptable-code.mdholds the convention once: the six zones, the# Data/# Plotmarkers and# role:grammar, the plot-code rules, the data shapes, the bar-grouped example, and the R, Julia and JavaScript forms.plot-generator.mdand Step 1 ofimpl-generate-claude.mdreference it.plot-generator.md: the template, Structure and Data Generation Strategy follow the six zones; Data storytelling gets rule-chosen callout targets;bbox_inches='tight'is gone.default-style-guide.md: "list all eight hues and use the first n", nobbox_inches='tight'.[:N]palette slices are gone from matplotlib and seaborn; Chart.js no longer recycles colours withi % length.impl-generate-claude.md: Step 3c (python -m core.smoke_run selfcheck, at most three attempts, never blocking), the commit lineAdaptability self-check: pass|fail (...), attempt N, the five regen-mindset amendments phrased "For Python (LANGUAGE=python)", a "Converting a Python file" bullet that aims at an identical render.impl-repair-claude.md: self-check lines are real defects; Step 5b reruns the self-check.B, generation side (
416065feb,3425ac00b)core/smoke_run.py(run,pair,selfcheck): one process per perturbed table and theme, process groups killed on timeout, errors mapped back to the original line, a stale-label hook for matplotlib and seaborn, andpair, which renders the new source and then the predecessor in one environment and compares the raw PNGs.core/perturbation.py:missing_number(NaN in everyfloat64column, binding) andmissing_nullable(<NA>inInt64,booleanandstringcolumns, advisory).impl-generate.yml"Adaptability smoke run (record-only)" and the privateadapt/staging in "Upload to GCS Staging";impl-repair.ymlreruns the smoke run and replaces or removesadapt/new-smoke.json.C, review side (
4b710e2c0)impl-review.yml"Adaptability check (record-only)": the workflow-ref checker mirror on the PR file and, on a regeneration, its predecessor;automation/scripts/adapt_record.pybuilds/tmp/anyplot-adapt-{new,prev}.json;regen_gate.py adapt-routebefore the review; the review is skipped only whenADAPT_REVIEW_FREEistrueand every condition holds.regen:adaptcount label, carried metadata (review.carried), and the staging download narrowed toplot-*soadapt/never reaches the blind reviewer.impl-merge.ymlpromotes onlyplot-*.regen_writeback.pyremovesreview.carriedon a write-back.D, gate (
0438a60a4)adapt_assesswith the contract's 20 ordered conditions;adapt_identityreplaces the keepsno_visible_improvement,no_defect_improvementandbelow_tolerance;--review-skippedmerges a counted route before the score checks;adapt_regressionturns any merge into keep. The record gains anadaptblock (RECORD_VERSIONstays 1), the outputs and notice gainadapt_countedandadapt_why. The gate stays stdlib-only and reads identity from impl-generate'spair.json.Integration (
db865896b)impl-review.ymlgets one unconditional "Set up Python" step (python-version-file: .python-version) before the first regen-tools call; part C's mirror-only step is gone and every tool runs on plainpython3. Since chore: move to Python 3.14, with the sandbox fixes it needs #12247 the gate,review_provenance.pyand the mirror use PEP 758except A, B:, and the runner image's Python is not pinned.smoke_why, so the adapt record counts a skipped first attempt as a pass only for a shape skip, asselfcheck's own exit 0 does.docs/workflows/overview.md),adapt_record.py(automation/scripts/README.md),core/smoke_run.py(docs/reference/repository.md).Review fixes (
05ec4a82b,602dd40c5,4fd7f7c57)staticandsmokeblocks isrecord_invalid(fail closed); the Chart.js and ECharts Data block examples run as written; the "ninth group fails loudly" rule names the idioms it holds for (lookup by position or group) inprompts/adaptable-code.mdandprompts/default-style-guide.md.pair.jsonrendered from the very sources the records checked (code_sha256on both sides), else the new reason tokenpair_stale.Second opinion (Gemini 4 Argon + Fable,
1e75be40a..a423e7b69)statusdoes not follow from itsstaticandsmokeblocks (record_invalid), andpair.jsonmust name itslibrary.decide --adapt-route-held, the record fieldadapt.route_pre_review, the outputadapt_overruledand the labelregen:adapt-overruledmake the review's overrules of the route countable (O1).adapt/is staged withgsutil rsync -rsonew/andprev/keep their names.execution_envdropsGITHUB_*,ACTIONS_*,RUNNER_*andGH_*before running an implementation.no_perturbations:<shape>, read the same way byselfcheckand the record.IMPRINT_PALETTE, in the convention and its example; theIMPRINTalias is stated once.Test plan
uv run ruff check .anduv run ruff format --check .: clean.uv run --extra typecheck --extra agents mypy api core agents: no issues in 151 source files.uv run pytest tests/unit tests/integration -n 4: 13158 passed, 3 skipped (seaborn is not installed locally). That is 443 more tests thanmain:test_smoke_run.py34,test_adapt_generate_steps.py36,test_adapt_review_steps.py85,test_adapt_record.py66, plus additions totest_regen_gate.py,test_perturbation.py,test_regen_writeback.pyandtest_prompts.py.uv run python -m tools.changelog check --base origin/main: passes.mainand on this branch, the same set (only line offsets in impl-generate's upload step moved).run,pairandselfcheckfed part C's mirror andadapt_record.py, which fed part D'sadapt-routeanddecide --review-skipped:route=true, mergeadapt_identity, identity 0.0000 in both themes, a valid gate record. The new side's smoke run passed all 7 binding tables;missing_nullablefailed as advisory (float() argument ... 'NAType'inax.bar).adapt_smoke,adapt_check,adapt_route,adapt_reviewandregen_gate ... adapt_*notices, and theadapt/folder of itsregen-pairartifact.Known gaps
Workflow changes have no branch run: claude-code-action refuses to run on a branch whose PR changes its own workflows, so the new steps are first exercised on
main. Locally they are covered by unit tests that run the exact step scripts against throwaway repositories with fakegsutilandgh, underbash -eo pipefail.regen:forced, so it exercises the convention text, not the Python steps; the first Python regeneration after the merge is the first run of the smoke run, the record-only check and the adapt route. The daily regeneration stays paused until this merges ("Rollout" step 3).gsutil rm -ron a missing prefix,rsync -rofadapt/to staging,cp -rback in impl-review, and whether bucket-level IAM keepsadapt/private; impl-merge's non-recursivecp "${STAGING}/plot-*"; the runtime of the Chrome-driven libraries (bokeh, plotly) inside the 20-minute step timeout (estimate 1 to 3 minutes); the seaborn, plotly, altair, plotnine, bokeh, pygal and letsplot paths against the runner (matplotlib ran locally);setup-pythonwith.python-versionin impl-review;vars.ADAPT_REVIEW_FREEresolution; the review-free path end to end..git/config, the same exposure as the generator's own Step 3 renders. The runner is not an isolation boundary; the protections are the workflow-ref copy ofcore/, the new-then-predecessor render order, and the owner-controlled switch.Int64, so an integer-valued sample (the design's own example) gets no binding NaN table, only the advisory<NA>one.Open decisions for the owner
regen:adaptmarks a merge on which the adapt route held before and through the review.regen:adapt-overruledmarks a PR whose route held before the review and whose review floor then turned it off, on a merge or a keep; its gate record hasadapt.route_pre_review: true. Turn the switch on (gh variable set ADAPT_REVIEW_FREE --body true) only when the overruled share stays near zero over at least 50 route-held PRs:missing_nullablestays advisory: in part A's idiom run it failed 5 of the 8 Python libraries, and it fails the design's reference example. Either the convention allows.astype(float)on drawn nullable-integer columns, or the loader changes.prompts/default-style-guide.md(two code-only sentences) and adds a "Data block" section to every library prompt, and the reviewer reads both.CLAUDE.mdasks for a candidatereview-retest.ymlarm on such edits; the design says none is needed, because no criterion, rubric line or review-prompt line changes. No arm was dispatched; dispatch one if you want the measurement before merging./regenand/updatelistprompts/adaptable-code.mdbut have no Step 3c, so a local regeneration is not self-checked.