Skip to content

feat: the autopilot's cap has no way on but a claim, and lost runs count - #857

Merged
soydiloreto merged 14 commits into
mainfrom
fix/856-the-autopilot-s-cap-has
Nov 19, 2026
Merged

soydiloreto merged 14 commits into
mainfrom
fix/856-the-autopilot-s-cap-has

Conversation

@dilux-bot

@dilux-bot dilux-bot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

📝 What changes

C1, the free run on a "do not build on" refusal. In autopilot.yml's apply job, the step applied writes refused=diverged to $GITHUB_OUTPUT just before it exits 1 with "The agent's commits do not build on $BRANCH as GitHub has it". The end rule gets it as REFUSED: ${{ steps.applied.outputs.refused }}. autopilot.py end treats it the way it treats a lost runner or a held-back make pre-pr (lost / held_back): the run's entry on the pull request's board is rewritten as run_entry(runs, True), so it stays out of run_numbers()'s count. The verdict still stops the run as it does today: it is free, but it does not continue. The rule for which runs are free moves into one pure helper, free_end(mode, stopped, asked, lost, held_back, refused), so the test can call it directly.

C2–C4, /autopilot more. Its first line is /autopilot more, and whatever follows is the hint. It is read on an issue (not a pull request) by a new job, more, in .github/workflows/issue-claim.yml, the reusable workflow that already reads /claim. The claim job's if: leaves out comments that start with /autopilot more, so that "still on it" inside a hint never extends a claim. Like the claim job, the more job runs no repository code, uses a fresh runner and its own dilux-bot token (issues and pull requests write, contents read for CODEOWNERS), and clones .dx-central. It does three things:

  1. It finds the open autopilot pull request that closes the issue.
  2. It asks deciders.py may continue "$LOGIN", the same check as for adding autopilot:continue (who-decides, read live).
  3. It runs autopilot.py more, which applies a pure planner, more_plan(allowed, state, intact, policy_cap, at_cap, said_before):
    • Not allowed (C3): no run is given, no label is added and the board is left alone. dilux-bot comments once on the issue, under a hidden dx-autopilot-more-refused marker so a repeat gets no second reply. The comment says a maintainer decides whether the autopilot gets more runs, gives the issue's full URL, and ends with the AI line.
    • Two extensions already (C4): refused with a reply that names both ways forward: anyone may /claim the issue and finish it, or a maintainer closes the pull request.
    • Not at its cap, or a board edited by hand: refused with a reply that says why. Nothing else changes.
    • Allowed (C2): extensions on the pull request's board gets one more entry, {"by": login, "at": iso, "comment": url}. needs-human comes off the pull request, autopilot:continue is added to the issue, and the issue's board gets no answered, so the run counts against the new cap.

The cap, read per pull request. A new pure helper, runs_cap(policy_cap, state), gives the cap with that pull request's extensions counted. may-continue and end use it. The sweep counts no runs of its own (sweep_step is called with none), and a continue it gives still goes through may-continue. A board edited by hand still reads as the cap reached.

The hint. answers() does not restart from a cap end (CAP is not among the bot's comments it restarts from): a cap end is no new start, so a hint a decider wrote before it, which no run read, also reaches the run /autopilot more gives. A /autopilot more comment from a decider becomes just the rest of its text, and is dropped if the rest is empty. So the next run's prompt (ANSWERS, which already reads the issue's comments and the pull request's) gets the rest of the comment as its hint. run_end and reaccepted are unchanged.

The cap message. cap_comment() (on the pull request, and with --issue on the issue) adds that a maintainer may give this pull request 5 more runs with /autopilot more <hint> on the issue, at most twice. policy/autonomy.yml gets a comment only (autopilot-runs stays 5). docs/agents.md, CONTRIBUTING.md, docs/workflows.md (the autopilot.yml and issue-claim.yml rows) and the header comment in issue-claim.yml describe the command.

What one extension adds (option a, the maintainer's answer). Each extension adds the policy's autopilot-runs: cap = autopilot-runs × (1 + extensions), counting at most 2 extensions. Here that is 10, then 15; a repository that lowered autopilot-runs to 3 gets 6, then 9, so nothing it chose is widened past its own scale.

When something fails. deciders.py exit 0 is a yes and 1 a no; 2, or any other exit, is "could not tell": no run is given, and dilux-bot replies under its own marker, dx-autopilot-more-unread, to try again in a while, said once until another /autopilot more answer follows it. The board is written first, so a failed write changes nothing else. If adding autopilot:continue to the issue then fails, nothing would start the run: the extension is taken back off the board, needs-human goes back on the pull request, and the job fails, so the same /autopilot more again extends it instead of finding it below its cap.

The issue at the cap. At the runs cap the issue also gets the way past it (cap-comment --issue, the same text as the pull request's cap_comment, unmarked so run_end and reaccepted read what they read before). OPEN_TO_CLAIM stays for the rounds cap, with no /autopilot more.

What this widens, for whoever merges it. A pull request may now spend up to 3 × autopilot-runs counted runs, and only on a comment from someone who-decides allows. A run that the "do not build on" refusal ends no longer counts. The new more job holds dilux-bot's key with permissions the claim job of the same workflow already holds (issues and pull requests write, contents read). No new secret is added, and no caller grants anything new.

💡 Why

Closes #856. The autopilot works it here in the open: one commit per criterion, each box ticked once its commit is pushed.

🎯 Acceptance criteria

  • C1 Given a run that ends with apply-and-end's "do not build on" refusal, when its entry is written on the pull request's board, then it is free and does not count towards the cap.
    Test: tests/workflows/test_autopilot_refused_free.py
  • C2 Given a pull request at its cap, when a person who-decides allows comments /autopilot more on its issue, then its board records the extension, its cap becomes 10, and the next run starts with the rest of the comment as its hint.
    Test: tests/workflows/test_autopilot_more.py
  • C3 Given the same comment from someone who-decides does not allow (a contributor from a fork, say), when it is read, then no run is given, dilux-bot answers once that a maintainer decides, with the issue's link, and the board is unchanged.
    Test: tests/workflows/test_autopilot_more.py
  • C4 Given a pull request with 2 extensions, when /autopilot more is commented again, then it is refused with a reply that names /claim and closing.
    Test: tests/workflows/test_autopilot_more.py
  • C5 Given policy/autonomy.yml, when it is read, then autopilot-runs is still 5.
    Test: tests/workflows/test_autopilot_more.py (checks that autopilot-runs is still 5 and that its comment names /autopilot more as the way one pull request goes past it. The comment check is what makes the test fail at the base.)

🧭 Decisions

  • 2026-11-19: autopilot.py more finds the issue's open autopilot pull request itself (c​losedByPullRequestsReferences, open, labelled autopilot, opened by dilux-bot) rather than the job's shell, so the lookup is tested with the rest of the command; deciders.py finds the same pull request on its own when PR is unset.
  • 2026-11-19: When deciders.py cannot tell (exit 2), /autopilot more gives no run and replies, without the refused marker, to comment again in a while: never a yes, and a later answer is not silenced by it.
  • 2026-11-19: What one /autopilot more adds (the maintainer's answer, option a): each extension adds the policy's autopilot-runs, so the cap is autopilot-runs × (1 + extensions). C2 reads: its cap becomes autopilot-runs × 2, 10 here (then × 3, 15). A repository that lowered autopilot-runs keeps its own scale, so nothing it chose is widened. docs/agents.md and CONTRIBUTING.md say so; the choice listed under What
  • 2026-11-19: When who decides cannot be read, /autopilot more replies under its own marker, dx-autopilot-more-unread, and says it again only once another /autopilot more answer follows it: said once, as the maintainer asked, without silencing a maintainer who tries again after a different outcome, nor the refused reply, whose marker stays its own.
  • 2026-11-19: At the runs cap the issue gets open_to_claim's text (cap-comment --issue), the same way past the cap as the pull request's cap_comment but unmarked: the issue's comments are not the pull request's ends, so run-end and reaccepted read what they read before. OPEN_TO_CLAIM stays for the rounds cap and older scripts, with no /autopilot more.
  • 2026-11-19: The earlier line on what one extension adds, in full: each /autopilot more adds the policy's autopilot-runs, so the cap is autopilot-runs × (1 + extensions); C2 reads its cap becomes autopilot-runs × 2, 10 here (then 15). docs/agents.md and CONTRIBUTING.md say so.

🙌 Credits

🧪 How I tested it

  • python3 -m pytest -q tests: 598 passed, 1 skipped. tests/workflows/test_autopilot_refused_free.py (C1) and tests/workflows/test_autopilot_more.py (C2–C5) were each seen failing at the base when written; the four review minors each got a test there that failed before its fix (the issue's cap text past the second extension, no pr_board plumbing left, the unread reply said once, a hint before the cap reaching the next run).
  • python3 scripts/autopilot.py --test: all pass, with checks for cap_way/open_to_claim, unread_said and the hint before a cap end.
  • Every scripts/*.py --test: all pass but api-commit.py (this sandbox cannot write git objects) and design-lint.py (stylelint not installed); this branch touches neither.
  • local-review.sh --no-claude: conventions OK; the diff read against the general profile.
  • This repository has no make pre-pr, so the checks above stand in for it.

🤖 AI-generated · Claude Opus 5.5 (Anthropic)

The draft pull request opens before any code: the Spec as its checklist.
@dilux-bot dilux-bot Bot added the autopilot Opened by the autopilot from an accepted issue label Nov 19, 2026
@dilux-bot

dilux-bot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor Author

The pull request's board

State
Waits on nobody

🤖 AI-generated · Claude Opus 5.5 (Anthropic)

dilux-bot Bot added 6 commits November 19, 2026 06:37
When apply-and-end refuses the agent's commits because they do not build on the branch as GitHub has it, the step now says so in its refused output and the end rule passes it as REFUSED. autopilot.py end rewrites the run's entry on the pull request's board as free, as it does for a lost runner or a held-back make pre-pr, so the engine's own race no longer spends the run. The verdict still stops the run. The rule lives in the pure helper free_end, and the rewrite in freed_runs.

Assisted-by: Claude Opus 5.5 (Anthropic)
A comment on an issue whose first line is /autopilot more, from someone who-decides allows, gives the issue's autopilot pull request at its cap one extension: an entry on its board, needs-human off it, and dilux-bot's autopilot:continue on the issue, with no answered flag so the run counts. issue-claim.yml's new more job reads it with its own token, and the claim job leaves the command out. The cap is read per pull request (runs_cap: the policy's autopilot-runs once more per extension) by may-continue, end and next_step. answers() restarts at a cap end and gives the rest of the comment as the next run's hint.

Assisted-by: Claude Opus 5.5 (Anthropic)
The same comment from someone who-decides does not allow gives no run: no label moves and the pull request's board is unchanged. dilux-bot answers once on the issue, under the hidden dx-autopilot-more-refused marker, that a maintainer decides, with the issue's link and the AI line; a repeat gets no second reply. When who decides cannot be read, it is never taken for a yes.

Assisted-by: Claude Opus 5.5 (Anthropic)
A pull request that already had its two extensions gets no third: the board and the labels stay as they are, and dilux-bot's reply names both ways forward, anyone's /claim of the issue or a maintainer closing the pull request. A pull request below its cap, or whose board was edited by hand, gets a reply that says why and nothing else.

Assisted-by: Claude Opus 5.5 (Anthropic)
autopilot-runs stays 5 in policy/autonomy.yml; its comment now says that one pull request goes past it only on /autopilot more by someone who-decides allows, at most twice. The runs cap's comment on the pull request and the OPEN_TO_CLAIM comment on the issue say how a maintainer gives more runs, the former only while an extension is left. docs/agents.md, CONTRIBUTING.md and docs/workflows.md describe the command and the free run on a "do not build on" refusal.

Assisted-by: Claude Opus 5.5 (Anthropic)
GitHub's startsWith also matches a comment such as "/autopilot moreover", which the more job then read as the command. autopilot.py more now checks the comment with more_hint first and does nothing, and says nothing, when it is not the command.

Assisted-by: Claude Opus 5.5 (Anthropic)
@dilux-bot

dilux-bot Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

The autopilot has a question before it goes on. Every criterion is done and pushed (C1 to C5, the full pytest suite and the scripts' self-tests pass). One choice the Spec left to a maintainer: what one /autopilot more adds to a pull request's cap. With the organisation's autopilot-runs: 5, all three give 10 after the first. (1) Recommended, and what the branch does now (runs_cap in scripts/autopilot.py): each extension adds the policy's autopilot-runs, so the cap is runs x (1 + extensions), 10 then 15 here. A repository that lowered autopilot-runs to 3 gets 6, then 9, so its own lower limit keeps scaling and nothing it chose is widened. (2) Each extension adds a fixed 5: 10 then 15 here, 8 then 13 where autopilot-runs is 3. (3) A fixed 10 whatever the count, so a second extension adds nothing, which goes against C4's limit of 2. Answer 1, 2 or 3. With 1, the next run only marks the pull request ready. With 2 or 3, it changes runs_cap and its tests first.

Answer here in your own words; the autopilot goes on with your answer.

🤖 AI-generated · Claude Opus 5.5 (Anthropic)

@dilux-bot

dilux-bot Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

@soydiloreto: answer the autopilot's question #857

🤖 AI-generated · Claude Opus 5.5 (Anthropic)

@soydiloreto

Copy link
Copy Markdown
Member

1: each extension adds the policy's autopilot-runs, as the branch does now, so a repository's lower limit keeps scaling and nothing it chose is widened. Say it in the Decisions and in docs/agents.md where the command is described (10 then 15 with the organisation's 5), and make C2's wording in the description match (its cap becomes autopilot-runs × 2, 10 here).

🤖 AI-assisted · Claude Opus 5.5 (Anthropic)

@dilux-bot
dilux-bot Bot marked this pull request as ready for review November 19, 2026 07:38
@dilux-bot
dilux-bot Bot requested a review from soydiloreto as a code owner November 19, 2026 07:38
dilux-bot Bot added 2 commits November 19, 2026 07:38
The maintainer chose that each extension adds the policy's autopilot-runs, so a repository's lower limit keeps scaling: docs/agents.md and CONTRIBUTING.md now say the cap becomes autopilot-runs × 2, then × 3 (10, then 15, with the organisation's 5).

Assisted-by: Claude Opus 5.5 (Anthropic)
@dilux-bot dilux-bot Bot added risk:high Set by the Claude review complexity:high Set by the Claude review type:feat The kind of change, read from the diff by the Claude review labels Nov 19, 2026
@dilux-bot

dilux-bot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Claude review · risk high · complexity high · type feat

Nothing that must be fixed · 4 minor

Incremental review: the new commits make more take the extension back off the board and put needs-human back when adding autopilot:continue fails, and treat any deciders.py exit other than 0 or 1 as "could not tell". Both fix the earlier settled thread. I found no blocker or major problem. The more job holds the same permissions as the claim job, runs no repository code, and reads the comment from the event file, not from ${{ }} in a script. A comment from someone who-decides does not allow changes nothing on the board. The end rule step runs with if: always(), so it does read refused=diverged after applied exits 1. receipts.py state set accepts the new extensions key, and None removes it on rollback. This looks like a new capability (a way past the runs cap): version:major only if the maintainer agrees. I did not run the tests; I read the diff and the code it relies on. Minors are below.

  • minor scripts/autopilot.py:6264: deciders.py and autopilot.py more may pick different pull requests when several open ones close the issue
  • minor scripts/autopilot.py:1692: A refused or below-cap /autopilot more from a decider still becomes a hint
  • minor .github/workflows/issue-claim.yml:556: GitHub's startsWith is case-insensitive and sees no leading whitespace, more_hint does
  • minor docs/agents.md:0: The description's Decisions list has a line cut off mid-sentence

Policy floor: high (touches high-risk paths: .github/workflows/autopilot.yml, .github/workflows/issue-claim.yml, CONTRIBUTING.md, docs/agents.md, policy/autonomy.yml …). Reviewed 0499cca (whole pull request; review 6 of 5 automatic). Author trusted for auto-merge: false.

🤖 AI review · claude-sonnet-5-5 (Anthropic) · $0.42, 22 turns

@dilux-bot dilux-bot Bot changed the title fix: the autopilot's cap has no way on but a claim, and lost runs count feat: the autopilot's cap has no way on but a claim, and lost runs count Nov 19, 2026
@soydiloreto
soydiloreto marked this pull request as draft November 19, 2026 08:35
@dilux-bot
dilux-bot Bot marked this pull request as ready for review November 19, 2026 08:46
dilux-bot Bot added 3 commits November 19, 2026 08:47
At the runs cap, the issue's comment now comes from autopilot.py cap-comment --issue (open_to_claim), from the same words as the pull request's (cap_way): it offers /autopilot more while an extension is left, and past the second names only /claim and a maintainer closing the pull request. OPEN_TO_CLAIM, for the rounds cap and scripts older than it, no longer offers /autopilot more, which gives nothing there.

The sweep's pr_board plumbing goes: autopilot-sweep.yml never fed it, and next_step's callers pass no runs, so the cap there is may-continue's.

The reply when who decides cannot be read carries its own marker, dx-autopilot-more-unread, and is said once, until another answer follows it.

answers() no longer restarts at a cap end, so a decider's hint written before the cap comment and the rest of a /autopilot more both reach the next run.

Assisted-by: Claude Opus 5.5 (Anthropic)
Comment thread scripts/autopilot.py Resolved
When autopilot:continue cannot be added to the issue, the extension comes off the pull request's board and needs-human goes back on it, so the same /autopilot more again extends it instead of reading it as below its cap. An exit of deciders.py other than 0 or 1 now reads as could not tell, never as a no whose marker silences later replies. Both with a test.

Assisted-by: Claude Opus 5.5 (Anthropic)
@soydiloreto soydiloreto added the review:full Ask the Claude review for a full pass on the next push label Nov 19, 2026
@dilux-bot dilux-bot Bot removed the review:full Ask the Claude review for a full pass on the next push label Nov 19, 2026
@soydiloreto soydiloreto added the review:full Ask the Claude review for a full pass on the next push label Nov 19, 2026
@dilux-bot dilux-bot Bot removed the review:full Ask the Claude review for a full pass on the next push label Nov 19, 2026
@soydiloreto soydiloreto added the review:full Ask the Claude review for a full pass on the next push label Nov 19, 2026
@dilux-bot dilux-bot Bot removed the review:full Ask the Claude review for a full pass on the next push label Nov 19, 2026
@soydiloreto
soydiloreto merged commit 264df31 into main Nov 19, 2026
97 of 99 checks passed
@soydiloreto
soydiloreto deleted the fix/856-the-autopilot-s-cap-has branch November 19, 2026 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autopilot Opened by the autopilot from an accepted issue complexity:high Set by the Claude review risk:high Set by the Claude review type:feat The kind of change, read from the diff by the Claude review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: the autopilot's cap has no way on but a claim, and lost runs count

1 participant