Custom processes: compose Steps from a catalogue in a file #877

Specified from the grilling in docs/grill-with-docs/877-composable-by-file.md (Q1.1–Q4.6), with research in docs/research/file-defined-factories.md and the decision in ADR-0024 (all in #1379). The original request was "like warp factory's factory.yaml and agent.md".

Problem Statement

Every kind of work thirdshift does is a Command fixed in code: a Ticket run, Spec run, Architect run, Pickup run or Security run. Each has its own sequence of agent sessions and Session prompts, and runs every session on one Harness, Model and Effort. To try a different way of working, such as three model families each running a custom bug-checking skill, a script between two sessions, or a grill, then a Spec, then a build, someone has to change thirdshift's Rust. They can't do it by describing the process themselves. Tools like Warp Factories and Archon let a file define the process. But a file that can declare everything could also turn a Self-merge into a merge with no checks, and the rules that keep unattended merging safe took many ADRs to get right.

Solution

A user defines a Custom process in a TOML file in ~/.thirdshift/processes/, and starts it with thirdshift process <name> [issue <issue>]. A Process is a named sequence of Steps. A Step is one of three things:

• an agent session running a named skill, on its own Harness, Model and Effort if it names them;
• a command or script;
• a catalogue Step whose rules stay in thirdshift: Delivery, Self-merge, or starting a Ticket run or Spec run on an issue an earlier Step published.

Steps run one after another or in parallel, each in a worktree the Process declares. A worktree is either throwaway, detached at the Base branch and discarded at the end, or on an Issue branch, holding a Claim and ending in Delivery. Each Step's result is passed on to the Steps after it. thirdshift process check <name> shows the plan as it would run without starting a session. Every run records one line per Step. A file can only tighten the limits the User config and the command line set, never loosen them (ADR-0024).

Today's run types become Built-in processes later, starting with the Ticket run, in a Spec of their own.

User Stories

1. As a thirdshift user, I want to define a Custom process in a TOML file in ~/.thirdshift/processes/, so that I can compose my own way of working without changing thirdshift's code.
2. As a thirdshift user, I want to start a Custom process with thirdshift process <name>, so that it starts the same way as other thirdshift commands.
3. As a thirdshift user, I want to pass issue <issue> (a number or an Issue URL) to a Custom process, so that its Steps work on that issue.
4. As a thirdshift user, I want thirdshift process with no name to list my Custom processes and their descriptions, so that I can see what I have.
5. As a thirdshift user, I want thirdshift process check <name> to validate the file and print the plan as it resolved it, without starting a session, so that I can find mistakes in seconds.
6. As a thirdshift user, I want check to show each Step's worktree, its Harness, Model and Effort, and where each of those came from, so that I know exactly what will run.
7. As a thirdshift user, I want check to show where each Step's skill was found, so that I know which copy of a skill will run.
8. As a thirdshift user, I want check to say whether each Step's Harness can start, so that a missing login shows up before a run.
9. As a thirdshift user, I want a refused file to name the rule it broke and the line it broke it on, so that I can fix it without guessing.
10. As a thirdshift user, I want unknown keys refused, so that a typo never silently changes what runs.
11. As a thirdshift user, I want every file to require version = 1, so that later changes to the format can tell me how to update my file.
12. As a thirdshift user, I want a Custom process with a Built-in process's name refused, so that a copied file can never quietly replace what a Pickup run does.
13. As a thirdshift user, I want a session Step to run a named skill, so that each Step's instructions live in a skill I can read and reuse.
14. As a thirdshift user, I want a Step's skill looked up among the repository's committed skills, my ~/.thirdshift/skills/, and the Factory skills, so that a Custom process can carry its own skills to every repository.
15. As a thirdshift user, I want a Step to name its own Harness, Model and Effort, so that one Process can use several model families.
16. As a thirdshift user, I want a Step that names none of the three to use the Command's, chosen as it is today, so that simple Processes need no per-Step settings.
17. As a thirdshift user, I want a Process to fail before any session if a Step's Harness can't start, so that a family I asked for is never quietly replaced by another.
18. As a thirdshift user, I want Steps to run in parallel, so that three bug checks finish in the time of one.
19. As a thirdshift user, I want each parallel Step to have its own throwaway worktree if it edits, so that parallel Steps can try changes without overwriting each other.
20. As a thirdshift user, I want a Process refused if parallel Steps share a worktree that one of them edits, so that concurrent edits can't collide.
21. As a thirdshift user, I want each Step's final message saved, and its location given to later Steps, so that a judging or fixing Step can read what earlier Steps found.
22. As a thirdshift user, I want a command Step that runs a command or script in its worktree, so that tests, file gathering or other deterministic work happen without spending tokens.
23. As a thirdshift user, I want a command Step's output passed on like a session's final message, so that a later agent Step can act on it.
24. As a thirdshift user, I want a command Step that exits non-zero to fail, so that a failing test stops the Process.
25. As a thirdshift user, I want the Process to stop at the first failed Step, so that later Steps never act on results that aren't there.
26. As a thirdshift user, I want to mark a Step optional = true, so that a family dropping out of a three-family check doesn't fail the whole Process.
27. As a thirdshift user, I want a parallel group to let its other Steps finish before it fails, so that their results are still recorded.
28. As a thirdshift user, I want a Process to declare named worktrees, so that I decide which Steps share a checkout.
29. As a thirdshift user, I want a throwaway worktree detached at the Base branch and discarded at the end, so that experiments leave nothing behind in my repository.
30. As a thirdshift user, I want an Issue-branch worktree, so that a Process can produce a pull request.
31. As a thirdshift user, I want a Process with an Issue-branch worktree refused unless it takes an issue and ends that worktree in Delivery, so that no work is lost when worktrees are cleaned up.
32. As a thirdshift user, I want an Issue-branch Process to hold a Claim on its issue as a Ticket run does, so that other runs and other machines leave the issue alone.
33. As a thirdshift user, I want a Delivery Step to open the pull request and run the Repair loop as a Ticket run's Delivery does, so that a Custom process gets the same mergeable, green pull request.
34. As a thirdshift user, I want a Self-merge Step that merges only when the command or the User config asks for a merge, so that a Process file can never turn merging on.
35. As a thirdshift user, I want a Process file to lower the Repair cap, forbid a Self-merge, or name protected paths, but never raise or loosen a limit, so that a Process copied from someone else can't turn off a safeguard my machine sets.
36. As a thirdshift user, I want a change touching a protected path to hold a Self-merge and leave the pull request ready for review, so that sensitive files always get a human look.
37. As a thirdshift user, I want Issue-branch work from a failed Process kept and named in the failure, as a Failed ticket run's is, so that I can take it over.
38. As a thirdshift user, I want every session Step that works on an issue to get the issue from thirdshift, with only Trusted account comments, and to be checked for untrusted edits before it starts, so that a Custom process is as safe as a Ticket run against steering.
39. As a thirdshift user, I want the last Step's final message printed to stdout, so that a quick Process works like thirdshift skill.
40. As a thirdshift user, I want every Step's message kept with the Command's records, so that thirdshift show and the Console can display what a cron-started Process found.
41. As a thirdshift user, I want one line per Step in the Command log, naming what ran, how long it took and whether it succeeded, failed or was skipped and why, so that trial and error is cheap.
42. As a thirdshift user, I want thirdshift to post nothing to GitHub for a Process itself, so that only Steps whose skills say so write to GitHub.
43. As a thirdshift user, I want a Custom process started with no issue to be a Pass, skipped when another Pass on the repository is still running on the machine, so that it can run from cron without piling up.
44. As a thirdshift user, I want a skipped Process to exit 0 and send no Run notification, as other Passes do, so that cron logs stay quiet.
45. As a thirdshift user, I want a catalogue Step that starts a Ticket run, or a Spec run for a Spec, on the issue an earlier Step published, so that a Process can grill, spec and then build, with parallel Ticket runs in separate worktrees.
46. As a thirdshift user, I want that published issue checked as an Architect plan is (this repository, open, by a Trusted account, created since the Step began), so that a Step can't hand thirdshift someone else's issue.
47. As a thirdshift user, I want Stop and Ctrl-C to interrupt a Process as they interrupt other Commands, so that I can stop one safely.
48. As a thirdshift user, I want the [resources] checks to apply before every session Step, so that a Process waits when my machine is busy.
49. As a thirdshift user, I want a Process's Command to appear in thirdshift status, the Console and the Activity log under its Process name, so that I can follow it like any other Command.
50. As a thirdshift user, I want a Process to send a Run notification on failure or success, following the same User config rules as other Commands, so that unattended Processes reach me.
51. As a thirdshift user, I want Session traces for every Step, so that I can inspect each session as I do today.
52. As a team, we want to keep ~/.thirdshift/processes/ and ~/.thirdshift/skills/ as clones of a shared git repository, so that we can review and share Processes without changing thirdshift.
53. As a site reader, I want thirdshift process documented next to thirdshift skill, marked experimental, so that I know it exists and that its format may still change.
54. As an author of a future Built-in process, I want the format and Steps to be the same for Built-in and Custom processes, so that the Ticket run can move into a file later.

Implementation Decisions

• New domain modules:
  • The Process definition. Reads one TOML file into a validated Process, or a refusal naming its rule and line. Only the thirdshift binary validates. check and running use the same code (research: validators that drift from the engine give confident, wrong diagnoses).
  • The Process runner. Runs the validated Process's Steps through what Commands already use: worktree acquisition and cleanup, Sessions with the session checks (trust, resources, Resume), Delivery, Self-merge, Claim, and the Architect run's dispatch of a Ticket run or Spec run.
  • It joins the existing Command lifecycle (Command log, Command record, interruption, Run notifications) as a new Command kind.
• CLI: thirdshift process lists, thirdshift process check <name> [issue <issue>] validates and prints the plan, and thirdshift process <name> [issue <issue>] runs. The issue takes a number or an Issue URL, resolved as thirdshift skill resolves it, origin-mismatch rule included. check and list are reserved and can't be Process names.
• File location and names: ~/.thirdshift/processes/<name>.toml. The name is the file stem. The Built-in names are reserved, including ticket-run, spec-run, architect-run, pickup-run and security-run, and a refused file says so. No folder in the target repository (Q2.1).
• File keys (the names may be adjusted in review, but the meaning is fixed):
  • Top level: version (required, must be 1), description, and optional limits: repair_cap (no higher than the effective cap), self_merge = false (forbids one), protected_paths (globs).
  • [worktrees.<name>] with kind = "throwaway" or kind = "issue-branch". A Process has at most one Issue-branch worktree.
  • [[steps]] in order, each with a name and exactly one of skill, command or catalogue (delivery, self-merge, run-published-issue).
  • Optional worktree, harness, model, effort and optional, plus parallel = "<group>". Consecutive Steps sharing a group run together.
• Validation rules (each named in refusals):
  • Unknown key or wrong version.
  • Reserved name.
  • A Step with zero or several kinds.
  • An unknown worktree.
  • An unknown Harness.
  • A skill not found.
  • An Issue-branch worktree with no issue at run time, or with no delivery Step after its last editing Step.
  • self-merge not directly after delivery.
  • Parallel Steps sharing a worktree that one of them edits. Any session or command Step counts as editing its worktree.
  • A limit that loosens.
  • run-published-issue not preceded by a session Step.
• Harness, Model and Effort: chosen for each Step, defaulting to the Command's, which is chosen as today (the command, then the User config, then the Harness default). Every distinct Harness gets the usual preflight before the first session, and one that can't start fails the Command with no fallback. When this ships, the glossary's "one per Command" for each becomes "one per Step, defaulting to the Command's".
• Skill lookup: the existing standalone-skill resolution (the exact name among the Factory skills and the repository's committed .agents/skills/ and .claude/skills/, refusing any that resolve outside the worktree). ~/.thirdshift/skills/<name>/SKILL.md comes after the repository and before the thirdshift-<name> Factory shorthand. Home skills are linked into the worktree as Factory skills are (ADR-0012).
• Session Step prompt: a new Session prompt, generated with the others (and so on the prompts page). It names the skill, the issue if any, the worktree's kind, and the paths of earlier Steps' results. A Step on an issue gets .thirdshift-issue.md and the trust checks before every session, Resume included, as a Ticket run does.
• Command Steps: run with the worktree as working directory. Environment variables give the issue URL (when there is one), the results folder, and the Step's own result file. A non-zero exit fails the Step. Stdout and stderr together become the Step's result.
• Results: each Step's final message or command output is stored with the Command's logs, one file per Step, outside every worktree. The last Step's result goes to stdout. A parallel group's results are each passed on.
• Failure: the first failed required Step stops the Process. A parallel group finishes its running Steps first. An optional Step's failure is recorded and the Process carries on. A failed Process with an Issue-branch worktree keeps its work the way a Failed ticket run does.
• Issue-branch worktrees: created and named as a Ticket run's, with a Claim from start to end and Continuation rules unchanged. delivery is the existing Delivery. self-merge is the existing Self-merge, done only when the command (merge flag) or the User config's merge_always asks for one, and held by a protected-path change or by self_merge = false, leaving the pull request ready for review.
• No issue means a Pass: a Process started with no issue follows the Pass rule of one running Pass per repository on a machine. It is skipped with exit 0 and no Run notification otherwise. It never takes Ready-issue or Architect-idea gates.
• run-published-issue: the preceding session Step's final message ends with the issue's URL on its last line, as an Architecture review names its plan. thirdshift checks it as it checks an Architect plan, then starts a Ticket run, or a Spec run for a Spec, as part of this Command (a child run, not a Command of its own).
• Records: a new stored Command kind, Process, with the Process name. ADR-0022's stored kinds are unchanged. Displays say "Process <name>". Endings are succeeded, failed, skipped and interrupted. One Command-log line per Step gives its name, kind, worktree, Harness/Model/Effort, duration, outcome and the cause of any failure.
• Site: site/index.html and site/how-it-works/ mention thirdshift process next to thirdshift skill, marked experimental, and docs/guide.md gains a Custom processes section with an example file.

Testing Decisions

• One seam: the thirdshift binary, run through the existing tests/support Scenario with real Git, the fake gh, and the fake Claude, Codex, agy, Grok, Muse and OpenCode Harnesses. There are no new test seams. check exercises every validation rule cheaply because it starts no session.
• Good tests assert what the user sees: exit code, stdout and stderr, the Command log's Step lines, which fake Harness was called with which model and prompt, the issue and pull request state on the fake gh, and what's left in the worktree list. They don't assert internal types.
• Prior art:
  • tests/skill.rs for skill lookup, standalone sessions and worktree cleanup.
  • tests/harness_choice.rs and tests/harness.rs for Harness, Model and Effort choice and preflight.
  • tests/merge_run.rs and tests/fresh_run.rs for Delivery, Self-merge and the Claim.
  • tests/architect_run.rs for dispatching a published issue.
  • tests/pickup_run.rs and tests/security_run.rs for Pass skipping.
  • tests/failed_run.rs for kept work.
  • tests/run_notification.rs, tests/status.rs and tests/console.rs for records and displays.
• Scenarios to cover:
  • The three-family bug check: three parallel session Steps on Claude, Codex and agy, each in its own throwaway worktree, then a judging Step that gets their result paths.
  • A command Step whose non-zero exit stops the Process, and an optional one that doesn't.
  • Every validation rule through check, with the rule and line in the output.
  • A Harness that can't start, failing before any session.
  • An Issue-branch Process reaching a ready pull request, and a Self-merge only when asked.
  • A protected-path change holding a Self-merge.
  • No issue: skipped while another Pass runs.
  • run-published-issue starting a Spec run whose Tickets run in parallel.
  • An untrusted issue edit failing a Step before its session.
• Final validation: run scripts/affected-tests main. Regenerate the prompts page with UPDATE_PROMPTS=1 cargo test prompts_page once the new Session prompt exists.

Out of Scope

• Moving the Ticket run, or any other run type, into a Built-in process. That is a later Spec (Q1.5, Q4.6 slice 5), which needs catalogue Steps for bug reproduction, Continuation, Resume and the Repair loop.
• A processes folder in the target repository (Q2.1).
• Triggers in the file, or thirdshift installing crontab lines (Q1.4, ADR-0009).
• Typed outputs and conditions between Steps (Q2.7). Steps pass whole results.
• Any setting that loosens a User config or command-line limit, or lets a file decide to merge (ADR-0024).
• Overriding a built-in's prompts, which is Put in thirdshift configuration the prompt and skills so that users can tweak the format of those. #118.
• Testing a Process with fixtures and no agent (Archon's workflow test). That needs a separate issue.
• Multi-family review: have another model family review a Run's work #429's Second opinion, which keeps its own plan (comment on Multi-family review: have another model family review a Run's work #429).

Further Notes

• The slicing agreed in Q4.6:

  1. The format, check, the CLI, session and command Steps, throwaway worktrees, parallel groups, failures and results.
  2. Issue-branch worktrees with the Claim, Delivery, Self-merge and the tightening limits.
  3. No issue means a Pass.
  4. run-published-issue.

  /to-tickets turns these into Tickets. None needs a release of an earlier Ticket, so the Spec doesn't need splitting at a release.

• Glossary terms: Process, Built-in process, Custom process and Step (Grill #877: Processes composed from a catalogue of Steps #1379).

• The format is a public interface from its first release, since anyone can write a Custom process, but the site marks it experimental until the Ticket run has moved into it (Q2.10).