Skip to content

Agent pipelines, an opt-in memory graph, and scheduled runs - #791

Open
DanilaZanin wants to merge 12 commits into
xintaofei:mainfrom
DanilaZanin:pr/agent-pipelines
Open

DanilaZanin wants to merge 12 commits into
xintaofei:mainfrom
DanilaZanin:pr/agent-pipelines

Conversation

@DanilaZanin

Copy link
Copy Markdown

One agent in a chat handles a targeted edit well. A larger task usually wants a
shape: plan it, write it, review it, send the review back to whoever wrote the
code. Today that means driving every hand-off by hand, and @ delegation only
gets you part of the way, because nothing brings the result back.

This adds a pipeline engine that runs those chains, an editor for them in the
composer, and an opt-in memory graph. Everything is off by default: Codeg still
opens in single-agent mode with memory disabled, and the existing chat is
untouched.

docs/agent-pipelines.md is the user-facing guide.

The engine

A linear chain with bounded back-edges. Two chains ship: Duet
(coder, reviewer) and Team (planner, coder, reviewer, tests), both looping
back to the coder for up to three fix rounds.

  • Every step carries its role, agent, model, timeout, prompt template and a
    read-only flag. Prompts substitute $task, $plan, $summary, $review
    and $memory.
  • A read-only step that edits the tree is caught: the engine hashes the
    worktree around it and downgrades the verdict to inconclusive, naming the
    paths that moved. Such a step is also told the rule in its prompt, which it
    previously was not.
  • Status transitions are compare-and-swap, each attempt carries its own id, and
    runs still open after a restart are closed as interrupted.
  • Graphs are validated up front: no cycles, no unknown targets, at most eight
    steps, one to ten iterations.
  • Steps launched by the engine answer their own permission prompts. Without
    this they simply sat until the step timed out, because there is no user
    attached to a pipeline step.

Verdicts

A reviewer or test step reports through the pipeline_verdict MCP tool; notes
are required when it asks for changes. When the tool is unavailable, a
VERDICT: marker in the agent's own output is parsed instead.

Isolation

Each run works in its own git worktree on a temporary branch, and lands through
a squash or a merge commit when the user applies it. Apply refuses when the
project has staged changes of its own rather than folding them into a commit
the user did not write.

Editing a chain

The role chips in the composer are the editor. A chip opens two lists, agent
and model; the models come from a live probe of that agent rather than a
hardcoded table, and a saved id the agent no longer lists stays visible instead
of being silently swapped for the first option. Agent and model move together,
since the old agent's model id almost never exists on the new one.

Editing Duet or Team stores an override of the built-in chain, with a Reset
beside the chips once one exists. The schema already carried preset_key, its
partial unique index and a save_preset function that nothing called; this
connects that seam.

Custom mode gains a + that asks for the role first, which also decides where
the step lands: a planner goes to the head, since it exists to brief the steps
after it.

While a pipeline mode is active the composer hides its own model and reasoning
pickers. The model belongs to each step now, and a second control for it would
point at a session the run does not use. The edit-permission mode stays: it
matters more once several agents write, not less.

The canvas keeps the full step editor (prompt, timeout, read-only, loop target)
on a pipeline card.

Automations

A run_pipeline action, so a chain can run on a schedule or on demand.

Memory (opt-in, off by default)

Three backends: off, a local SQLite graph with FTS5 search and two-hop edge
traversal, or an external MCP server.

What gets remembered is the user's choice: built-in kinds (decision, fixed bug,
task summary, preference) plus their own, each set to automatic, on request or
off, scoped to the project or global. memory_write, memory_search and
memory_link are exposed only while memory is on, everything written passes a
secret redactor first, and injected context is wrapped in
<memory untrusted="true">.

Bugs found along the way and fixed here

Most of these surfaced while running the feature against a real repository
rather than in tests.

  • A draft was lost when creating a conversation from the canvas failed, and
    when a tab was switched before the debounce flushed.
  • Sending while a conversation was still being created dropped the message.
  • A delegated task could be completed by a stale child connection.
  • working_dir from tool arguments was not checked for being an absolute,
    existing directory.
  • Permissive CORS applied to every route rather than the public ones, and
    backup uploads had no size cap.
  • Adding a pipeline card on the canvas always failed: the service requires
    pipeline_id for that kind and the create command hardcoded None.
  • The canvas step inspector wrote to React state only, so reassigning a step to
    another agent looked applied, was gone on reopen, and the run used the old
    chain.
  • Apply with the squash strategy only staged the merge, then deleted the
    run's branch and worktree, leaving the work as an uncommitted diff with its
    only record gone.
  • The read-only guard always reported "reviewer modified files" whichever step
    it was and whatever it touched. In practice the culprit is often a
    session-start hook or a build cache the repo does not ignore, which the old
    message gave no way to discover.
  • The composer looked its presets up by display name while the API returns them
    keyed by preset key, so the role chips never rendered at all.
  • fitView on the canvas card only ran for the first render, so a newly added
    step landed outside the card with nothing to show it existed.

Testing

The full gate passes: lint, the frontend suite, the static export, and
cargo check / test / clippy for codeg, codeg-server and codeg-mcp.

New coverage includes an end-to-end duet run driven through the real engine
(loop-back then pass, the fix-round limit, and an inconclusive review), the
preset overlay and reset, the composer chip editor, the graph edit helpers, and
the read-only instruction staying in step with the flag.

Beyond the automated gate, the duet was run against a live repository end to
end: planner, coder on a different vendor's model, reviewer reporting through
the MCP tool, then Apply landing a real commit whose asserts fail on the old
implementation.

… runs

Single-agent chat handles a targeted edit well, but a larger task wants a
shape: plan, write, review, test, and send the review back to the coder. Until
now that meant driving every hand-off by hand.

This adds a pipeline engine that runs those chains itself.

Engine (src-tauri/src/pipeline/)
- A linear step chain with bounded back-edges. Two presets ship: Duet
  (coder -> reviewer) and Team (planner -> coder -> reviewer -> tests), both
  looping back to the coder for up to three fix rounds.
- Each step carries its role, agent, model, timeout, prompt template and a
  read-only flag. Prompts substitute $task, $plan, $summary, $review, $memory.
- A read-only step that edits the tree is caught: the engine hashes the
  worktree before and after and downgrades the verdict to inconclusive.
- Status transitions are compare-and-swap, every attempt carries its own id,
  and runs still open after a restart are closed as interrupted.
- The graph is validated up front: no cycles, no unknown targets, at most
  eight steps, one to ten iterations.

Verdicts
- The pipeline_verdict MCP tool reports pass, changes_requested or
  inconclusive; notes are required when changes are requested.
- When the tool is unavailable, a VERDICT: marker in the agent's own output is
  parsed instead.

Isolation
- Each run works in its own git worktree on a temporary branch, and lands
  through a squash or a merge commit when the user applies it.

Interface
- A mode switch in the composer (single, duet, team, custom) with role chips,
  remembered per folder.
- A run card in the feed with step progress, round count and verdicts.
- A "chat + code" diff panel: file tree, unified diff, line notes, and the
  actions to send those notes back for another round, stop for a manual fix,
  or apply the result.
- A pipeline node on the canvas with a step inspector.

Memory (opt-in, off by default)
- Three backends: off, a local SQLite graph with FTS5 search and two-hop edge
  traversal, or an external MCP server.
- What gets remembered is the user's choice: built-in kinds (decision, fixed
  bug, task summary, preference) plus their own, each set to automatic,
  on request, or off, scoped to the project or global.
- memory_write, memory_search and memory_link are exposed only while memory is
  on, and everything written passes a secret redactor first.

Automations
- A run_pipeline action, so a pipeline can run on a schedule.

Bugs found along the way and fixed here
- A draft was lost when creating a conversation from the canvas failed, and
  when a tab was switched before the debounce flushed.
- Sending while a conversation was still being created dropped the message.
- A delegated task could be completed by a stale child connection.
- working_dir from tool arguments was not checked for being an absolute,
  existing directory.
- Permissive CORS applied to every route rather than the public ones, and
  backup uploads had no size cap.
Picking "Custom" rendered no role chips, because the composer never loaded
the pipeline it was about to run. Mixing agents (one writes, another reviews)
was invisible until the run had already started. The composer now loads the
same pipeline the send path picks and hands it to the mode switch.
Two holes closed, both found by trying to reassign a step to another agent:

- Adding a pipeline card always failed. The service requires pipeline_id for
  that kind, and the create command hardcoded None, so the card could never be
  created at all. The field now travels from the caller, and the menu creates
  a pipeline from the duet preset first so a new card opens on a working chain.
- The step inspector wrote to React state only. Changing a step's agent or
  model looked applied, was gone on reopen, and the run used the old chain.
  Every edit now goes through one commit point that writes the row back.
…mit the squash

All three came out of running a real duet against a real repository.

- A read-only step was policed but never told: the engine hashes the worktree
  around it and kills the run when files moved, while the prompt said nothing
  about not editing. The planner rewrote the file it was meant to plan for and
  the run died. Every read-only step now carries the instruction, and a test
  keeps the flag and the wording in step.
- The guard's note always read "reviewer modified files", whichever step it
  was and whatever it touched. It now names the step and lists the paths — in
  practice the culprit is often a session-start hook or a build cache the repo
  does not ignore, which the old message gave no way to discover.
- Apply with the squash strategy only STAGED the merge, then deleted the run's
  branch and worktree: the work survived as an uncommitted diff with its only
  record gone. It now commits, like the merge-commit path already did, and
  refuses to run at all when the project has staged changes of its own rather
  than sweeping them into a commit the user did not write.
The model was a free text box next to an agent dropdown: assigning a step to
another model meant knowing that agent's internal id by heart, and a typo only
surfaced when the step ran. The inspector now probes the selected agent the
way the delegation defaults already do and offers its own list, reading the
grouped options too (Antigravity publishes its models only in groups). Agents
that advertise no models, and a probe that fails, keep the text box.
Steps run in array order and the only way to add one appended it to the end,
so a planner could never be put in front of a coder that already existed —
the chain could be built in exactly one direction. The inspector now moves a
step one place either way. A move that would turn a fix-round loop forwards is
refused instead of quietly dropping the loop.
fitView on the inner flow only runs for the first render, so "Add step" put
the new step outside the card with nothing to show it had been created.
The duet and team chains were compiled in and could never be edited, while
the DB already carried a preset_key column, its partial unique index and a
save_preset function that nothing called. This connects that seam: the preset
list now overlays saved overrides on the built-ins, and a new command saves or
resets one. A read that fails propagates rather than silently handing back the
built-in chain, which the next save would have overwritten.
Re-pointing a step at another agent or model meant leaving the chat, opening
the canvas and finding the card — for the two chains the composer offers by
name it was not possible at all. The role chips are now the editor:

- A chip opens a small popover with two lists, agent and model. The models
  come from a live probe of that agent, not a hardcoded table, and a saved id
  the agent no longer lists stays visible instead of being silently swapped
  for the first option. Agent and model move together, because the old
  agent's model id almost never exists on the new one.
- Duet and team save as an override of the built-in chain; a Reset appears
  once one exists. A custom chain saves as itself.
- Custom mode gains a "+" that asks for the role first, which is also what
  decides where the step lands: a planner goes to the head, since it exists to
  brief the steps after it. Deleting is inside the popover rather than an x on
  the chip, and the last step cannot go.
- In a pipeline mode the session-wide model and reasoning effort disappear
  from the composer: the model now belongs to each step, and a second knob for
  it would point at a session the run does not use. Everything else stays —
  the edit-permission mode matters more once several agents write, not less.

The fix-round counter stopped being chip-styled: the chips beside it open on
click and it has nothing to open.
Editing a built-in chain from the composer saved correctly and the chip went
on showing the untouched preset, so the change looked like it had not
happened. The switch preferred its own preset fetch everywhere except custom
mode; whatever the caller passes is the chain that will run, so it wins in
every pipeline mode now.
The old page read like reference material generated from the models: it led
with the data structures, described the composer as it was before the chips
became editable, and left a stray JSON fragment mid-section. Rewritten around
what someone actually does, with the reference kept at the end.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant