The stack I run AI research agents on: the map, the principles, and the machinery, in one place.
Install it in Claude Code, one line:
/plugin install rstack --marketplace rpinho/rstack
Or in two steps, which works on older versions: /plugin marketplace add rpinho/rstack, then
/plugin install rstack@rstack. From a shell: claude plugin marketplace add rpinho/rstack && claude plugin install rstack@rstack.
The plugin gives you /rstack:onboard, the orchestrator and voice skills, and the guard hooks in
rstack/hooks/ (each refuses with a reason; none sends anything anywhere). The
research harness itself is a clone and an install script, below.
New to coding agents? You will not trust the AI from day one, so you have to start small. At first it does stuff bad, and then it starts doing stuff better, and your trust meter goes up. rstack is the harness I built over time around how I work: text files, Python scripts, shell scripts and hooks, with a yes or no gate for a lot of stuff that I do. They're not perfect, but you teach the AI how to catch itself. Most of this repo is more than you need. Install the plugin, or clone the repo and run claude in it, and
type /onboard (/rstack:onboard from the plugin). It asks whether you're a beginner, intermediate or advanced, checks that with three
questions, and walks you through the part that fits: beginner,
intermediate, advanced.
Agents that do real work accumulate machinery. Guards, schedulers, linters, memory, publishing steps. The failure mode is not ignorance; it is changing the wrong part of the system: adding a note when what was needed was an automatic check, editing an installed copy when the source is a repository, writing a rule in prose when the same sentence could be a program that refuses. rstack says which layer is responsible for a problem, and keeps an honest account of which rules have something enforcing them and which are still only prose.
It has produced real results, and the short list is near the bottom.
How it works, what it found and what still doesn't work, with a source for every number: PAPER.md.
Named after, and owed to, pstack, which is where the idea of a single stack-wide entry point came from.
| Layer | What it owns |
|---|---|
| Orchestration | one conductor, many workers; a wake is a deliberate decision, not a side effect |
| Guards | a guard refuses rather than crashes, and ships with the test that proves it |
| Outbound communication | linted before it leaves, approved as the exact bytes that will be sent |
| The work itself | frozen tests, independent scoring, a yield check, baselines you can run |
| Publishing and egress | nothing leaves unscanned; an agent whose output leaves has an empty identity |
| Memory | a worktree per session, then land; one fact per file, with an index |
| Scheduled jobs | wait on an event instead of polling; heavy compute goes elsewhere |
| Session mechanics | context limits and written handoffs instead of silent compaction |
Three parts of that live in this repository. Until recently they were three repositories.
harness/: bounded agents on open research problems. It lets you fix the test and the pass mark before the first result, keep final scoring away from the agent doing the searching, make reviewers answer a set list of questions with evidence, stop a line of work that spends without finding anything, and require a published result to be reproduced by a command before it counts as a baseline. Several checks are opt-in and the isolation is yours to supply, not the harness's to create; its Limits section says which. A lane can require a Claude Science review before a result counts (claude_science_required), andharness/cs.pydrives Claude Science from the command line so agents can send it proofs, certificates and plans to referee.kb/: a file-based knowledge base an agent maintains over years. Each session works in its own copy of the repository and merges when it is done, so two agents never fight over the same file. The memory is the filesystem, not a context window.SKILL.mdandrstack/: the map itself. The layer manifest, the principles, and two scripts:inventory.pylists what exists, andcoverage.pychecks the map and the enforcement claims in it.rstack/hooks/andrstack/tools/: the guards and tools themselves, made generic. Seven hooks that refuse rather than crash, an event waiter that replaces polling, a relay that wakes the conductor only for decisions, an egress scan, a clean-identity runner and a lint for the files every session rereads. Each ships with the test that proves it refuses what it should. None ships a term list; you supply yours.
rstack/STACK.md is the system they came from, counted on 13 November 2026:
503 items across 25 hooks, 80 launch agents, 23 lints, 19 skills and 348 memory rules, every one
mapped to a layer. 131 of the 356 rules have something enforcing them; 225 are still prose, and
the page lists the ones broken most. It also goes through the main hooks and scheduled jobs with the
failure that made each necessary.
When a rule keeps breaking, it moves up a rung, after poteto's hierarchy. See architecture, not prose.
Point a coding agent at SKILL.md; that is the entry point and it explains how to route
a task to a layer. Then build a map of your own setup. The manifest shipped here is an example with
invented data, because a real manifest names your jobs, your notes and your rules, which is not a
list anyone should publish.
python3 rstack/coverage.py --config rstack/layers.example.json # see it work on the example
cp rstack/layers.example.json layers.json # then edit `sources` and `layers`
python3 rstack/inventory.py --config layers.json # what exists, names only
python3 rstack/coverage.py --config layers.json # mapped? enforced? FAILs if not
python3 rstack/coverage.py --config layers.json --top 10 # the prose-to-checks queueAs shipped, coverage.py exits 1 with exactly one failure: an example guard that no rule claims.
That failure is the tool doing its job on the example configuration. It also
refuses a catch-all rule, a rule that matches nothing, and a rule pointing at an enforcer that does
not exist, because each of those is how a map starts lying.
inventory.py reads disk and has no opinions. coverage.py has all of them. Keeping those apart is
deliberate: a map written by the same pass that grades it cannot be checked.
One file each in rstack/principles/, with the evidence behind it and the
mechanism that enforces it, or an admission that there isn't one.
- Results, not activity: every easy check measures activity instead. One line of work ran 38 workers for two days, wrote 528 claims, had its reviewer pass 98% of them, and produced nothing worth sending, with every check green.
- The producer never grades itself: a search that can see the scorer optimises the scorer.
- Prior art first, and run it: reproduce the published number with a command; do not quote it. Three of four candidate reports from one lane had already been published by other people.
- Every deletion is reversible: trash, not delete; verify a copy exists before removing anything you did not just create.
- Architecture, not prose: when a rule is broken twice, build a mechanism, not a firmer sentence.
- Short asks to the human: a question they can answer, where they actually read.
- The goal must not become process: gates counted as progress. The word cap meant to prevent this got it wrong twice in one day.
- Build a guard only after a real failure that cost something : not a hole someone could imagine using. This one names its own incident: a patch of mine that answered two things which had never happened.
- A guard must refuse, not crash: most hook systems block only on a specific exit code, so a guard that raises is no guard, and it fails silently. Five of twelve audited guards did exactly that.
- A session's context is a resource: content that breaks a session, written into the files that session rereads, breaks it again on every start. Compaction does not help, because the summary is rebuilt from those same files.
- A failed read is never all clear: a watchdog that cannot read a source says so loudly. One read failed for eight hours and every lane read as fine.
- A failure that only reaches a log has not been reported:
a step crashed every round for two days and only its log knew; engagement fell from 77 likes a day to 5.
job-healthputs failed jobs and stale outputs in front of the conductor. - Degrade, never suppress: ship a flagged partial result, never nothing. Prose for now.
- Verify before a destructive claim: "unused" cites the check that showed it, or says it is an assumption.
- Close every loop: name the consumer before building the producer, and measure a loop by what reached its end. Lane reports now cannot stop at the conductor.
- Reports outlive sessions: a result is a file the sweep reads, not a message a live session was going to send.
- Refresh inputs before acting: a snapshot older than a day, or than the last change to what it describes, is re-read from its source first.
- Revert first, then reorder: after three forward fixes on one target, go back to what worked and run the proven path first. Prose for now.
- Autonomous subsystems refill themselves: one alert when every automatic source is empty, never a routine ask. Prose here; the laptop checks it.
- Change direction after a dry streak: three tries with nothing usable, then a different method and the dead end written down.
- Reciprocate: a reply mirrors the message it answers, in length, greeting, warmth, questions and language. The outreach lint checks each one.
- Instructions name only what exists: agents follow a stale instruction file over the code about 90% of the time, so every path, link, command and option an instruction file names is resolved by a test.
The last two cost the most to learn. Read them before adding anything.
All of it unrefereed by a human expert unless stated. harness/README.md has the full account, with every number, link and correction.
- Mathematics. Six bounds merged into Terence Tao's public table of constants from extremal problems, sixteen more pull requests open. Certificates and independent checkers at math-certificates.
- Coding theory. Six linear codes over GF(5) and GF(9), each one above the current lower bound, sent to the codetables.de maintainer on 6 November 2026; submitted, not yet accepted.
- Exoplanets. Four candidate reports from NASA TESS data under review at RNAAS, all four corrected after a re-audit, corrected versions on Zenodo. Pipeline: planet-hunt.
- Collected where each field can find it. Two public hubs gather the work and the code behind it: ai-biology (protein design, the Arc Virtual Cell and Virtual Embryo challenges, open drug-target discovery, and refereed bug reports on scientists' published software) and ai-physics (certified bounds on published results, exact checks with independent referees, and bug reports on physics software). Both went public on 7 November 2026. They are where the results above are indexed, not results in their own right.
- The negative results were the useful ones. In two protein design challenges the structure prediction models' own confidence scores could not rank designs; in a cell biology challenge a local score ranked two candidates in the opposite order to the official scorer.
rstack is not built from scratch. What each source gave it:
- Lauren Tan (@poteto), pstack — github.com/cursor/plugins/tree/main/pstack and her guides to it (post): the single stack-wide entry point, and the instinct to make a rule a lint rule instead of prose. rstack is named after it.
- Lilian Weng, Harness Engineering for Self-Improvement (2026) — the field's literature review, and the open challenges the harness docs answer: negative results, weak evaluators, context lifecycle.
- Cynthia Dwork et al., Generalization in Adaptive Data Analysis and Holdout Reuse (2015) — why a reused holdout stops being evidence; the harness records a holdout disposition.
- Avrim Blum and Moritz Hardt, The Ladder (2015) — how repeated public feedback overfits a leaderboard.
- Google DeepMind, AlphaEvolve (2025) — related work on proposing programs and scoring them automatically.
- Edwin B. Wilson (1927), the score interval — the role ranker ranks on its lower endpoint, z = 1.96.
- Hugo Fernandes (2026-11-03) — the ten anti-overfitting practices mapped in docs/overfitting.md.
- Comparable systems, as related work rather than inputs: AI Scientist, aviary, ScienceAgentBench, MLE-bench, CORE-bench.
- The external tables its results were merged into: Tao's table of optimization constants and codetables.de.
docs/credits.md states at length what is and is not implemented here.
MIT. The design owes a debt to published work on holdout reuse and leaderboard reliability; sources and attribution describe the relationship, and harness/README.md credits the specific papers.

