Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions docs/incidents/001-dirty-tree-residue-swept-into-commit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Incident 001 — prior-cycle residue swept into a live commit by an agent's fallback path

**Source:** a public GitHub issue filed by a maintainer against their own AI-agent-driven commit pipeline, describing a real shipped defect. Anonymized here at the maintainer's-consent bar (no consent on file yet) — no repo, org, or handle named. Reconstructed independently and run through getAdvantage's live gate; not the maintainer's own words beyond the class of failure.

## What happened

An agent-driven pipeline runs a multi-step cycle: plan, implement, then commit. A prior cycle had been aborted partway through — it left the working tree dirty: a test file gutted from hundreds of lines down to a handful, and an unrelated view edit that was never finished or reverted.

The next cycle started on top of that dirty tree. Its own changes were in scope and correct — a later transcript audit confirmed the cycle's own diff touched only the files it was supposed to touch. But the automated commit step hit a conflict ("patch does not apply") against the leftover residue and fell back to a broad `git add -A`-style commit. That swept the gutted test file and the unrelated broken edit into the same commit as the cycle's real work.

CI ran green — the reduced test file still passed, it just tested less. The commit merged. The unrelated broken edit shipped as a live regression that caused data loss, and was only caught and hand-fixed after the fact.

The maintainer's own root-cause note: an older, human-driven version of this pipeline used to abort outright on a dirty working tree. That doctrine did not carry over when the pipeline was mechanized — nothing re-asserted it, so nothing stopped the fallback from running on a dirty tree.

## Reconstruction: what the gate does with this

Reproduced the shape of the failure locally: a clean initial commit, then a file modified in place without being committed or staged for review (standing in for the prior cycle's residue), then ran `getadvantage check` cold, exactly as it would run in CI or a pre-commit hook.

```
✗ Dirty-tree guard — 1 tracked file modified/staged — a 'vercel --prod' would ship this unintended work.
M test/sample.test.js
Commit, stash, or revert before shipping. Deploy from a clean detached worktree of the intended commit.

Verdict
✓ 6 ⚠ 1 ✗ 1 – 2 skipped

NO-GO — 1 blocking issue. Do not ship until these are clear.
```

Wired into the commit-pr entry point (the exact point this incident's fallback ran), this is a hard halt: NO-GO, before the sweep, before CI, before merge. It names the modified file directly, so the operator sees the residue instead of a green check hiding it.

## What it would NOT have caught

Being direct about the edges, because a gate that only advertises catches isn't trustworthy:

- **A residue file that gets legitimately committed first.** If the leftover edit had landed in its own small commit before the next cycle started, the tree reads clean at the next check — the guard checks working-tree hygiene at the moment it runs, not the history of how a file got there.
- **A logic bug inside a file that was always meant to change.** The guard flags *unexpected* modifications; it has no opinion on whether an intended, cleanly-committed change is correct. The dirty-tree class and a bad-logic class are different failures — this incident happened to be the first, not the second.
- **Anything outside where it's wired in.** The gate only stops a ship if something actually calls it — ad hoc, uninstalled-in-CI runs give zero protection. This is the exact gap [invisible-mode / hook installs](../launch/SOCIAL-MEDIA-PACK.md) closes: the check runs automatically on every agent session instead of depending on someone remembering to call it.
- **A deliberate, reviewed override.** getAdvantage supports a tracked ignore path for a specific finding — that is a feature (disclosed, reviewable, never silent), but it also means a rushed or careless override can wave through the exact thing the guard was built to stop. The gate makes the override visible; it doesn't prevent someone from clicking past it.

## Try it

`npx getadvantage check` on your own repo takes under a minute and reads nothing but your local tree — no network, no account. If it mis-fires on your project, open an issue; fixed within a day.
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Incident 002 — an agent's git integration force-pushed unconditionally; only git's own refusal on a dirty tree stopped data loss

**Source:** a public GitHub issue filed by a maintainer against their own AI-agent framework's git integration, describing a real incident the maintainer traced by transcript. Anonymized here at the maintainer's-consent bar (no consent on file yet, no live touch conversation open on this thread) — no repo, org, or handle named. Reconstructed independently and run through getAdvantage's live gate; not the maintainer's own words beyond the class of failure.

## What happened

An agent framework's git integration resets and force-pushes a working tree as part of its normal session lifecycle, with no read-only or dry-run mode and no check for local uncommitted work before doing it. The maintainer's own transcript-verified account: this ran three times in one session. Each time, the only thing that stopped it from actually destroying data was git's own built-in refusal to `checkout` over a dirty working tree — a safety net that belongs to git, not to the agent tool. A directory that happened to have a clean tree, or committed state instead of loose local edits, would not have been protected at all.

## Reconstruction: what the gate does with this

Reproduced the shape of the failure locally: a clean initial commit, then a local edit made and never committed (standing in for the loose session work that was one force-push away from being gone), then ran `getadvantage check` cold — exactly as it would run before an agent's git step, if wired into a pre-flight hook.

```
✗ Dirty-tree guard — 1 tracked file modified/staged — a 'vercel --prod' would ship this unintended work.
M work.js
Commit, stash, or revert before shipping. Deploy from a clean detached worktree of the intended commit.

Verdict
✓ 5 ⚠ 1 ✗ 1 – 3 skipped

NO-GO — 1 blocking issue. Do not ship until these are clear.
```

Run before the agent's git step instead of after, this turns "silently protected by luck, this one time" into a named, visible halt: NO-GO, the modified file listed by name, before any reset or force-push runs.

## What it would NOT have caught

Being direct about the edges, because a gate that only advertises catches isn't trustworthy:

- **A tree that happens to be clean when the destructive command runs.** The maintainer's own account says the tree was dirty each of the three times this ran — that is what git's own refusal caught. If the same force-push had run against a directory with no loose edits (work already committed, or nothing pending), the dirty-tree guard has nothing to flag and would not stop it. The gate protects uncommitted local work specifically; it does not make a force-push or reset itself safe in general.
- **The destructive git command running before anyone calls the gate.** getAdvantage is a pre-flight check, not a runtime interceptor of git commands — it does not sit in the path of `git push --force` or `git reset --hard` and block them directly. It only helps if something calls `getadvantage check` before that step executes. An agent tool that force-pushes as its very first action, with no gate wired into that entry point, gets zero protection from installing getAdvantage elsewhere in the repo.
- **Anything outside where it's wired in.** Ad hoc, uninstalled-in-the-agent's-own-lifecycle runs give no protection at all. This is the exact gap [invisible-mode / automatic hook installs](../launch/SOCIAL-MEDIA-PACK.md) is built to close: the check running automatically on every agent session, including right before a git step, instead of depending on the tool's own authors remembering to call it.
- **Whether the force-push target was the right one.** The gate reads working-tree hygiene, not intent — it has no opinion on whether resetting or pushing to a given branch was the correct operation in the first place. A clean-tree force-push to the wrong branch is a different failure this class of check was never built to catch.

## Try it

`npx getadvantage check` on your own repo takes under a minute and reads nothing but your local tree — no network, no account. If it mis-fires on your project, open an issue; fixed within a day.
39 changes: 39 additions & 0 deletions docs/incidents/003-committed-env-secret-leak.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Incident 003 — a committed .env ships a live key, and the scanner only catches the one it recognizes

**Source:** composite, built from a public industry statistic, not a single maintainer's incident. GitGuardian's *State of Secrets Sprawl 2026* report (cited via Docker's engineering blog, published 2026-07-28) found that code written with an AI coding agent leaks credentials at roughly double the rate of code written without one. No team or repo is named here; this is a reconstruction of the failure class the statistic describes, run through getAdvantage's own gate.

## What happened

The pattern behind that 2x number is simple and repeats across teams: an agent scaffolds a project, drops a working `.env` next to the code so the app runs locally, and either forgets the `.gitignore` line or adds it one commit too late. `git init && git add -A` on cycle one sweeps the file in before anyone thinks to check. The repo now ships a live database URL and one or more provider keys to every clone, and stays in git history even if someone deletes the file later — `.gitignore` only stops new untracked files, it has no effect on what git already tracked.

## Reconstruction: what the gate does with this

Built a minimal repo with a `.env` holding three realistic fixture values — a Postgres connection string, a Stripe live-shaped secret key, and an AWS-shaped access key — committed it on a clean tree, then ran `npx getadvantage check` cold.

```
✗ Secret scan — 2 possible secrets in committed/staged files — remove + rotate before shipping.
.env:2 → Stripe live secret key: sk_live_…0000 (35 chars) · auth 7d3e4150553791318e1e3e917d32ad59a594f02bd50cb162fe0379133766baa4
.env:3 → AWS access key id: AKIAIO…0000 (20 chars) · auth 7b6e44c8d529c15d5de3f17dc7d7d6933d35fb2a1b68e91b1a25060f073ffd11
Smallest safe next edit — 2 blocking secret findings (same remedy for each; every finding named above with its own auth id):
remove the value from the tree, rotate the credential at the provider, commit the removal, then re-run check.

✗ Tracked .env file — 1 .env file tracked by git — a committed .env is a leak by itself, whatever it contains.
.env
Remove it from git (git rm --cached <file>), add it to .gitignore, and ROTATE every key that file ever held — git history keeps old values.

Verdict
✓ 4 ⚠ 1 ✗ 2 – 3 skipped
NO-GO — 2 blocking issues. Do not ship until these are clear.
```

Two independent checks catch this, which matters because they fail differently: the secret scanner looks for known credential shapes, the tracked-.env check does not care what is inside the file at all — a committed `.env` is a leak by itself, full stop. Wired into a commit hook or CI, this is a hard NO-GO before the file reaches a second clone.

## What it would NOT have caught

- **The pattern-matched secret scanner missed one of the three fixture values.** Of the Postgres URL, Stripe key, and AWS-shaped key committed in this reconstruction, the Stripe key and the AWS-shaped access key were both recognized and named by the secret scanner. The plain database URL — no password embedded in the connection string — was not flagged by that check on its own; the same URL **with** a password embedded in it is recognized as a separate finding ("Database URL with embedded password"). The tracked-.env-file check still caught the whole file regardless of what patterns it contains — that check is what actually saves you here for anything the pattern list doesn't recognize. A secret shape the scanner doesn't recognize, committed somewhere other than a file named `.env` (a config.json, a shell script, a docker-compose override), would not be caught by either check in this run.
- **A key rotated after commit but left in history.** The gate reads the current tracked state, not git history. Removing the file and rotating the credential is the fix; the gate cannot detect an old, already-rotated key sitting in a prior commit and will not warn that history still holds it — that part is on the operator to know and do.
- **Anything outside where it's wired in.** Same limit as every check in this series: this only stops a ship if something actually calls it on the commit or in CI. That is the exact gap [invisible mode](../launch/SOCIAL-MEDIA-PACK.md) closes — the check runs automatically on every agent session instead of depending on someone remembering to call it.

## Try it

`npx getadvantage check` on your own repo takes under a minute and reads nothing but your local tree — no network, no account. If it mis-fires on your project, open an issue; fixed within a day.
51 changes: 51 additions & 0 deletions docs/incidents/004-agent-self-report-scope-drift.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Incident 004 — an agent's own summary said "just the auth fix"; the diff said otherwise

**Source:** a public GitHub issue filed by a maintainer against their own production repo, describing why a release agent's self-reported change list could not be trusted on its own. Anonymized here at the maintainer's-consent bar (no consent on file yet, no live touch conversation open on this thread) — no repo, org, or handle named. Reconstructed independently as a generic scope-drift scenario and run through getAdvantage's live gate; not the maintainer's own words beyond the class of failure.

## What happened

A release agent was coordinating a scoped change and reporting back what it had touched. The maintainer's own account is blunt about the limit of that self-report: the agent "does not have access to that project and must not fabricate the list" of what it actually changed. The team could not tell, from the agent's own summary, whether the real diff matched the declared task — they had to go verify by hand.

This is the ordinary shape of a much broader problem: an agent says "I only touched the auth flow," and either it did, or it also picked up an unrelated file along the way — a leftover edit, a convenience refactor, a file it thought was related. Nothing about the agent's own narration tells you which one happened. The only ground truth is the diff.

## Reconstruction: what the gate does with this

Reproduced the shape of the failure locally with getAdvantage's Intent Contract: pinned a baseline commit, declared the real task in writing before any agent work started (`goal: "Add password reset flow"`, `allow: src/auth/**, tests/auth/**`), froze that contract as its own commit, then let the "agent" commit — genuinely scoped work in `src/auth/reset.js`, plus one extra line in an unrelated `src/billing/charge.js`, exactly the kind of drift a self-report narration would describe as "just the password reset." Ran `getadvantage intent check` cold against that history.

```
Intent Contract
✗ Intent Contract — NO-GO — 1 scope violation against the Intent Contract.
goal: Add password reset flow
contract: sha256:5cb2bd6081852bcecb488e291f93f925d3a2259770a0afd735683a33ed88526a
receipt: sha256:62573d0408d2393991edd4483bf718ffc481e61dfa9a109443d0f3daf9a1aa8f
baseline: 484af8991baa
freeze: 35cb8ea594e0
src/billing/charge.js — outside allowlist
Smallest safe next edit — path(s) outside Intent Contract allow list (src/billing/charge.js):
Preferred: unstage/remove the out-of-scope path(s) so the commit stays inside the frozen envelope at .getadvantage/intent.json.
Example: git restore --staged --worktree -- src/billing/charge.js
To authorize a wider envelope: start a branch from a trusted base with NO intent history, then:
getadvantage intent init --goal "…" --allow "relevant/**" --allow "…"
git add .getadvantage/intent.json && git commit -m "chore: intent contract"
Note: editing a frozen .getadvantage/intent.json cannot self-authorize (unsigned local mode: one freeze per clean lineage).
scope verified; semantic correctness not proven

Verdict
NO-GO — changes left the authorized scope (or trust failed).
scope verified; semantic correctness not proven
```

The check doesn't read the agent's summary at all. It diffs everything that actually happened after the declared baseline — committed, staged, unstaged, even untracked — against the envelope a human wrote down before the work started, and names the exact file that fell outside it. No narration to trust or distrust; just the diff against the contract.

## What it would NOT have caught

Being direct about the edges, because a gate that only advertises catches isn't trustworthy:

- **No contract, no verdict.** This only works if a human wrote the Intent Contract down before the agent started. A team that skips that step gets no scope check at all — `getadvantage check` never fakes a "verified" result for a project with no frozen contract.
- **A change that's technically in-scope but wrong.** The gate proves the diff stayed inside `src/auth/**`; it has no opinion on whether the password-reset logic itself is correct. Its own honesty line says it plainly: "scope verified; semantic correctness not proven."
- **A contract written too loosely.** If the human had allowed `src/**` instead of `src/auth/**`, the billing edit would pass cleanly — the gate enforces exactly the envelope it was given, not the envelope that should have been written.
- **Anything outside where it's wired in.** An ad hoc, uninstalled-in-CI run gives zero protection. This is the exact gap [invisible-mode / automatic hook installs](../launch/SOCIAL-MEDIA-PACK.md) is built to close: the check running automatically at commit time instead of depending on someone remembering to call it, or trusting the agent's own summary instead.

## Try it

`npx getadvantage check` on your own repo takes under a minute and reads nothing but your local tree — no network, no account. If it mis-fires on your project, open an issue; fixed within a day.
Loading