A passing test. A failing suite. An explanation you can replay.
Quick start · Demo evidence · How it works · Verification · Documentation · Türkçe
Test Autopsy investigates order-dependent Vitest failures and the Redis state left behind by earlier tests. It reduces a known failing file order, records state changes and tracked write call sites, then tests whether restoring suspect keys changes the victim's outcome.
v0.1.0 · Experimental developer preview. Built for one Vitest Node project, serial test files, ioredis, and a dedicated Redis namespace. The evidence below comes from automated tests and a planted fixture; independent real-project adoption and time savings have not been established.
A login test passes alone and fails after a user-blocking test. The assertion tells you what failed. It does not tell you which earlier file left the blocking state behind.
Test Autopsy connects a reduced, verified file order, the observed Redis changes, and a controlled restore experiment in one inspectable report. When verified, minimality means that removing any one retained predecessor makes the victim pass; it is not a global minimum.
|
Require the victim to pass alone and fail with the same fingerprint in the supplied order. Keep unresolved execution separate. Understand the evidence → |
Remove unnecessary predecessor files while preserving their order and the victim failure. Report the limits of minimality. Inspect the reduction method → |
|
Connect changed Redis keys with tracked write call sites. Restore candidate keys before the victim file and compare outcomes. Connect your Redis client → |
Keep readable reports and a replay manifest containing the reduced order, victim identity, Redis prefix, and test-file hashes. Explore commands and artifacts → |
Predecessor files retain their tests. In the victim file, only the selected test runs with its applicable setup and hooks; other test cases are skipped. The reducer operates on files.
The included fixture leaves autopsy:demo:blocked:user:42 behind. The recorded investigation retains the polluter and victim files; restoring that key makes both intervention trials pass. These are results from this planted fixture, not a real-world success-rate benchmark.
Read the captured evidence · Inspect the example tests · Understand the evaluation limits
Use Node.js 22.12+ and npm 11+; CI uses npm 11.17.0. If your Node installation bundles npm 10, follow the npm upgrade guidance. The documented distribution is this source repository.
git clone https://github.com/manyZXZ/test-autopsy.git
cd test-autopsy
npm ci
npm run build
docker compose up -dThe Compose service starts a disposable Redis on port 16379. Set the endpoint and run the planted example:
macOS / Linux
export AUTOPSY_REDIS_URL=redis://127.0.0.1:16379
npm run demoWindows PowerShell
$env:AUTOPSY_REDIS_URL = 'redis://127.0.0.1:16379'
npm run demoThe demo verifies the reduced sequence, restore intervention, and restored baseline. Open examples/redis-pollution/.test-autopsy/report.html to inspect the result. Run docker compose down when finished. If Docker is unavailable, use your own dedicated Redis and set its actual endpoint.
Integrate your own project — client setup, configuration, diagnosis, and replay
Point both your tests and Test Autopsy at the same dedicated test Redis. Give this investigation its own namespace, such as autopsy:myapp:. No other process should read or write that namespace while a diagnosis is running.
POSIX shell:
export AUTOPSY_REDIS_URL=redis://127.0.0.1:6379/15PowerShell:
$env:AUTOPSY_REDIS_URL = 'redis://127.0.0.1:6379/15'The database number is only an example. A dedicated instance is preferable to sharing a database with unrelated workloads. The prefix is the actual snapshot and restore boundary.
Initialize any expected baseline fixtures before starting Test Autopsy. The tool snapshots existing state; it does not know which values your application considers correct.
From the Test Autopsy repository:
npm packThen, from your application repository, install the generated archive using its actual path:
npm install --save-dev /absolute/path/to/manyzxz-test-autopsy-0.1.0.tgzUse a path appropriate to your operating system. Keep the resulting dependency change local until you decide to adopt the tool in that project.
Wrap the ioredis client used by your application tests:
import Redis from 'ioredis';
import { trackRedis } from '@manyzxz/test-autopsy/vitest';
export const redis = trackRedis(
new Redis(process.env.AUTOPSY_REDIS_URL!),
);Your application must use the same client instance. Wrapping an unused client cannot attribute writes made through a different connection. The instrumentation records commands, keys, and source stacks; it does not log command values. The CLI injects its Vitest runner while preserving your setup files; no additional setup hook is required.
State snapshots can still show a changed key when a write came from an untracked client, but that change has no reliable tracked write call site. See integration and attribution.
Create autopsy.config.json in your application repository:
{
"root": ".",
"vitestConfig": "vitest.config.ts",
"files": [
"test/harmless.test.ts",
"test/user-block.test.ts",
"test/login.test.ts"
],
"victim": {
"file": "test/login.test.ts",
"name": "authentication > permits an active user"
},
"redis": {
"urlEnv": "AUTOPSY_REDIS_URL",
"keyPrefix": "autopsy:myapp:",
"allowRemote": false,
"maxKeys": 1000,
"maxBytes": 10485760
},
"repetitions": 2,
"maxTrials": 60,
"timeoutMs": 30000,
"outputDir": ".test-autopsy"
}Replace file paths and the full victim name with values from your own suite. This is an explicit sequence, not a glob search. The victim file belongs at the end of the sequence. The numbers above are example settings, not performance guarantees.
From the application repository:
npx --no-install test-autopsy doctor --config autopsy.config.json
npx --no-install test-autopsy diagnose --config autopsy.config.jsondoctor validates the configured environment. diagnose executes trusted project code repeatedly and restores only the configured Redis scope between trials.
A confirmed diagnosis exits with code 1, so the shell can surface a reproduced problem. This is expected behavior; it is not itself a tool crash.
The output directory contains report.html, report.md, report.json, and replay.json. Open the HTML report in a browser or read the Markdown report directly. Before sharing artifacts, review key names, test names, source stacks, and error messages for sensitive information.
Use the replay file emitted by the diagnosis:
npx --no-install test-autopsy replay .test-autopsy/replay.json --config autopsy.config.jsonPrepare the same clean Redis baseline again before replay. Replay refuses changed test files, victim identity, or Redis prefix; it is not a full environment snapshot. An application-code fix can be checked against unchanged tests. If your fix changes a test file, run a fresh diagnosis to create new metadata. Verify the original order and ordinary suite as well.
A self-contained visual report with ordered files, Redis evidence, source call sites, and the trial matrix. Download the captured HTML example → |
A readable account of the investigation for local review and a carefully sanitized issue or discussion. Read the curated example → |
Structured trial outcomes, candidate keys, tracked mutations, limitations, and intervention results. Inspect the report types → |
The retained order, exact victim, prefix, test-file hashes, and failure fingerprint for a guarded rerun. Read the replay reference → |
Open the real HTML report preview
The screenshot shows the actual report renderer. Machine-specific paths were removed from the downloadable example.
Each trial starts from the captured Redis baseline. The restore boundary is the configured physical key prefix; the tool does not call FLUSHDB or FLUSHALL to reset a trial. A restoration intervention happens immediately before the victim file.
Read the architecture · Read the methodology
This is a fixed verification snapshot of commit cc1c8d5: 50 unit tests + 39 real Redis integration tests, with all six jobs successful in the recorded CI run. The badge at the top links to current CI status. These counts are not coverage percentages or a measure of real-world diagnosis accuracy.
The checks exercise reduction, namespace isolation, restoration, replay guards, output escaping, timeouts, cancellation, client tracking, environment pinning, and clean package installation. See validation for commands and evidence boundaries.
| Component | Supported starting point |
|---|---|
| Node.js | 22.12.0 or later |
| npm | 11 or later; CI and package installation use 11.17.0 |
| Module format | TypeScript/JavaScript; Test Autopsy itself is ESM |
| Vitest | 4.1.11; peer range ~4.1.11 |
| Redis client for attribution | ioredis 5; development dependency pinned to 5.11.1 |
| Redis environment | Standalone, dedicated test instance and exclusive namespace |
| Vitest configuration | One project, Node environment, no concurrent test cases |
Tracked clients must use awaited individual commands on a standalone TCP/TLS connection. Pipelines, transactions, auto-pipelining, Unix sockets, Sentinel, custom connectors, and Redis Cluster are outside this preview. See configuration and limitations.
| Result | Meaning | Next action |
|---|---|---|
diagnosed |
A repeatable order-dependent victim failure was retained through the investigation. | Inspect the retained sequence, suspects, and intervention evidence. |
not-reproduced |
Every confirmation trial of the configured order passed. | Verify the order, baseline, and failing test identity. |
fails-alone |
The victim failed without predecessor files. | Establish a clean passing baseline first. |
inconclusive |
Execution or consistency checks did not support a reliable conclusion. | Inspect trial details and resolve the missing condition. |
An intervention marked supported means the victim passed in repeated trials after the selected suspect keys were restored before its file. This supports a Redis-state explanation in the tested conditions. It does not prove that each restored key is causal, that a single test is solely responsible, or that every future execution will behave the same way.
Minimality, exit codes, and cancellation
When reduction.minimal is true, the result is 1-minimal at the file level under the trial oracle: every individual removal of a retained predecessor was verified to pass with the victim file retained. It is not a mathematical guarantee of the shortest sequence. If the budget runs out or unresolved trials prevent the guarantee, the report must not present a certified minimum.
| Code | Meaning |
|---|---|
0 |
Successful non-failing operation, no reproduced diagnosis, or replay passed. |
1 |
Confirmed diagnosis or replay reproduced the failure. |
2 |
Invalid configuration, failed prerequisites, victim fails alone, or inconclusive investigation. |
Cancellation exits with 130 after an attempt to restore the captured state.
Do not classify every nonzero exit as “pollution detected.” See CLI reference for command-specific behavior.
Configure exact file order, the victim, and a dedicated Redis namespace. Configuration · CLI reference |
Read the failure oracle, reduction guarantees, state model, and intervention semantics. Methodology · Architecture |
Diagnose missing victims, changing outcomes, timeouts, and retained private artifacts. Troubleshooting · Privacy and execution |
Reproduce the known fixture, inspect the limits, and help test real project failures. Validation · Roadmap · Contributing |
Türkçe başlangıç rehberi → · Captured example · Visual asset maintenance
Use Test Autopsy when you know a failing order and a victim that can pass alone, Redis is a plausible interaction source, and you can give the investigation exclusive access to a disposable namespace.
Supported boundaries and limits of a finding
- Redis is the reset scope. Files, SQL databases, external services, environment changes, and other side effects are not restored by the tool.
- The baseline is supplied by you. Pre-existing polluted data is faithfully captured as baseline data.
- A file is the unit of reduction. Tests inside a file are not rearranged or independently minimized.
- Intervention happens before the victim file. Earlier tests or setup inside that file can modify state afterward.
- Tracking is explicit. Unwrapped clients can contribute state differences without source attribution.
- Serial execution is required. Concurrent tests, background workers, and shared namespaces make attribution or reproduction ambiguous.
- Snapshotting changes timing. Time-sensitive failures and expiring keys require particular care.
- Repetition is evidence, not certainty. A small number of repeated trials cannot establish a statistical reliability bound.
- Replay is local metadata. It does not package your code, credentials, services, fixtures, or toolchain.
- An observed change is not a proven defect. Some state changes are intentional. Read the intervention and source context together.
These are product boundaries for the preview, not claims that wider flaky-test diagnosis is solved.
What is read, restored, and retained locally
Test Autopsy runs locally. It has no hosted dashboard, telemetry service, model API, or automatic artifact upload.
Redis payloads are kept in memory and a private temporary local snapshot to restore the baseline. The baseline file is deleted after successful restoration; worker logs and diagnostic work files remain locally for troubleshooting. A restore failure retains the snapshot for recovery, and forced termination can leave it behind. Public reports use state fingerprints instead of serialized values. Key names, source stacks, and private worker logs may contain sensitive information. Review artifacts before publishing.
The test runner executes your project, its configuration, and its dependencies. It is not a sandbox. Use trusted repositories and dedicated development services. More detail: privacy and security.
npm ci
npm run check
npm run package:smoke
npm run test:integrationpackage:smoke builds and packs the package, installs it into a temporary project, verifies the CLI and public exports, and checks that diagnostic work files are excluded from the archive. The CI workflow defines Node.js 22/24 jobs on Windows and Linux, plus Linux integration jobs using Redis 7.4.9; the badge links to their current status.
Integration tests require a reachable dedicated Redis instance at redis://127.0.0.1:16379, or the URL provided by AUTOPSY_INTEGRATION_REDIS_URL. The included planted example runs with npm run demo after setting AUTOPSY_REDIS_URL; see validation for prerequisites and interpretation.
For contribution expectations, reproducible bug reports, and review criteria, see CONTRIBUTING.md. Security-sensitive reports belong in the process described in SECURITY.md.
README graphics are editable repository assets. Run npm run readme:assets to regenerate the light/dark SVG set; see visual maintenance.
Test pollution and order-dependent testing are established topics. This project does not claim to invent them.
- asottile/detect-test-pollution isolates predecessor tests associated with a pytest failure and offers order fuzzing.
- JS-TOD: Detecting Order-Dependent Flaky Tests in Jest studies extracting, reordering, and rerunning Jest tests to reveal order dependency.
- vitest-leak-detector reports async resources left behind by Vitest tests.
Test Autopsy's particular focus is connecting a reduced Vitest file order with scoped Redis state evidence and a repeatable restore intervention. These references provide context; they are not an exhaustive market survey or an endorsement of this tool.
The most useful early contribution is a small real-world reproducer with a known failure, the manual investigation, and evidence of whether Test Autopsy reaches a useful finding. New stores and runners should follow demonstrated usefulness and clear reset semantics.
MIT © 2026 Ataberk Koç. See LICENSE.
