Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,8 @@ backupdrill drill # latest snapshot; add --snapshot <timestam
backupdrill drill --verify-all-files # also checksum every Storage file, not just a sample
```

After the restore, the drill counts the rows of every table in the sandbox and compares each count with the number of rows that table has in the backup file. One row more or less in any table fails the drill. The backup's count comes from reading the dump itself, not from your database, so this adds no queries against production; it costs the drill one extra read of the dump. Materialized views and partitioned parent tables have no rows of their own in the dump, and tables created by an extension hold rows the extension adds itself, so these are checked for being non-empty instead. `restore` runs the same comparison on the target project.

Backups record your installed Postgres extensions in the manifest, and the drill pre-installs them in the sandbox before restoring — including switching to a pgvector-enabled sandbox image when your database uses `vector` columns.

The sandbox restore runs up to 4 parallel `pg_restore` jobs, sized from the CPUs and memory Docker reports, so a 2 vCPU / 8 GB machine gets 2. Table data and index builds run in parallel. Foreign keys, triggers and policies run on a single connection to keep their error messages separate, including errors about Supabase-managed objects such as `auth.users`. `backupdrill restore` into a real Supabase project stays on a single connection.
Expand Down
45 changes: 26 additions & 19 deletions examples/reproducible-drill/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ check is what stops a restore from overwriting the project it backed up.

## What we measured

Run on 2026-11-06 with CLI 2.10.0 and the PostgreSQL 17 client tools
Run on 2026-11-07 with CLI 2.11.0 and the PostgreSQL 17 client tools
(`pg_dump`/`pg_restore`/`psql` 17.11), the same major version the hosted service
uses.

Expand All @@ -80,8 +80,8 @@ Backup:
```
→ Inspecting database…
✓ Postgres 17.6 — schema(s) public: 14 tables, ~1,496,499 rows
→ Dumping → s3://acme-backups/backupdrill/acme-prod/2026-11-06T03-58-17-486Z/dump.pgcustom
✓ Database dumped (21.6 MB, sha256 f160e1c9536d…)
→ Dumping → s3://acme-backups/backupdrill/acme-prod/2026-11-07T11-35-38-795Z/dump.pgcustom
✓ Database dumped (21.6 MB, sha256 c5617d2f2cdf…)
✓ Auth users backed up (1,000 users, 1,000 sign-in identities, 0.2 MB)
→ Syncing Storage files…
→ user-uploads: 250 files
Expand All @@ -97,18 +97,19 @@ Drill:
```
✓ Auth users: 1,000 users restored (1,000 identities); 1 foreign key(s) into the Auth tables created; 1 trigger(s) recreated; a test sign-up worked

✓ archive integrity — sha256 matches (f160e1c9536d…)
✓ archive integrity — sha256 matches (c5617d2f2cdf…)
✓ storage file integrity — all 250 files match their manifest checksums
✓ sandbox extensions — installed pg_trgm, pgcrypto
✓ pg_restore — completed in 1.9s (2 parallel jobs)
✓ pg_restore — completed in 1.8s (2 parallel jobs)
✓ post-data objects — user objects restored; 1 Supabase-managed object(s) skipped (they reference tables such as auth.users, which the drill sandbox doesn't have; restoring into a new project needs the rows they point at to be there first)
✓ table count — restored 14, manifest 14
✓ no missing tables — all manifest tables present
✓ populated tables came back — no populated table restored empty
✓ row counts match the backup — all 12 tables match the backup exactly (1,476,250 rows)
✓ API grants — access for anon, authenticated and service_role set as it was at backup time on 24 object(s)
✓ app checks — command exited 0 in 0.1s

✓ Drill PASSED — snapshot 2026-11-06T03-58-17-486Z, 14 tables / 1,496,250 rows restored in 1.9s
✓ Drill PASSED — snapshot 2026-11-07T11-35-38-795Z, 14 tables / 1,496,250 rows restored in 1.8s
```

The Auth line is printed first but runs last: once the checks below it have
Expand All @@ -121,9 +122,10 @@ reports on its own line and does not change the verdict.
inside the sandbox container — the drill hands it the sandbox's connection
string via `BACKUPDRILL_SANDBOX_URL`, so it needs whatever runtime your check
is written in, and it has your machine's privileges, not a container's. It
asserts
the exact row count of five tables and the exact revenue total, so losing part
of the data fails the drill. Its other two assertions — no orphaned foreign
asserts the exact row count of five tables and the exact revenue total. The row
counts repeat what the drill's own "row counts match the backup" check already
covers for every table; the revenue total is the kind of check only your code
can make. Its other two assertions — no orphaned foreign
keys, materialized view agreeing with its base table — are near-tautologies a
completed restore forces to hold; they are in there as a template for the
assertions only your own code can make.
Expand All @@ -144,13 +146,14 @@ as in a new project:
✓ table count: restored 14, manifest 14
✓ no missing tables: all manifest tables present
✓ populated tables came back: no populated table restored empty
✓ row counts match the backup: all 12 tables match the backup exactly (1,476,250 rows)
✓ API grants: access for anon, authenticated and service_role set as it was at backup time on 24 object(s)
✓ Database restored (14 tables, 1,496,250 rows)
✓ Auth users restored: 1,000 users (1,000 identities); trigger(s) recreated: on_auth_user_created
! Everyone signs in again: sessions aren't in the snapshot. Reconfigure Auth in the Supabase dashboard before switching your app over: sign-in providers (OAuth client IDs and secrets), SMTP, redirect URLs, the site URL and SSO.
→ Downloading 250 Storage files → …/recovered-files
✓ Storage files written to …/recovered-files
✓ Restore complete — snapshot 2026-11-06T03-58-17-486Z: database restored, 250 Storage files written to …/recovered-files
✓ Restore complete — snapshot 2026-11-07T11-35-38-795Z: database restored, 250 Storage files written to …/recovered-files
==> sign in on the restored project
✓ user1@example.com signed in with the password it had before the backup (HTTP 200, access token issued)
✓ the same account with a wrong password is refused (HTTP 400)
Expand All @@ -163,8 +166,8 @@ output; they are cut here for length.

## What is and isn't identical when you rerun this

We ran it three times in July 2026, again in October, and again in November
after adding the Auth users. Worth knowing before you compare:
We ran it three times in July 2026, again in October, and twice in November:
after adding the Auth users, and after adding the row count check. Worth knowing before you compare:

| Value | Stable across runs? | Why |
|---|---|---|
Expand All @@ -173,13 +176,17 @@ after adding the Auth users. Worth knowing before you compare:
| Dump and `auth.dump` sha256 | **No** | Two reasons: the random uuid keys, `now()` stamps and bcrypt salts above mean the bytes genuinely differ, and `pg_dump` writes a creation timestamp into the archive header, so even identical data would not hash the same |
| `pg_restore` seconds | **No** | Hardware |

The first two rows are the reason the drill measures instead of trusting the
manifest: on one of our runs the manifest claimed 250,000 more rows than the
database actually held, and the drill's count was right anyway. The manifest's
row counts are estimates, which is exactly why the check is "every table that
had rows came back non-empty" rather than "the row counts match".

**Restore time is hardware, not a product claim.** 1.9 s is a 21.6 MB dump
The first two rows are the reason the drill never compares against the
manifest's row counts: on one of our runs the manifest claimed 250,000 more rows
than the database actually held. "Row counts match the backup" counts the rows
of each table in the dump file itself and compares them with the restored
tables, one table at a time. 12 of the 14 tables are compared that way; the
partitioned parent `events` has no rows of its own in the dump, and the
materialized view `order_totals` is rebuilt by `REFRESH`, so those two are only
checked for being non-empty. That is why 1,476,250 rows are compared out of the
1,496,250 restored: the difference is the view's 20,000 rows.

**Restore time is hardware, not a product claim.** 1.8 s is a 21.6 MB dump
restored into a container on the same laptop. Your number depends on dump size,
disk, and network — which is the point of measuring it on a schedule instead of
guessing.
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "backupdrill",
"version": "2.10.1",
"version": "2.11.0",
"description": "Open-source CLI to back up a Supabase project — Postgres database and Storage files — to your own S3/R2/B2 bucket, with a checksummed manifest.",
"type": "module",
"bin": {
Expand Down
98 changes: 90 additions & 8 deletions src/drill.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,10 @@ import {
restoreDatabaseArtifact,
installSandboxShim,
tableDataInDump,
countRowsInDump,
} from "./restore-engine.js";
import { connectPg } from "./supabase-ca.js";
import { orphanedForeignKeysCheck, type OrphanedForeignKey } from "./orphaned-foreign-keys.js";
import { orphanedForeignKeysCheck, tableKey, type OrphanedForeignKey } from "./orphaned-foreign-keys.js";
import { grantsCheck, type GrantRestore, type GrantedObject } from "./grants.js";
import { localPgRestoreMajor, restoreToolBreaksOnTarget, sandboxMajorFor } from "./pgbin.js";
import { log } from "./log.js";
Expand Down Expand Up @@ -448,16 +449,80 @@ async function pgRestore(

// ── 校验 ────────────────────────────────────────────────────────────

/** 恢复出的一张表(verifyRestored 里查到的口径) */
interface RestoredTable {
schema: string;
name: string;
kind: string;
extension_member: boolean;
}

/**
* 第 4 项检查:恢复后每张表的行数和备份文件里的确切行数逐表相等。
* 只比普通表(含分区的叶子):matview 的数据是 REFRESH、分区父表不存行,dump 里都没有行可数;
* 扩展建的表 dump 里只有一部分行(见 verifyRestored 的 extension_member)。
* 备份文件数不出来、某张表在备份文件里找不到数据段,都判失败:比不了就是没验证过,不拿估算顶上。
*/
export function rowCountsCheck(
inBackup: Map<string, number> | null,
manifest: Manifest,
restoredTables: RestoredTable[],
/** 键 = tableKey */
restoredRows: Map<string, number>
): DrillCheck {
const name = "row counts match the backup";
if (!inBackup) {
return { name, pass: false, detail: "couldn't count the rows in the backup file, so row counts weren't compared" };
}
const inManifest = new Set(manifest.database.tables.map((t) => tableKey(t)));
const format = (n: number) => n.toLocaleString("en-US");
const differ: string[] = [];
const notInFile: string[] = [];
let compared = 0;
let rows = 0;
for (const t of restoredTables) {
if (t.kind !== "r" || t.extension_member || !inManifest.has(tableKey(t))) continue;
const label = `${t.schema}.${t.name}`;
const expected = inBackup.get(tableKey(t));
if (expected === undefined) {
notInFile.push(label);
continue;
}
compared++;
rows += expected;
const restored = restoredRows.get(tableKey(t)) ?? 0;
if (restored !== expected) differ.push(`${label} has ${format(restored)}, the backup has ${format(expected)}`);
}
const problems = [...differ, ...notInFile.map((label) => `${label} has no data section in the backup file`)];
if (problems.length > 0) {
const shown = problems.slice(0, 5).join("; ");
const more = problems.length > 5 ? `; and ${problems.length - 5} more` : "";
return { name, pass: false, detail: `${shown}${more}` };
}
return {
name,
pass: true,
detail:
compared === 0
? "no tables to compare"
: `all ${format(compared)} tables match the backup exactly (${format(rows)} rows)`,
};
}

/** 导出:真实恢复(restore.ts)复用同一套表级验证——两条路径对同一快照必须同一结论。 */
export async function verifyRestored(
connString: string,
manifest: Manifest,
/** 恢复用的那份 dump:恢复出空表时回查它,见第 3 项检查 */
/** 恢复用的那份 dump:逐表行数从它里面数(第 4 项检查),恢复出空表时也回查它(第 3 项) */
dumpPath?: string
): Promise<{ checks: DrillCheck[]; tableCount: number; rowTotal: number }> {
// pgConnectOptions:真实 Supabase 目标(restore 复用本函数)需要打包根 CA;
// 沙箱的 docker 连接串非 Supabase 主机,原样透传,行为不变
const client = await connectPg(connString);
// 备份文件里的确切行数只读本地文件、不连库,和下面逐表 count(*) 同时跑,省掉一段等待。
// 连上库之后才开始;这一步中途失败时由 finally 停掉它,不让它在演练收尾之后还在后台读整份 dump
const stopCounting = new AbortController();
const backupRowsP = dumpPath ? countRowsInDump(dumpPath, stopCounting.signal) : null;
try {
const schemas = manifest.database.schemas;
// 口径必须与 manifest 统计端(backup.ts inspectDatabase)完全一致:
Expand All @@ -469,9 +534,14 @@ export async function verifyRestored(
name: string;
kind: string;
populated: boolean;
extension_member: boolean;
}>(
// extension_member:扩展建的表(比如装在 public 里的 PostGIS spatial_ref_sys)。CREATE EXTENSION 会自带
// 初始行,dump 里只有过了扩展过滤条件的那部分,恢复后的行数天然比 dump 多,不做逐行比对
`select n.nspname as schema, c.relname as name,
c.relkind as kind, c.relispopulated as populated
c.relkind as kind, c.relispopulated as populated,
exists (select 1 from pg_depend d
where d.classid = 'pg_class'::regclass and d.objid = c.oid and d.deptype = 'e') as extension_member
from pg_class c
join pg_namespace n on n.oid = c.relnamespace
where c.relkind in ('r', 'p', 'm')
Expand All @@ -486,7 +556,8 @@ export async function verifyRestored(
const unpopulatedMatviews = new Set<string>();
let rowTotal = 0;
for (const t of restoredTables) {
const key = `${t.schema}.${t.name}`;
// tableKey(NUL 分隔),不用点号拼:public."a.b" 和 "public.a".b 拼出来都是 public.a.b,会串表
const key = tableKey(t);
// 未填充的 matview 无法 count(*)(Postgres 直接报错)。它要么备份时就没数据
// (dump 里本来就没有 REFRESH),要么 REFRESH 在 post-data 阶段失败——后者已由
// post-data 检查如实记录,这里按 0 行处理即可。
Expand All @@ -497,8 +568,11 @@ export async function verifyRestored(
}
// 名字来自恢复出的库(= 来自 dump = 桶里的数据):必须走标识符转义,
// 内嵌引号的合法名不炸,构造名不注入
// 普通表用 only:老式继承的父表 count(*) 会连子表的行一起算,dump 里父表只有它自己的行。
// 分区父表本身不存行,照旧数全部分区(只用于"有没有数据",不参与逐表比对)
const only = t.kind === "r" ? "only " : "";
const c = await client.query<{ n: string }>(
`select count(*)::bigint as n from ${quoteIdent(t.schema)}.${quoteIdent(t.name)}`
`select count(*)::bigint as n from ${only}${quoteIdent(t.schema)}.${quoteIdent(t.name)}`
);
const n = Number(c.rows[0].n);
counts.set(key, n);
Expand All @@ -519,7 +593,7 @@ export async function verifyRestored(
// 2. manifest 里的每张表都还原出来了
const restoredSet = new Set(counts.keys());
const missing = manifest.database.tables.filter(
(t) => !restoredSet.has(`${t.schema}.${t.name}`)
(t) => !restoredSet.has(tableKey(t))
);
checks.push({
name: "no missing tables",
Expand All @@ -537,8 +611,8 @@ export async function verifyRestored(
const restoredEmpty = manifest.database.tables.filter(
(t) =>
t.estimatedRows > 0 &&
!unpopulatedMatviews.has(`${t.schema}.${t.name}`) &&
(counts.get(`${t.schema}.${t.name}`) ?? 0) === 0
!unpopulatedMatviews.has(tableKey(t)) &&
(counts.get(tableKey(t)) ?? 0) === 0
);
// manifest 的行数是 pg_dump 之前另开连接取的 n_live_tup(统计估算),和 dump 不是同一个快照。
// 队列、会话、一次性密钥这类表,取统计时有行、dump 时已经删光,恢复出 0 行是对的
Expand All @@ -562,8 +636,16 @@ export async function verifyRestored(
: "no populated table restored empty",
});

// 4. 逐表行数和备份文件里一模一样。manifest 的行数是统计估算,只能管"别变空";确切的备份行数
// 从 dump 本身数(dump 是 pg_dump 在一个一致快照里导出的,它的行数就是这次备份下来的行数),
// 不去源库 count(*):那要在用户的库上全表扫一遍。数不出来就判失败,不退回估算
if (backupRowsP) checks.push(rowCountsCheck(await backupRowsP, manifest, restoredTables, counts));

return { checks, tableCount: restoredTables.length, rowTotal };
} finally {
// 正常走完时它早已结束,这两步什么都不做
stopCounting.abort();
await backupRowsP;
await client.end();
}
}
Expand Down
8 changes: 6 additions & 2 deletions src/dump-sql.ts
Original file line number Diff line number Diff line change
Expand Up @@ -157,8 +157,12 @@ export function parseGeneratedColumns(sql: string): Map<string, Set<string>> {
return tables;
}

/** 一个 COPY 头最长读这么多字符;超过就当作读不了,免得坏归档把内存吃满。 */
export const MAX_COPY_HEADER = 100_000;
/**
* 一个 COPY 头最长读这么多字符;超过就当作读不了,免得坏归档把内存吃满。
* 合法的头可以很长:1600 列(上限)、每个列名 63 字节且带引号要转义时,光列清单就有二十多万字符。
* 1 MB 留足余量,仍然挡得住坏归档。
*/
export const MAX_COPY_HEADER = 1_000_000;

/**
* 数据段的 COPY 头(dumpTableData_copy):"COPY <表> <列清单> FROM stdin;"。没有列的表列清单是空串,
Expand Down
3 changes: 2 additions & 1 deletion src/manifest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ export interface TableStat {
schema: string;
name: string;
// 来自 n_live_tup:是估算值(planner 统计),不是精确 count(*);分区父表恒为 0。
// 对"体积/行数骤降"这类异常检测足够;需要精确值时云版会在演练里重新计数。
// 只用于展示、异常检测和演练的"有行的表别变空"。演练的逐表确切比对不用它:
// 确切行数从备份文件本身数(restore-engine countRowsInDump),不在源库上 count(*)。
estimatedRows: number;
// ---- 以下为验证升级包预留字段(恢复闭环 PRD §5.1.2 / §0.1 D3):随 v2 一并定稿
// 以免二次升版,但本期不生成。只存聚合结果,绝不落抽样行原始值或 PII。
Expand Down
Loading
Loading