diff --git a/apps/cli/docs/go-cli-divergences.md b/apps/cli/docs/go-cli-divergences.md index 50613bb51a..d1180f107f 100644 --- a/apps/cli/docs/go-cli-divergences.md +++ b/apps/cli/docs/go-cli-divergences.md @@ -114,6 +114,13 @@ These commands exist in the TS CLI today but have no direct top-level equivalent `supabase start`, `db start`, `--from-backup`, and shadow containers (the last is why the shadow baseline cache's cold export can stop/start in ~1s). Timing is not part of the Go-parity surface (ADR 0016). +- Bundled pg-delta SQL for `db diff`/`db pull`/`db schema declarative generate`/`sync` + defaults to uppercase keywords, indent 2, width 180, trailing commas, and column/key + alignment when `[experimental.pgdelta] format_options` is unset. `format_options = "null"` + emits raw renderer output; partial JSON overlays those defaults. `schema pull` and + `schema generate` always use the same defaults (they do not read `format_options`). The + Go edge-runtime engine already defaulted to upper + width 180 (library fills the rest); + TS now makes indent/align explicit on the next-engine path. - `db schema declarative generate`/`sync` default declarative directory is `supabase/schemas`; the old Go CLI reference (pre-`7b469f5b3`) used `supabase/database`. The move aligns the default with the product-wide declarative-schemas convention. To keep the upgrade visible, diff --git a/apps/cli/docs/supabase/db/diff.md b/apps/cli/docs/supabase/db/diff.md index fb407e4349..6ad22f7e1a 100644 --- a/apps/cli/docs/supabase/db/diff.md +++ b/apps/cli/docs/supabase/db/diff.md @@ -12,7 +12,7 @@ By default, all schemas in the target database are diffed. Use the `--schema pub Projects created by a recent `supabase init` default to the pg-delta diff engine (`[experimental.pgdelta] enabled = true` in `config.toml`). Existing projects are unaffected and keep using migra unless they opt in. To fall back to the legacy migra engine, set `enabled = false` under `[experimental.pgdelta]`, or pass `--use-migra` for a single run. -With the bundled pg-delta engine, diff SQL defaults to lowercase keywords and a maximum width of 180, matching its declarative export. When `-f` writes migrations, execution-aware transaction semantics are preserved as ordered per-unit files; non-transactional units carry a directive that the CLI apply path honors. Flattened review output retains the rendered SQL and preambles, but not the unit boundaries supplied to a migration runner. Configure overrides with `[experimental.pgdelta] format_options`, or set `format_options = "null"` to emit raw, unformatted statements. +With the bundled pg-delta engine, diff SQL defaults to uppercase keywords, indent 2, a maximum width of 180, trailing commas, and column/key alignment, matching its declarative export. When `-f` writes migrations, execution-aware transaction semantics are preserved as ordered per-unit files; non-transactional units carry a directive that the CLI apply path honors. Flattened review output retains the rendered SQL and preambles, but not the unit boundaries supplied to a migration runner. Configure overrides with `[experimental.pgdelta] format_options`, or set `format_options = "null"` to emit raw, unformatted statements. While the diff command is able to capture most schema changes, there are cases where it is known to fail. Currently, this could happen if you schema contains: diff --git a/apps/cli/docs/supabase/db/schema-declarative-generate.md b/apps/cli/docs/supabase/db/schema-declarative-generate.md index 164176d6c0..1cd416e747 100644 --- a/apps/cli/docs/supabase/db/schema-declarative-generate.md +++ b/apps/cli/docs/supabase/db/schema-declarative-generate.md @@ -6,4 +6,6 @@ Exports the schema of a live database (local, linked, or custom URL) into SQL fi The bundled pg-delta engine writes one directory per schema at the root of that directory (`supabase/schemas/public/tables/users.sql`, `supabase/schemas/public/schema.sql`), with cluster-level objects that belong to no schema under a reserved `_cluster/` directory (`supabase/schemas/_cluster/roles.sql`). A schema literally named `_cluster` or `_custom`, in any casing, has its leading underscore percent-encoded (`%5Fcluster/`) so it can never claim a directory the export owns. Hand-authored SQL that pg-delta does not model belongs in `_custom/`, which the export never writes to and never prunes. +Emitted SQL uses the same default format as `db pull` (uppercase keywords, indent 2, width 180, column-aligned). Override with `[experimental.pgdelta] format_options`, or set `format_options = "null"` for raw statements. + Requires `--experimental` flag or `[experimental.pgdelta] enabled = true` in config. diff --git a/apps/cli/package.json b/apps/cli/package.json index d5cc742b7e..1f4f102bc9 100644 --- a/apps/cli/package.json +++ b/apps/cli/package.json @@ -55,7 +55,7 @@ "@parcel/watcher": "^2.6.0", "@supabase/api": "workspace:*", "@supabase/config": "workspace:*", - "@supabase/pg-delta": "1.0.0-alpha.42", + "@supabase/pg-delta": "1.0.0-alpha.46", "@supabase/pg-topo": "1.0.0-alpha.5", "@supabase/process-compose": "workspace:*", "@supabase/stack": "workspace:*", diff --git a/apps/cli/src/legacy/cli/legacy-complete.unit.test.ts b/apps/cli/src/legacy/cli/legacy-complete.unit.test.ts index 08cb85509a..cfdff142fa 100644 --- a/apps/cli/src/legacy/cli/legacy-complete.unit.test.ts +++ b/apps/cli/src/legacy/cli/legacy-complete.unit.test.ts @@ -100,7 +100,7 @@ describe("legacyRespondToComplete", () => { // `--debug ""` used to return zero candidates entirely: the leftover-args // computation counted `--debug` itself as "positional leftover," gating // out subcommand-name completion the way cobra never does for a - // persistent flag: `__complete --debug ''` lists all 36 root commands. + // persistent flag: `__complete --debug ''` lists all root commands. const result = legacyRespondToComplete(legacyRoot, ["__complete", "--debug", ""]); expect(result?.directive).toBe(LegacyCompletionDirective.NoFileComp); expect(result?.candidates.map((c) => c.name)).toContain("branches"); diff --git a/apps/cli/src/legacy/cli/root.ts b/apps/cli/src/legacy/cli/root.ts index db904dca84..784e9ca1a9 100644 --- a/apps/cli/src/legacy/cli/root.ts +++ b/apps/cli/src/legacy/cli/root.ts @@ -17,6 +17,8 @@ import { legacyLinkCommand } from "../commands/link/link.command.ts"; import { legacyLoginCommand } from "../commands/login/login.command.ts"; import { legacyLogoutCommand } from "../commands/logout/logout.command.ts"; import { legacyMigrationCommand } from "../commands/migration/migration.command.ts"; +import { legacyMigrationsCommand } from "../commands/migrations/migrations.command.ts"; +import { legacySchemaCommand } from "../commands/schema/schema.command.ts"; import { legacyNetworkBansCommand } from "../commands/network-bans/network-bans.command.ts"; import { legacyNetworkRestrictionsCommand } from "../commands/network-restrictions/network-restrictions.command.ts"; import { legacyOrgsCommand } from "../commands/orgs/orgs.command.ts"; @@ -78,11 +80,13 @@ export const legacyRoot = Command.make("supabase").pipe( legacyLoginCommand, legacyLogoutCommand, legacyMigrationCommand, + legacyMigrationsCommand, legacyNetworkBansCommand, legacyNetworkRestrictionsCommand, legacyOrgsCommand, legacyPostgresConfigCommand, legacyProjectsCommand, + legacySchemaCommand, legacySecretsCommand, legacySeedCommand, legacyServicesCommand, diff --git a/apps/cli/src/legacy/commands/db/schema/declarative/declarative.flow.ts b/apps/cli/src/legacy/commands/db/schema/declarative/declarative.flow.ts index 06669d82c3..4b1bff7103 100644 --- a/apps/cli/src/legacy/commands/db/schema/declarative/declarative.flow.ts +++ b/apps/cli/src/legacy/commands/db/schema/declarative/declarative.flow.ts @@ -1,3 +1,4 @@ +import { declaredSqlExtensions } from "../../../../../shared/schema/prepare-declarative-shadow.ts"; import type { LegacyPgDeltaImplementation } from "../../../../shared/legacy-pgdelta-next-flag.ts"; import { legacySchemaToCsvField } from "../../../../shared/legacy-schema-flags.ts"; import type { LegacyPgDeltaRemovalSummary } from "../../shared/legacy-pgdelta-engine.service.ts"; @@ -159,18 +160,6 @@ function matchImplicitExtension(message: string): LegacyImplicitExtensionMatch | }; } -/** - * Masks SQL comments and strings while preserving offsets. Extension declarations - * are DDL, so occurrences inside comments, quoted values, and dollar bodies must - * not suppress compatibility guidance. - */ -function maskSqlNonCode(sql: string): string { - return sql.replaceAll( - /--[^\r\n]*|\/\*[\s\S]*?\*\/|'(?:''|[^'])*'|\$(?:[a-zA-Z_][\w$]*)?\$[\s\S]*?\$(?:[a-zA-Z_][\w$]*)?\$/g, - (matched) => matched.replaceAll(/[^\r\n]/g, " "), - ); -} - function maskSqlComments(sql: string): string { return sql.replaceAll(/--[^\r\n]*|\/\*[\s\S]*?\*\//g, (matched) => matched.replaceAll(/[^\r\n]/g, " "), @@ -180,16 +169,7 @@ function maskSqlComments(sql: string): string { export function legacyDeclaredExtensions( files: readonly LegacyDeclarativeSqlFile[], ): ReadonlySet { - const declared = new Set(); - const pattern = - /\bCREATE\s+EXTENSION\s+(?:IF\s+NOT\s+EXISTS\s+)?(?:"([^"]+)"|([a-zA-Z_][\w$-]*))/gi; - for (const file of files) { - for (const match of maskSqlNonCode(file.sql).matchAll(pattern)) { - const extension = match[1] ?? match[2]; - if (extension !== undefined) declared.add(extension.toLowerCase()); - } - } - return declared; + return declaredSqlExtensions(files); } function declaredImplicitExtensions( diff --git a/apps/cli/src/legacy/commands/db/schema/declarative/declarative.orchestrate.integration.test.ts b/apps/cli/src/legacy/commands/db/schema/declarative/declarative.orchestrate.integration.test.ts index 1e0c6c51b2..18dc0bfa52 100644 --- a/apps/cli/src/legacy/commands/db/schema/declarative/declarative.orchestrate.integration.test.ts +++ b/apps/cli/src/legacy/commands/db/schema/declarative/declarative.orchestrate.integration.test.ts @@ -190,7 +190,12 @@ describe("legacyDiffDeclarativeToMigrations", () => { writeFileSync(join(declDir, "ignored.txt"), "ignored"); writeFileSync( join(declDir, ".pgdelta-export.json"), - JSON.stringify({ formatVersion: 1, redactSecrets: true, scope: "database" }), + JSON.stringify({ + formatVersion: 1, + redactSecrets: true, + scope: "database", + loadOrder: ["z.sql", "nested/a.sql"], + }), ); const calls: LegacyPgDeltaDeclarativePlanInput[] = []; const engine = Layer.succeed( @@ -240,7 +245,11 @@ describe("legacyDiffDeclarativeToMigrations", () => { { name: "nested/a.sql", sql: "select 'a';" }, { name: "z.sql", sql: "select 'z';" }, ]); - expect(calls[0]?.manifest).toEqual({ redactSecrets: true, scope: "database" }); + expect(calls[0]?.manifest).toEqual({ + redactSecrets: true, + scope: "database", + loadOrder: ["z.sql", "nested/a.sql"], + }); expect(calls[0]?.debug).toBe(true); expect(calls[0]?.noCache).toBe(true); expect(calls[0]?.strictCoverage).toBe(true); diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.next.layer.integration.test.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.next.layer.integration.test.ts index e1b0190fa8..77ab310bce 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.next.layer.integration.test.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.next.layer.integration.test.ts @@ -72,6 +72,10 @@ function setup() { Effect.fail(new LegacyDeclarativeShadowDbError({ message: "stop after routing" })), ), ), + provisionPlatform: () => + Effect.fail(new LegacyDeclarativeShadowDbError({ message: "stop after routing" })), + provisionDeclarative: () => + Effect.fail(new LegacyDeclarativeShadowDbError({ message: "stop after routing" })), provisionPlan: (opts) => Effect.sync(() => { state.plan += 1; diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.next.layer.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.next.layer.ts index aaeafcf1ec..409eb41dd7 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.next.layer.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.next.layer.ts @@ -1,6 +1,11 @@ import { Clock, Effect, FileSystem, Layer, Path } from "effect"; import type { Pool } from "pg"; +import { + filesForDeclarativeShadowLoad, + prepareDeclarativeShadow, +} from "../../../../shared/schema/prepare-declarative-shadow.ts"; +import { SchemaEngineError } from "../../../../shared/schema/schema-errors.ts"; import { Output } from "../../../../shared/output/output.service.ts"; import { legacyLayeredParseEnv, @@ -42,7 +47,9 @@ function legacyPgDeltaNextConnectSuggestion(cause: unknown): string | undefined export const legacyPgDeltaNextEngineError = (cause: unknown) => { if (cause instanceof LegacyPgDeltaEngineError) return cause; - const suggestion = legacyPgDeltaNextConnectSuggestion(cause); + const suggestion = + legacyPgDeltaNextConnectSuggestion(cause) ?? + (cause instanceof SchemaEngineError ? cause.suggestion : undefined); const diagnostics = cause instanceof LegacyPgDeltaNextError ? cause.diagnostics : undefined; return new LegacyPgDeltaEngineError({ message: @@ -350,10 +357,11 @@ export const legacyPgDeltaNextEngineLayer = Layer.effect( ], { concurrency: 2 }, ); + yield* prepareDeclarativeShadow(declarativePool, input.files); const result = yield* adapter.planDeclarativeSchema({ targetPool: migrationsPool, shadowPool: declarativePool, - files: input.files, + files: filesForDeclarativeShadowLoad(input.files), allowDrops: true, ...(shadow.allowSameDatabaseIdentity ? { allowSameDatabaseIdentity: true } : {}), debug: input.debug, diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.service.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.service.ts index 7f71f6814a..837a057614 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.service.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-engine.service.ts @@ -43,6 +43,7 @@ export interface LegacyPgDeltaExportManifest { readonly baselineDigest?: string; readonly defaultOwner?: string | null; readonly files?: ReadonlyArray; + readonly loadOrder?: ReadonlyArray; } export interface LegacyPgDeltaRenderedFile { @@ -75,7 +76,8 @@ export type LegacyPgDeltaHazardKind = | "access_exclusive_lock" | "unmodeled_kind" | "unmodeled_drift" - | "unresolved_security_label"; + | "unresolved_security_label" + | "vault_presence"; interface LegacyPgDeltaActionHazard { readonly actionIndex: number; diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-files.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-files.ts index e73738e446..91c98882d0 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-files.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-files.ts @@ -76,6 +76,7 @@ export const LegacyReadPgDeltaExportManifest = Effect.fnUntraced(function* ( const baselineDigest = readManifestValue(decoded, "baselineDigest"); const defaultOwner = readManifestValue(decoded, "defaultOwner"); const files = readManifestValue(decoded, "files"); + const loadOrder = readManifestValue(decoded, "loadOrder"); return { redactSecrets, scope, @@ -83,6 +84,9 @@ export const LegacyReadPgDeltaExportManifest = Effect.fnUntraced(function* ( ...(typeof baselineDigest === "string" ? { baselineDigest } : {}), ...(typeof defaultOwner === "string" || defaultOwner === null ? { defaultOwner } : {}), ...(Array.isArray(files) && files.every((file) => typeof file === "string") ? { files } : {}), + ...(Array.isArray(loadOrder) && loadOrder.every((file) => typeof file === "string") + ? { loadOrder } + : {}), } satisfies LegacyPgDeltaExportManifest; }); diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-adapter.layer.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-adapter.layer.ts index dec237f75c..e417cf3284 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-adapter.layer.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-adapter.layer.ts @@ -20,7 +20,12 @@ import { import { classifyPlanHazards, plan, serializePlan } from "@supabase/pg-delta/plan"; import type { Plan as PgDeltaPlan } from "@supabase/pg-delta/plan"; import type { Policy } from "@supabase/pg-delta/policy"; -import { formatSqlStatements, type SqlFormatOptions } from "@supabase/pg-delta/sql-format"; +import type { SqlFormatOptions } from "@supabase/pg-delta/sql-format"; +import { schemaIsolatedPlanOptions } from "../../../../shared/schema/schema-plan-options.ts"; +import { + formatSchemaSql, + SCHEMA_SQL_FORMAT_DEFAULTS, +} from "../../../../shared/schema/sql-format-defaults.ts"; import { LegacyPgDeltaNextAdapter, @@ -273,99 +278,6 @@ function legacySkippedStatementDiagnostics( })); } -function legacyIsPgDeltaNextParameterAclDiagnostic( - diagnostic: LegacyPgDeltaNextLibraryDiagnostic, -): boolean { - return diagnostic.code === "unmodeled_kind" && diagnostic.context?.["kind"] === "parameter ACL"; -} - -/** - * The parameter-ACL catalog is cluster-wide, so a co-located declarative shadow - * observes Supabase platform grants too. Keep strict coverage for every ACL - * other than the exact platform bootstrap grant while removing the aggregate - * diagnostic when that bootstrap grant is the only observed parameter ACL. - */ -export function legacyFilterPgDeltaNextPlatformParameterAclDiagnostics( - diagnostics: readonly LegacyPgDeltaNextLibraryDiagnostic[], - userOwnedParameterAcls: readonly string[], -): LegacyPgDeltaNextLibraryDiagnostic[] { - const names = [...new Set(userOwnedParameterAcls)].sort(); - const filtered: LegacyPgDeltaNextLibraryDiagnostic[] = []; - for (const diagnostic of diagnostics) { - if (!legacyIsPgDeltaNextParameterAclDiagnostic(diagnostic)) { - filtered.push(diagnostic); - continue; - } - if (names.length === 0) continue; - const samples = names.slice(0, 5); - const more = names.length > samples.length ? ", …" : ""; - filtered.push({ - ...diagnostic, - message: - `${names.length} unmodeled "parameter ACL" object${names.length === 1 ? "" : "s"} ` + - `not managed by this engine (e.g. ${samples.join(", ")}${more}) — ` + - "v1 detects but does not model this kind", - context: { kind: "parameter ACL", count: names.length, samples }, - }); - } - return filtered; -} - -interface LegacyPgDeltaNextParameterAclGrant { - readonly name: string; - readonly grantee: string; - readonly privilege: string; -} - -// Supabase's platform bootstrap grants these so privileged platform roles can -// manage the setting and the Realtime owner can replay routines whose proconfig -// contains `SET log_min_messages ...`. Parameter ACLs have cluster scope, so -// the grants are also visible from sibling shadow DBs. -const legacyPgDeltaNextPlatformParameterAcls = new Set([ - "log_min_messages\u0000supabase_admin\u0000ALTER SYSTEM", - "log_min_messages\u0000supabase_admin\u0000SET", - "log_min_messages\u0000supabase_realtime_admin\u0000SET", -]); - -function legacyPgDeltaNextParameterAclKey(grant: LegacyPgDeltaNextParameterAclGrant): string { - return `${grant.name}\u0000${grant.grantee}\u0000${grant.privilege}`; -} - -export function legacyPgDeltaNextUserOwnedParameterAcls( - grants: readonly LegacyPgDeltaNextParameterAclGrant[], -): string[] { - return [ - ...new Set( - grants - .filter( - (grant) => - !legacyPgDeltaNextPlatformParameterAcls.has(legacyPgDeltaNextParameterAclKey(grant)), - ) - .map((grant) => grant.name), - ), - ].sort(); -} - -async function legacyFilterPgDeltaNextPlatformDiagnostics( - pool: Pool, - diagnostics: readonly LegacyPgDeltaNextLibraryDiagnostic[], -): Promise[]> { - if (!diagnostics.some(legacyIsPgDeltaNextParameterAclDiagnostic)) return [...diagnostics]; - const result = await pool.query( - `SELECT DISTINCT pa.parname AS name, - COALESCE(grantee.rolname, 'PUBLIC') AS grantee, - acl.privilege_type AS privilege - FROM pg_parameter_acl pa - CROSS JOIN LATERAL aclexplode(pa.paracl) acl - LEFT JOIN pg_roles grantee ON grantee.oid = acl.grantee - ORDER BY pa.parname, grantee, privilege`, - ); - return legacyFilterPgDeltaNextPlatformParameterAclDiagnostics( - diagnostics, - legacyPgDeltaNextUserOwnedParameterAcls(result.rows), - ); -} - function legacyNormalizePgDeltaNextRenderedFiles( files: readonly LegacyPgDeltaNextLibraryRenderedFile[], ): LegacyPgDeltaNextRenderedFile[] { @@ -412,17 +324,12 @@ export function legacyPgDeltaNextProfile( return { ...supabaseProfile, policy }; } -const legacyPgDeltaNextHumanFormatOptions: SqlFormatOptions = { - keywordCase: "lower", - maxWidth: 180, -}; - function legacyPgDeltaNextFormatOptions(raw: string | undefined): SqlFormatOptions | undefined { - if (raw === undefined || raw.trim().length === 0) return legacyPgDeltaNextHumanFormatOptions; + if (raw === undefined || raw.trim().length === 0) return SCHEMA_SQL_FORMAT_DEFAULTS; const parsed: unknown = JSON.parse(raw); if (parsed === null) return undefined; if (typeof parsed !== "object" || Array.isArray(parsed)) { - return legacyPgDeltaNextHumanFormatOptions; + return SCHEMA_SQL_FORMAT_DEFAULTS; } const value = (key: string): unknown => Reflect.get(parsed, key); const keywordCase = value("keywordCase"); @@ -435,7 +342,7 @@ function legacyPgDeltaNextFormatOptions(raw: string | undefined): SqlFormatOptio const preserveViewBodies = value("preserveViewBodies"); const preserveRuleBodies = value("preserveRuleBodies"); return { - ...legacyPgDeltaNextHumanFormatOptions, + ...SCHEMA_SQL_FORMAT_DEFAULTS, ...(keywordCase === "upper" || keywordCase === "lower" || keywordCase === "preserve" ? { keywordCase } : {}), @@ -450,11 +357,6 @@ function legacyPgDeltaNextFormatOptions(raw: string | undefined): SqlFormatOptio }; } -function legacyTerminatePgDeltaNextStatement(sql: string): string { - const trimmed = sql.trimEnd(); - return trimmed.endsWith(";") ? trimmed : `${trimmed};`; -} - function legacyFormatPgDeltaNextRenderedFiles( files: readonly LegacyPgDeltaNextLibraryRenderedFile[], format: SqlFormatOptions | undefined, @@ -462,9 +364,7 @@ function legacyFormatPgDeltaNextRenderedFiles( if (format === undefined) return files; return files.map((file) => ({ ...file, - contents: `${formatSqlStatements([file.contents], format) - .map(legacyTerminatePgDeltaNextStatement) - .join("\n\n")}\n`, + contents: formatSchemaSql(file.contents, format), })); } @@ -480,17 +380,25 @@ function legacyPgDeltaNextExportOptions(input: LegacyPgDeltaNextDeclarativeExpor function legacyPgDeltaNextPlanOptions(input: LegacyPgDeltaNextDeclarativePlanInput) { let manifest; if (input.manifest !== undefined) { - const { files, ...metadata } = input.manifest; - manifest = { ...metadata, ...(files !== undefined ? { files: [...files] } : {}) }; + const { files, loadOrder, ...metadata } = input.manifest; + manifest = { + ...metadata, + ...(files !== undefined ? { files: [...files] } : {}), + ...(loadOrder !== undefined ? { loadOrder: [...loadOrder] } : {}), + }; } + // Isolated load only — do not pin scope/redactSecrets; the sidecar owns those. return { + isolatedShadow: schemaIsolatedPlanOptions.isolatedShadow, + seedAssumedSchemas: schemaIsolatedPlanOptions.seedAssumedSchemas, + strictDataStatements: schemaIsolatedPlanOptions.strictDataStatements, + reorder: schemaIsolatedPlanOptions.reorder, + connectionReuse: schemaIsolatedPlanOptions.connectionReuse, profile: legacyPgDeltaNextProfile(input.schema), ...(manifest !== undefined ? { manifest } : {}), - isolatedShadow: true, - ...(input.allowSameDatabaseIdentity === true ? { allowSameDatabaseIdentity: true } : {}), - seedAssumedSchemas: false, - strictDataStatements: true, - reorder: true, + ...(input.allowSameDatabaseIdentity === true + ? { allowSameDatabaseIdentity: true } + : { allowSameDatabaseIdentity: false }), }; } @@ -652,56 +560,26 @@ function legacyMakePgDeltaNextAdapter[2], schema?: readonly string[], - ) => { - const resolved = await resolveProfile(pool, legacyPgDeltaNextProfile(schema), options); - return { - ...resolved, - extract: async ( - extractPool: Pool, - extractOptions?: Parameters[1], - ) => { - const result = await resolved.extract(extractPool, extractOptions); - return { - ...result, - diagnostics: await legacyFilterPgDeltaNextPlatformDiagnostics( - extractPool, - result.diagnostics, - ), - }; - }, - }; - }, + ) => resolveProfile(pool, legacyPgDeltaNextProfile(schema), options), plan, renderPlanFiles, - buildSchemaExport: async (pool: Pool, input: LegacyPgDeltaNextLibraryExportOptions) => { - const result = await buildSchemaExport(pool, input); - return { - ...result, - diagnostics: await legacyFilterPgDeltaNextPlatformDiagnostics(pool, result.diagnostics), - }; - }, - planSchemaFiles: async ( + buildSchemaExport, + planSchemaFiles: ( targetPool: Pool, shadowPool: Pool, files: readonly LegacyPgDeltaNextSqlFile[], input: LegacyPgDeltaNextLibraryPlanOptions, - ) => { - const result = await planSchemaFiles( + ) => + planSchemaFiles( targetPool, shadowPool, files.map((file) => ({ name: file.name, sql: file.sql })), input, - ); - const [loadDiagnostics, targetDiagnostics] = await Promise.all([ - legacyFilterPgDeltaNextPlatformDiagnostics(shadowPool, result.loadDiagnostics), - legacyFilterPgDeltaNextPlatformDiagnostics(targetPool, result.targetDiagnostics), - ]); - return { ...result, loadDiagnostics, targetDiagnostics }; - }, + ), serializeSnapshot, serializePlan, summarizeRemovals: legacySummarizePgDeltaNextRemovals, diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-adapter.unit.test.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-adapter.unit.test.ts index c4c47df6fe..fa276b675c 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-adapter.unit.test.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-adapter.unit.test.ts @@ -8,9 +8,7 @@ import { describe, expect } from "vitest"; import { legacyPgDeltaNextAdapterLayerFromLibraries, - legacyFilterPgDeltaNextPlatformParameterAclDiagnostics, legacyPgDeltaNextProfile, - legacyPgDeltaNextUserOwnedParameterAcls, legacySummarizePgDeltaNextHazards, legacySummarizePgDeltaNextRemovals, type LegacyPgDeltaNextLibraries, @@ -300,66 +298,6 @@ describe("LegacyPgDeltaNextAdapter", () => { }); }); - it("filters platform parameter ACL coverage without hiding user-owned ACLs", () => { - const diagnostics = [ - { - origin: "declarativeLoad" as const, - code: "unmodeled_kind", - severity: "warning" as const, - message: "2 unmodeled parameter ACLs", - context: { - kind: "parameter ACL", - count: 2, - samples: ["log_min_messages", "work_mem"], - }, - }, - { - origin: "declarativeLoad" as const, - code: "unsupported_extension", - severity: "warning" as const, - message: "extension is externally managed", - }, - ]; - - expect(legacyFilterPgDeltaNextPlatformParameterAclDiagnostics(diagnostics, [])).toEqual([ - diagnostics[1], - ]); - expect( - legacyFilterPgDeltaNextPlatformParameterAclDiagnostics(diagnostics, ["work_mem"]), - ).toEqual([ - { - ...diagnostics[0], - message: - '1 unmodeled "parameter ACL" object not managed by this engine (e.g. work_mem) — v1 detects but does not model this kind', - context: { kind: "parameter ACL", count: 1, samples: ["work_mem"] }, - }, - diagnostics[1], - ]); - }); - - it("recognizes only the exact Supabase platform parameter grant tuples", () => { - expect( - legacyPgDeltaNextUserOwnedParameterAcls([ - { name: "log_min_messages", grantee: "supabase_admin", privilege: "SET" }, - { name: "log_min_messages", grantee: "app_user", privilege: "SET" }, - { name: "work_mem", grantee: "supabase_realtime_admin", privilege: "SET" }, - { name: "work_mem", grantee: "app_user", privilege: "SET" }, - ]), - ).toEqual(["log_min_messages", "work_mem"]); - expect( - legacyPgDeltaNextUserOwnedParameterAcls([ - { name: "log_min_messages", grantee: "supabase_admin", privilege: "ALTER SYSTEM" }, - { name: "log_min_messages", grantee: "supabase_admin", privilege: "SET" }, - { name: "log_min_messages", grantee: "supabase_realtime_admin", privilege: "SET" }, - ]), - ).toEqual([]); - expect( - legacyPgDeltaNextUserOwnedParameterAcls([ - { name: "log_min_messages", grantee: "supabase_realtime_admin", privilege: "ALTER SYSTEM" }, - ]), - ).toEqual(["log_min_messages"]); - }); - it("renders selected-schema state without leaking other user or platform objects", () => { const schemaPublic = { kind: "schema", name: "public" } satisfies StableId; const schemaAuth = { kind: "schema", name: "auth" } satisfies StableId; @@ -543,7 +481,14 @@ describe("LegacyPgDeltaNextAdapter", () => { pool: targetPool, }); expect(state.exportInputs[1]).toMatchObject({ - format: { keywordCase: "lower", maxWidth: 180 }, + format: { + keywordCase: "upper", + indent: 2, + maxWidth: 180, + commaStyle: "trailing", + alignColumns: true, + alignKeyValues: true, + }, }); const planned = yield* adapter.planDeclarativeSchema({ @@ -558,11 +503,14 @@ describe("LegacyPgDeltaNextAdapter", () => { expect(state.declarativeInputs).toHaveLength(1); expect(state.declarativeInputs[0]).toMatchObject({ reorder: true, + connectionReuse: "reconnect-on-stuck", isolatedShadow: true, allowSameDatabaseIdentity: true, seedAssumedSchemas: false, strictDataStatements: true, }); + expect(state.declarativeInputs[0]).not.toHaveProperty("scope"); + expect(state.declarativeInputs[0]).not.toHaveProperty("redactSecrets"); expect(planned.diagnostics.map((diagnostic) => diagnostic.origin)).toEqual([ "declarativeLoad", "declarativeTarget", diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.ts index 0cb80a0369..2a573dac7f 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.ts @@ -9,10 +9,7 @@ import { } from "../../../../shared/legacy/global-flags.ts"; import { Output } from "../../../../shared/output/output.service.ts"; import { RuntimeInfo } from "../../../../shared/runtime/runtime-info.service.ts"; -import { - LegacyDbConnection, - type LegacyDbSession, -} from "../../../shared/legacy-db-connection.service.ts"; +import { LegacyDbConnection } from "../../../shared/legacy-db-connection.service.ts"; import { LegacyDockerRun } from "../../../shared/legacy-docker-run.service.ts"; import { legacyToPostgresURL } from "../../../shared/legacy-postgres-url.ts"; import { @@ -33,7 +30,6 @@ import { legacyRunPlanShadowProvisions, } from "./legacy-pgdelta-next-shadow.plan.ts"; import { - legacyConnectShadowDatabase, legacyMigrateNextShadowDatabase, legacyRemoveShadowDatabase, legacyShadowRunInputFromLocalContainerInputs, @@ -46,6 +42,7 @@ import * as HttpClient from "effect/unstable/http/HttpClient"; import { LegacyPgDeltaNextShadow, type LegacyPgDeltaNextMigrationsShadow, + type LegacyPgDeltaNextPlatformShadow, type LegacyPgDeltaNextPlanShadows, type LegacyPgDeltaNextShadowInput, } from "./legacy-pgdelta-next-shadow.service.ts"; @@ -115,25 +112,6 @@ export function legacyAllowSameDatabaseIdentityForPlanShadows(opts: { return opts.declarativeRestoredFromPgDataSnapshot && opts.sameSnapshotKey; } -/** - * Removes extensions that the legacy PG14 platform baseline installs implicitly - * so the declarative shadow reflects only extension declarations in schema files. - * `pgjwt` has a hard extension dependency on `pgcrypto`, and `storage.objects.id` - * depends on `uuid-ossp`, so both dependencies must be detached before the - * user-manageable extensions can be dropped with the default RESTRICT behavior. - */ -export const legacyPreparePgDeltaNextDeclarativeBaseline = Effect.fnUntraced(function* ( - session: Pick, - majorVersion: number, -) { - if (majorVersion === 14) { - yield* session.exec("ALTER TABLE storage.objects ALTER COLUMN id DROP DEFAULT"); - yield* session.exec("DROP EXTENSION IF EXISTS pgjwt"); - } - yield* session.exec("DROP EXTENSION IF EXISTS pgcrypto"); - yield* session.exec('DROP EXTENSION IF EXISTS "uuid-ossp"'); -}); - const setupRunInput = (input: NativeShadowInput, handle: LegacyShadowAcquiredHandle) => ({ fs: input.base.fs, path: input.base.path, @@ -273,6 +251,7 @@ export const legacyPgDeltaNextShadowLayer = Layer.effect( input: NativeShadowInput, opts: LegacyShadowCacheOpts, onBaselineSeam: Effect.Effect = Effect.void, + outputService: typeof Output.Service = output, ) => Effect.gen(function* () { const handle = yield* acquireShadow(input, opts); @@ -295,6 +274,17 @@ export const legacyPgDeltaNextShadowLayer = Layer.effect( return { migrationsUrl: legacyToPostgresURL(setup.connConfig), } satisfies LegacyPgDeltaNextMigrationsShadow; + }).pipe(Effect.provide(runtimeWith(outputService)), Effect.mapError(nextShadowError)); + + const provisionPlatform = (input: NativeShadowInput, opts: LegacyShadowCacheOpts) => + Effect.gen(function* () { + const handle = yield* acquireShadow(input, opts); + yield* awaitShadowReady(input, handle); + const setup = setupRunInput(input, handle); + yield* legacySetupShadowDatabase(input.spawner, setup, {}, handle); + return { + platformUrl: legacyToPostgresURL(setup.connConfig), + } satisfies LegacyPgDeltaNextPlatformShadow; }).pipe(Effect.provide(runtime), Effect.mapError(nextShadowError)); const provisionDeclarative = ( @@ -307,15 +297,6 @@ export const legacyPgDeltaNextShadowLayer = Layer.effect( yield* awaitShadowReady(input, handle); const setup = setupRunInput(input, handle); yield* legacySetupShadowDatabase(input.spawner, setup, { webhooks: "disabled" }, handle); - yield* Effect.scoped( - Effect.gen(function* () { - const session = yield* legacyConnectShadowDatabase(setup.connConfig); - yield* legacyPreparePgDeltaNextDeclarativeBaseline( - session, - input.base.setup.majorVersion, - ); - }), - ); return { declarativeUrl: legacyToPostgresURL(setup.connConfig), restoredFromPgDataSnapshot: handle._tag === "warm", @@ -331,12 +312,31 @@ export const legacyPgDeltaNextShadowLayer = Layer.effect( }); return LegacyPgDeltaNextShadow.of({ - provisionMigrations: (opts) => + provisionMigrations: (opts, outputService) => + Effect.gen(function* () { + const port = yield* nextPort(); + const built = yield* buildNativeBase(opts); + const input = buildNativeInput(opts, built, port); + return yield* provisionMigrations( + input, + cacheOpts(opts, "config"), + Effect.void, + outputService ?? output, + ); + }).pipe(Effect.mapError(nextShadowError)), + provisionPlatform: (opts) => + Effect.gen(function* () { + const port = yield* nextPort(); + const built = yield* buildNativeBase(opts); + const input = buildNativeInput(opts, built, port); + return yield* provisionPlatform(input, cacheOpts(opts, "config")); + }).pipe(Effect.mapError(nextShadowError)), + provisionDeclarative: (opts) => Effect.gen(function* () { const port = yield* nextPort(); const built = yield* buildNativeBase(opts); const input = buildNativeInput(opts, built, port); - return yield* provisionMigrations(input, cacheOpts(opts, "config")); + return yield* provisionDeclarative(input, cacheOpts(opts, "disabled")); }).pipe(Effect.mapError(nextShadowError)), provisionPlan: (opts) => Effect.gen(function* () { diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.unit.test.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.unit.test.ts index 9ce2a54731..d40c19c4f0 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.unit.test.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.layer.unit.test.ts @@ -1,50 +1,6 @@ -import { it } from "@effect/vitest"; -import { Effect } from "effect"; import { describe, expect, it as vitestIt } from "vitest"; -import { - legacyAllowSameDatabaseIdentityForPlanShadows, - legacyPreparePgDeltaNextDeclarativeBaseline, -} from "./legacy-pgdelta-next-shadow.layer.ts"; - -function recordingSession() { - const statements: string[] = []; - return { - statements, - session: { - exec: (sql: string) => - Effect.sync(() => { - statements.push(sql); - }), - }, - }; -} - -describe("legacyPreparePgDeltaNextDeclarativeBaseline", () => { - it.effect("detaches the PG14 platform dependencies before dropping extensions", () => { - const { session, statements } = recordingSession(); - return Effect.gen(function* () { - yield* legacyPreparePgDeltaNextDeclarativeBaseline(session, 14); - expect(statements).toEqual([ - "ALTER TABLE storage.objects ALTER COLUMN id DROP DEFAULT", - "DROP EXTENSION IF EXISTS pgjwt", - "DROP EXTENSION IF EXISTS pgcrypto", - 'DROP EXTENSION IF EXISTS "uuid-ossp"', - ]); - }); - }); - - it.effect("does not modify PG15+ platform objects before dropping extensions", () => { - const { session, statements } = recordingSession(); - return Effect.gen(function* () { - yield* legacyPreparePgDeltaNextDeclarativeBaseline(session, 17); - expect(statements).toEqual([ - "DROP EXTENSION IF EXISTS pgcrypto", - 'DROP EXTENSION IF EXISTS "uuid-ossp"', - ]); - }); - }); -}); +import { legacyAllowSameDatabaseIdentityForPlanShadows } from "./legacy-pgdelta-next-shadow.layer.ts"; describe("legacyAllowSameDatabaseIdentityForPlanShadows", () => { vitestIt.each([ diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.service.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.service.ts index ef9dcfa729..db01144d82 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.service.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta-next-shadow.service.ts @@ -1,5 +1,6 @@ import { Context, type Effect, type Scope } from "effect"; +import { Output } from "../../../../shared/output/output.service.ts"; import type { LegacyDeclarativeShadowDbError } from "./legacy-pgdelta.errors.ts"; import type { LegacyDbTomlValues } from "../../../shared/legacy-db-config.toml-read.ts"; import type { LegacyPgDeltaContext } from "../../../shared/legacy-pgdelta.ts"; @@ -10,6 +11,11 @@ export interface LegacyPgDeltaNextMigrationsShadow { readonly migrationsUrl: string; } +/** Platform baseline with no project migrations and no declaration-prep drops. */ +export interface LegacyPgDeltaNextPlatformShadow { + readonly platformUrl: string; +} + /** The two live databases needed to plan declarative SQL with pg-delta next. */ export interface LegacyPgDeltaNextPlanShadows extends LegacyPgDeltaNextMigrationsShadow { /** Independent platform baseline owned by `planSchemaFiles` while loading desired SQL. */ @@ -33,14 +39,35 @@ interface LegacyPgDeltaNextShadowShape { /** * Provisions only the migrated next-engine shadow needed by database diffs. * The container is removed when the current Effect scope closes. + * Optional Output so schema-first can filter shadow replay without changing live apply. */ readonly provisionMigrations: ( opts: LegacyPgDeltaNextShadowInput, + outputService?: typeof Output.Service, ) => Effect.Effect< LegacyPgDeltaNextMigrationsShadow, LegacyDeclarativeShadowDbError, Scope.Scope >; + /** + * Platform baseline only: no project migrations. Shares the migrations cache + * key (`webhooks: config`). Removed when the current Effect scope closes. + */ + readonly provisionPlatform: ( + opts: LegacyPgDeltaNextShadowInput, + ) => Effect.Effect; + /** + * Platform baseline with webhooks disabled and no project migrations. Image + * extensions stay installed; declaration prep runs later when files are known. + * Removed when the current Effect scope closes. + */ + readonly provisionDeclarative: ( + opts: LegacyPgDeltaNextShadowInput, + ) => Effect.Effect< + { readonly declarativeUrl: string }, + LegacyDeclarativeShadowDbError, + Scope.Scope + >; /** * Provisions the independent migrated and declarative shadows needed by a * declarative plan. Concurrency is strategy-driven (see diff --git a/apps/cli/src/legacy/commands/link/SIDE_EFFECTS.md b/apps/cli/src/legacy/commands/link/SIDE_EFFECTS.md index 20837cfca3..0198af7949 100644 --- a/apps/cli/src/legacy/commands/link/SIDE_EFFECTS.md +++ b/apps/cli/src/legacy/commands/link/SIDE_EFFECTS.md @@ -11,12 +11,12 @@ counterpart exists for this behavior. ## Files Read -| Path | Format | When | -| ------------------------------------ | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `supabase/config.toml` | TOML (`project_id`) | NOT read for ref resolution itself — read by `LegacyCliConfig` for workdir/project-id discovery generally (`--workdir` resolution, `SUPABASE_PROJECT_ID` passthrough) | -| `supabase/.temp/linked-project.json` | JSON (`ref` field) | only when the given `[ref-or-branch]`/`--project-ref` value is not ref-shaped, as the 2nd parent-project candidate for branch-name resolution (CLI-2167, TS-only) | -| `supabase/.temp/project-ref` | plain text | only when the given `[ref-or-branch]`/`--project-ref` value is not ref-shaped, as the 3rd (last) parent-project candidate for branch-name resolution (CLI-2167, TS-only) | -| `~/.supabase/access-token` | plain text | when `SUPABASE_ACCESS_TOKEN` is unset and the keyring is unavailable | +| Path | Format | When | +| ------------------------------------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `supabase/config.toml` | TOML (`project_id`, `db.major_version`) | NOT read for ref resolution. `LegacyCliConfig` uses it for workdir/project-id. After writing `postgres-version`, `link` also reads `[db] major_version` so it can align the shadow major to the remote. | +| `supabase/.temp/linked-project.json` | JSON (`ref` field) | only when the given `[ref-or-branch]`/`--project-ref` value is not ref-shaped, as the 2nd parent-project candidate for branch-name resolution (CLI-2167, TS-only) | +| `supabase/.temp/project-ref` | plain text | only when the given `[ref-or-branch]`/`--project-ref` value is not ref-shaped, as the 3rd (last) parent-project candidate for branch-name resolution (CLI-2167, TS-only) | +| `~/.supabase/access-token` | plain text | when `SUPABASE_ACCESS_TOKEN` is unset and the keyring is unavailable | > For resolving the final linked ref, the on-disk `supabase/.temp/project-ref` file is **not** > read — `link` never falls back to it there. It **is** read for the TS-only branch-name lookup above, which resolves @@ -45,6 +45,7 @@ All under `/supabase/.temp/` (plain text, created with parent dirs as n | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `project-ref` | always, after services link (mandatory — a write failure fails the command) | | `postgres-version` | when the project status is 200 and `database.version` is non-empty | +| `../config.toml` | when `postgres-version` is written and `[db] major_version` differs from the remote major — rewritten in place. Prints `Shadow major is now N (was M). The running local database is still M. Next: supabase db reset` | | `storage-migration` | best-effort — storage config `migrationVersion` | | `pooler-url` | best-effort — processed PRIMARY pooler connection string; **removed** when `--skip-pooler` | | `rest-version` | best-effort — PostgREST swagger `info.version`, prefixed `v` | @@ -141,6 +142,8 @@ Tenant service gateway (`https://.`, `apikey: ` + - spinner: `Resolving branch...` while listing branches for a branch-name lookup (CLI-2167, TS-only; suppressed in `json`/`stream-json` mode like every other `output.task`). - stdout: `Finished supabase link.` +- stdout (info): `Shadow major is now N (was M). The running local database is still M. Next: supabase db reset` when + `config.toml` `[db] major_version` was rewritten to match the remote. ### `--output-format json` / `stream-json` @@ -153,10 +156,10 @@ in these modes (stderr in `json`; a structured `log` event in `stream-json`) rat ## Known divergence -- The cosmetic `WARNING: Local database version differs from the linked project.` message is - **not** reproduced: it requires loading the local `config.toml` `[db].major_version` with CLI - defaults, which the legacy shell does not surface. The `postgres-version` file (the meaningful - side effect) is still written. +- When `postgres-version` is written and `[db] major_version` in `supabase/config.toml` differs + from the remote major, `link` rewrites the TOML and prints + `Shadow major is now N (was M). The running local database is still M. Next: supabase db reset`. The old Go CLI only + warned that the versions differed. - The `Finished supabase link.` line is emitted as **plain text**; the old Go CLI rendered `supabase link` in ANSI cyan. This matches the established legacy-port convention (color helpers are rendered plain); ANSI-stripping scripts are unaffected. diff --git a/apps/cli/src/legacy/commands/link/link.handler.ts b/apps/cli/src/legacy/commands/link/link.handler.ts index 2f506b8fcb..63e277c44d 100644 --- a/apps/cli/src/legacy/commands/link/link.handler.ts +++ b/apps/cli/src/legacy/commands/link/link.handler.ts @@ -11,6 +11,11 @@ import { } from "../../config/legacy-project-ref.service.ts"; import { LegacyLinkedProjectCache } from "../../telemetry/legacy-linked-project-cache.service.ts"; import { LegacyTelemetryState } from "../../telemetry/legacy-telemetry-state.service.ts"; +import { + alignConfigPostgresMajor, + formatShadowMajorAlignedMessage, + parsePostgresMajor, +} from "../../../shared/migrations/remote-postgres.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { Analytics } from "../../../shared/telemetry/analytics.service.ts"; import { withAnalyticsContext } from "../../../shared/telemetry/analytics-context.ts"; @@ -351,6 +356,16 @@ export const legacyLink = Effect.fn("legacy.link")(function* (flags: LegacyLinkF const version = project.value.database.version; if (version.length > 0) { yield* writeTempFile(paths.postgresVersion, version); + const remoteMajor = parsePostgresMajor(version); + if (remoteMajor !== undefined) { + const configPath = path.join(cliConfig.workdir, "supabase", "config.toml"); + const toml = yield* fs.readFileString(configPath).pipe(Effect.orElseSucceed(() => "")); + const aligned = alignConfigPostgresMajor(toml, remoteMajor); + if (aligned !== undefined) { + yield* fs.writeFileString(configPath, aligned.toml); + yield* output.info(formatShadowMajorAlignedMessage(remoteMajor, aligned.previousMajor)); + } + } } } diff --git a/apps/cli/src/legacy/commands/link/link.integration.test.ts b/apps/cli/src/legacy/commands/link/link.integration.test.ts index 548b065e2c..25a6ae96c7 100644 --- a/apps/cli/src/legacy/commands/link/link.integration.test.ts +++ b/apps/cli/src/legacy/commands/link/link.integration.test.ts @@ -370,6 +370,35 @@ describe("legacy link integration", () => { }).pipe(Effect.provide(layer)); }); + it.live("aligns config.toml major_version to the linked database", () => { + const { layer, out, workdir } = setup({ + project: { + ok: { + ...HEALTHY_PROJECT, + database: { ...HEALTHY_PROJECT.database, version: "17.6.1.090" }, + }, + }, + }); + mkdirSync(join(workdir, "supabase"), { recursive: true }); + writeFileSync( + join(workdir, "supabase", "config.toml"), + 'project_id = "x"\n[db]\nmajor_version = 15\n', + ); + return Effect.gen(function* () { + yield* legacyLink(flags()); + expect(readFileSync(join(workdir, "supabase", "config.toml"), "utf8")).toContain( + "major_version = 17", + ); + expect(out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: + "Shadow major is now 17 (was 15). The running local database is still 15. Next: supabase db reset", + }), + ); + }).pipe(Effect.provide(layer)); + }); + it.live("writes linked-project.json with ref/name/org metadata", () => { const { layer, workdir } = setup(); return Effect.gen(function* () { diff --git a/apps/cli/src/legacy/commands/migration/migration.command.ts b/apps/cli/src/legacy/commands/migration/migration.command.ts index ff92238ecb..d095e97f93 100644 --- a/apps/cli/src/legacy/commands/migration/migration.command.ts +++ b/apps/cli/src/legacy/commands/migration/migration.command.ts @@ -10,7 +10,6 @@ import { legacyMigrationFetchCommand } from "./fetch/fetch.command.ts"; export const legacyMigrationCommand = Command.make("migration").pipe( Command.withDescription("Manage database migration scripts."), Command.withShortDescription("Manage database migration scripts"), - Command.withAlias("migrations"), Command.withSubcommands([ legacyMigrationListCommand, legacyMigrationNewCommand, diff --git a/apps/cli/src/legacy/commands/migration/migration.integration.test.ts b/apps/cli/src/legacy/commands/migration/migration.integration.test.ts index dc67ea14fa..b26fc71456 100644 --- a/apps/cli/src/legacy/commands/migration/migration.integration.test.ts +++ b/apps/cli/src/legacy/commands/migration/migration.integration.test.ts @@ -5,37 +5,48 @@ import { CliOutput, Command } from "effect/unstable/cli"; import { textCliOutputFormatter } from "../../../shared/output/text-formatter.ts"; import { LEGACY_GLOBAL_FLAGS } from "../../../shared/legacy/global-flags.ts"; import { legacyMigrationCommand } from "./migration.command.ts"; +import { legacyMigrationsCommand } from "../migrations/migrations.command.ts"; // `withGlobalFlags` must come AFTER `withSubcommands` — see // `start.string-slice-flags.integration.test.ts`'s identical comment. const legacyTestRoot = Command.make("supabase").pipe( - Command.withSubcommands([legacyMigrationCommand]), + Command.withSubcommands([legacyMigrationCommand, legacyMigrationsCommand]), Command.withGlobalFlags(LEGACY_GLOBAL_FLAGS), ); -describe("legacy migration command integration", () => { - it.live("accepts the Go-compatible plural migrations alias", () => { - // After CLI-1969, `squash` is native and no `migration` subcommand is proxied - // any more — so the plural alias is now proven at the PARSER instead: a - // `migrations squash --nope` must fail with squash's own unknown-flag error, - // which never builds the command's `Command.provide` runtime layer. +describe("legacy migration and migrations commands", () => { + it.live("keeps singular migration as the Go-parity group", () => { const run = Effect.gen(function* () { const exit = yield* Command.runWith(legacyTestRoot, { version: "0.0.0-test" })([ - "migrations", + "migration", "squash", "--nope", ]).pipe(Effect.exit); expect(Exit.isFailure(exit)).toBe(true); if (Exit.isFailure(exit)) { const causeJson = JSON.stringify(exit.cause); - // The alias resolved: the parse error is scoped to the squash LEAF, not the root. expect(causeJson).toContain('"commandPath":["supabase","migration","squash"]'); - expect(causeJson).not.toContain('"subcommand":"migrations"'); } }).pipe(Effect.provide(CliOutput.layer(textCliOutputFormatter()))); - // Command.runWith's Environment type is retained even though this path only needs CliOutput - // at runtime. + return run as Effect.Effect; + }); + + it.live("routes plural migrations to the schema-first group", () => { + const run = Effect.gen(function* () { + const exit = yield* Command.runWith(legacyTestRoot, { version: "0.0.0-test" })([ + "migrations", + "apply", + "--nope", + ]).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const causeJson = JSON.stringify(exit.cause); + expect(causeJson).toContain('"commandPath":["supabase","migrations","apply"]'); + expect(causeJson).not.toContain('"commandPath":["supabase","migration","apply"]'); + } + }).pipe(Effect.provide(CliOutput.layer(textCliOutputFormatter()))); + return run as Effect.Effect; }); }); diff --git a/apps/cli/src/legacy/commands/migration/repair/repair.command.ts b/apps/cli/src/legacy/commands/migration/repair/repair.command.ts index c698793b63..17c1c11582 100644 --- a/apps/cli/src/legacy/commands/migration/repair/repair.command.ts +++ b/apps/cli/src/legacy/commands/migration/repair/repair.command.ts @@ -39,7 +39,10 @@ const config = { } as const; export const legacyMigrationRepairCommand = Command.make("repair", config).pipe( - Command.withDescription("Repair the migration history table."), + Command.withDescription( + "Repair the migration history table.\n\n" + + "--status applied upserts version, name, and statements from the local file and does not run SQL.", + ), Command.withShortDescription("Repair the migration history table"), Command.withHandler((flags) => legacyMigrationRepair({ diff --git a/apps/cli/src/legacy/commands/migrations/apply/apply.command.ts b/apps/cli/src/legacy/commands/migrations/apply/apply.command.ts new file mode 100644 index 0000000000..d6df48f187 --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/apply/apply.command.ts @@ -0,0 +1,24 @@ +import { Command } from "effect/unstable/cli"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withLegacyCommandInstrumentation } from "../../../telemetry/legacy-command-instrumentation.ts"; +import { legacySchemaRuntimeLayer } from "../../../schema/legacy-schema-runtime.layer.ts"; +import { legacyMigrationsApply } from "./apply.handler.ts"; + +export const legacyMigrationsApplyCommand = Command.make("apply").pipe( + Command.withDescription( + "Apply pending migration files to the local database.\n\n" + + "This runs the files in supabase/migrations. It does not read supabase/schemas.\n" + + "To try declaration edits without a migration yet, use schema apply.", + ), + Command.withShortDescription("Apply pending migrations locally"), + Command.withExamples([ + { + command: "supabase migrations apply", + description: "Apply pending files to the local database", + }, + ]), + Command.withHandler(() => + legacyMigrationsApply().pipe(withLegacyCommandInstrumentation(), withJsonErrorHandling), + ), + Command.provide(legacySchemaRuntimeLayer(["migrations", "apply"])), +); diff --git a/apps/cli/src/legacy/commands/migrations/apply/apply.handler.ts b/apps/cli/src/legacy/commands/migrations/apply/apply.handler.ts new file mode 100644 index 0000000000..28e38d99d8 --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/apply/apply.handler.ts @@ -0,0 +1,8 @@ +import { Effect } from "effect"; +import { applyMigrations } from "../../../../shared/migrations/apply-migrations.ts"; +import { renderSchemaResult } from "../../../../shared/schema/schema-render.ts"; + +export const legacyMigrationsApply = Effect.fn("legacy.migrations.apply")(function* () { + const result = yield* applyMigrations(); + yield* renderSchemaResult("Apply migrations", result); +}); diff --git a/apps/cli/src/legacy/commands/migrations/diff/diff.command.ts b/apps/cli/src/legacy/commands/migrations/diff/diff.command.ts new file mode 100644 index 0000000000..6311b2fce2 --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/diff/diff.command.ts @@ -0,0 +1,42 @@ +import { Command, Flag } from "effect/unstable/cli"; +import type * as CliCommand from "effect/unstable/cli/Command"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withLegacyCommandInstrumentation } from "../../../telemetry/legacy-command-instrumentation.ts"; +import { legacySchemaRuntimeLayer } from "../../../schema/legacy-schema-runtime.layer.ts"; +import { legacyMigrationsDiff } from "./diff.handler.ts"; + +const config = { + against: Flag.string("against").pipe( + Flag.withDescription( + "Live database to compare. Defaults to local. Also accepts linked or a connection string.", + ), + Flag.withDefault("local"), + ), + file: Flag.string("file").pipe( + Flag.withDescription("Write preview SQL to a file without applying it."), + Flag.withAlias("f"), + Flag.optional, + ), +} as const; + +export type LegacyMigrationsDiffFlags = CliCommand.Command.Config.Infer; + +export const legacyMigrationsDiffCommand = Command.make("diff", config).pipe( + Command.withDescription( + "Preview the SQL that would make migration replay match a live database.\n\n" + + "Does not apply anything. Defaults to the local database.\n\n" + + "To record a live edit, write --file then `migration repair --status applied` (repair upserts statements and does not run SQL).", + ), + Command.withShortDescription("Preview drift between migrations and a database"), + Command.withExamples([ + { command: "supabase migrations diff", description: "Preview local drift" }, + { command: "supabase migrations diff --against linked", description: "Preview remote drift" }, + ]), + Command.withHandler((flags) => + legacyMigrationsDiff(flags).pipe( + withLegacyCommandInstrumentation({ flags, config, aliases: { f: "file" } }), + withJsonErrorHandling, + ), + ), + Command.provide(legacySchemaRuntimeLayer(["migrations", "diff"])), +); diff --git a/apps/cli/src/legacy/commands/migrations/diff/diff.handler.ts b/apps/cli/src/legacy/commands/migrations/diff/diff.handler.ts new file mode 100644 index 0000000000..fe474857e2 --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/diff/diff.handler.ts @@ -0,0 +1,14 @@ +import { Effect, Option } from "effect"; +import { diffMigrations } from "../../../../shared/migrations/diff-migrations.ts"; +import { renderSchemaResult } from "../../../../shared/schema/schema-render.ts"; +import type { LegacyMigrationsDiffFlags } from "./diff.command.ts"; + +export const legacyMigrationsDiff = Effect.fn("legacy.migrations.diff")(function* ( + flags: LegacyMigrationsDiffFlags, +) { + const result = yield* diffMigrations({ + against: flags.against, + file: Option.getOrUndefined(flags.file), + }); + yield* renderSchemaResult("Diff migrations", result); +}); diff --git a/apps/cli/src/legacy/commands/migrations/list/list.command.ts b/apps/cli/src/legacy/commands/migrations/list/list.command.ts new file mode 100644 index 0000000000..1c74a6ef0e --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/list/list.command.ts @@ -0,0 +1,43 @@ +import { Command, Flag } from "effect/unstable/cli"; +import type * as CliCommand from "effect/unstable/cli/Command"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withLegacyCommandInstrumentation } from "../../../telemetry/legacy-command-instrumentation.ts"; +import { legacySchemaRuntimeLayer } from "../../../schema/legacy-schema-runtime.layer.ts"; +import { legacyMigrationsList } from "./list.handler.ts"; + +const config = { + against: Flag.string("against").pipe( + Flag.withDescription( + "Database to compare. Defaults to local. Also accepts linked or a connection string.", + ), + Flag.withDefault("local"), + ), +} as const; + +export type LegacyMigrationsListFlags = CliCommand.Command.Config.Infer; + +export const legacyMigrationsListCommand = Command.make("list", config).pipe( + Command.withDescription( + "Show which migration files are applied, pending, or remote-only.\n\n" + + "This is history alignment (file versions vs the database history table), not a schema diff.\n\n" + + "Defaults to the local database. Pass --against linked to list the linked project.", + ), + Command.withShortDescription("List applied, pending, and remote-only migrations"), + Command.withExamples([ + { + command: "supabase migrations list", + description: "Show applied vs pending on the local database", + }, + { + command: "supabase migrations list --against linked", + description: "Show history alignment on the linked project", + }, + ]), + Command.withHandler((flags) => + legacyMigrationsList(flags).pipe( + withLegacyCommandInstrumentation({ flags, config }), + withJsonErrorHandling, + ), + ), + Command.provide(legacySchemaRuntimeLayer(["migrations", "list"])), +); diff --git a/apps/cli/src/legacy/commands/migrations/list/list.handler.ts b/apps/cli/src/legacy/commands/migrations/list/list.handler.ts new file mode 100644 index 0000000000..14e0fb0098 --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/list/list.handler.ts @@ -0,0 +1,11 @@ +import { Effect } from "effect"; +import { listMigrations } from "../../../../shared/migrations/list-migrations.ts"; +import { renderSchemaResult } from "../../../../shared/schema/schema-render.ts"; +import type { LegacyMigrationsListFlags } from "./list.command.ts"; + +export const legacyMigrationsList = Effect.fn("legacy.migrations.list")(function* ( + flags: LegacyMigrationsListFlags, +) { + const result = yield* listMigrations({ against: flags.against }); + yield* renderSchemaResult("List migrations", result); +}); diff --git a/apps/cli/src/legacy/commands/migrations/migrations.command.ts b/apps/cli/src/legacy/commands/migrations/migrations.command.ts new file mode 100644 index 0000000000..5028861d3c --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/migrations.command.ts @@ -0,0 +1,23 @@ +import { Command } from "effect/unstable/cli"; +import { legacyMigrationsApplyCommand } from "./apply/apply.command.ts"; +import { legacyMigrationsDiffCommand } from "./diff/diff.command.ts"; +import { legacyMigrationsListCommand } from "./list/list.command.ts"; +import { legacyMigrationsNewCommand } from "./new/new.command.ts"; +import { legacyMigrationsPullCommand } from "./pull/pull.command.ts"; +import { legacyMigrationsPushCommand } from "./push/push.command.ts"; + +export const legacyMigrationsCommand = Command.make("migrations").pipe( + Command.withDescription( + "Work with supabase/migrations as the deployment recipe.\n\n" + + "These commands do not load supabase/schemas. The only command that changes a remote schema is migrations push.", + ), + Command.withShortDescription("Manage migration files and history"), + Command.withSubcommands([ + legacyMigrationsNewCommand, + legacyMigrationsListCommand, + legacyMigrationsDiffCommand, + legacyMigrationsApplyCommand, + legacyMigrationsPushCommand, + legacyMigrationsPullCommand, + ]), +); diff --git a/apps/cli/src/legacy/commands/migrations/new/new.command.ts b/apps/cli/src/legacy/commands/migrations/new/new.command.ts new file mode 100644 index 0000000000..83c50ebc2d --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/new/new.command.ts @@ -0,0 +1,47 @@ +import { Argument, Command, Flag } from "effect/unstable/cli"; +import type * as CliCommand from "effect/unstable/cli/Command"; +import { REVOKE_API_PRIVILEGES_TEMPLATE } from "../../../../shared/migrations/privilege-offer.ts"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withLegacyCommandInstrumentation } from "../../../telemetry/legacy-command-instrumentation.ts"; +import { legacySchemaRuntimeLayer } from "../../../schema/legacy-schema-runtime.layer.ts"; +import { legacyMigrationsNew } from "./new.handler.ts"; + +const TEMPLATES = [REVOKE_API_PRIVILEGES_TEMPLATE] as const; + +const config = { + name: Argument.string("name").pipe( + Argument.withDescription("Migration name."), + Argument.optional, + ), + template: Flag.choice("template", TEMPLATES).pipe( + Flag.withDescription( + "Seed the file. revoke-api-privileges writes the turn-off revoke SQL (no paste).", + ), + Flag.optional, + ), +} as const; + +export type LegacyMigrationsNewFlags = CliCommand.Command.Config.Infer; + +export const legacyMigrationsNewCommand = Command.make("new", config).pipe( + Command.withDescription( + "Create a migration file to write by hand.\n\n" + + "Prefer schema generate when the change lives in supabase/schemas.\n" + + "`--template revoke-api-privileges` seeds the turn-off revoke SQL.", + ), + Command.withShortDescription("Create a migration file"), + Command.withExamples([ + { + command: "supabase migrations new add_custom_data", + description: "Create supabase/migrations/_add_custom_data.sql", + }, + { + command: `supabase migrations new revoke_api_privileges --template ${REVOKE_API_PRIVILEGES_TEMPLATE}`, + description: "Create the turn-off revoke file, then migrations push", + }, + ]), + Command.withHandler((flags) => + legacyMigrationsNew(flags).pipe(withLegacyCommandInstrumentation(), withJsonErrorHandling), + ), + Command.provide(legacySchemaRuntimeLayer(["migrations", "new"])), +); diff --git a/apps/cli/src/legacy/commands/migrations/new/new.handler.ts b/apps/cli/src/legacy/commands/migrations/new/new.handler.ts new file mode 100644 index 0000000000..22fe4ef7d1 --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/new/new.handler.ts @@ -0,0 +1,14 @@ +import { Effect, Option } from "effect"; +import { newMigration } from "../../../../shared/migrations/new-migration.ts"; +import { renderSchemaResult } from "../../../../shared/schema/schema-render.ts"; +import type { LegacyMigrationsNewFlags } from "./new.command.ts"; + +export const legacyMigrationsNew = Effect.fn("legacy.migrations.new")(function* ( + flags: LegacyMigrationsNewFlags, +) { + const result = yield* newMigration({ + name: Option.getOrUndefined(flags.name), + template: Option.getOrUndefined(flags.template), + }); + yield* renderSchemaResult("Create migration", result); +}); diff --git a/apps/cli/src/legacy/commands/migrations/pull/pull.command.ts b/apps/cli/src/legacy/commands/migrations/pull/pull.command.ts new file mode 100644 index 0000000000..c1ddff022f --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/pull/pull.command.ts @@ -0,0 +1,38 @@ +import { Command, Flag } from "effect/unstable/cli"; +import type * as CliCommand from "effect/unstable/cli/Command"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withLegacyCommandInstrumentation } from "../../../telemetry/legacy-command-instrumentation.ts"; +import { legacySchemaRuntimeLayer } from "../../../schema/legacy-schema-runtime.layer.ts"; +import { legacyMigrationsPull } from "./pull.handler.ts"; + +const config = { + from: Flag.string("from").pipe( + Flag.withDescription("Remote database. Defaults to linked. Also accepts a connection string."), + Flag.withDefault("linked"), + ), +} as const; + +export type LegacyMigrationsPullFlags = CliCommand.Command.Config.Infer; + +export const legacyMigrationsPullCommand = Command.make("pull", config).pipe( + Command.withDescription( + "Fetch remote migration history files from schema_migrations (version, name, statements).\n\n" + + "Defaults to the linked project. Writes supabase/migrations/_.sql. Does not execute SQL.\n\n" + + "Same version and SQL is skipped. A SQL mismatch leaves the local file and writes the remote copy under .supabase/remote-migrations/.", + ), + Command.withShortDescription("Fetch remote migration history files"), + Command.withExamples([ + { command: "supabase migrations pull", description: "Fetch linked-project history files" }, + { + command: "supabase migrations pull --from ", + description: "Fetch history files from a connection string", + }, + ]), + Command.withHandler((flags) => + legacyMigrationsPull(flags).pipe( + withLegacyCommandInstrumentation({ flags, config }), + withJsonErrorHandling, + ), + ), + Command.provide(legacySchemaRuntimeLayer(["migrations", "pull"])), +); diff --git a/apps/cli/src/legacy/commands/migrations/pull/pull.handler.ts b/apps/cli/src/legacy/commands/migrations/pull/pull.handler.ts new file mode 100644 index 0000000000..0662a227e9 --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/pull/pull.handler.ts @@ -0,0 +1,13 @@ +import { Effect } from "effect"; +import { pullMigrations } from "../../../../shared/migrations/pull-migrations.ts"; +import { renderSchemaResult } from "../../../../shared/schema/schema-render.ts"; +import type { LegacyMigrationsPullFlags } from "./pull.command.ts"; + +export const legacyMigrationsPull = Effect.fn("legacy.migrations.pull")(function* ( + flags: LegacyMigrationsPullFlags, +) { + const result = yield* pullMigrations({ + from: flags.from, + }); + yield* renderSchemaResult("Pull remote migrations", result); +}); diff --git a/apps/cli/src/legacy/commands/migrations/push/push.command.ts b/apps/cli/src/legacy/commands/migrations/push/push.command.ts new file mode 100644 index 0000000000..947fdc2dff --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/push/push.command.ts @@ -0,0 +1,59 @@ +import { Command, Flag } from "effect/unstable/cli"; +import type * as CliCommand from "effect/unstable/cli/Command"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withLegacyCommandInstrumentation } from "../../../telemetry/legacy-command-instrumentation.ts"; +import { legacySchemaRuntimeLayer } from "../../../schema/legacy-schema-runtime.layer.ts"; +import { legacyMigrationsPush } from "./push.handler.ts"; + +const config = { + yes: Flag.boolean("yes").pipe( + Flag.withDescription( + "Skip the dirty first-push catalog confirm. Still type the project ref unless --project-ref is set. Live verify still runs.", + ), + Flag.withAlias("y"), + ), + projectRef: Flag.string("project-ref").pipe( + Flag.withDescription("Must match the linked project."), + Flag.optional, + ), + allowRemote: Flag.boolean("allow-remote").pipe( + Flag.withDescription("Required when pushing to a raw --db-url connection string."), + ), + dbUrl: Flag.string("db-url").pipe( + Flag.withDescription("Raw connection string. Requires --allow-remote."), + Flag.optional, + ), + skipVerify: Flag.boolean("skip-verify").pipe( + Flag.withDescription( + "Skip checks that schemas and the remote still match the migration files.", + ), + ), +} as const; + +export type LegacyMigrationsPushFlags = CliCommand.Command.Config.Infer; + +export const legacyMigrationsPushCommand = Command.make("push", config).pipe( + Command.withDescription( + "Apply pending migration files to the linked project.\n\n" + + "This is the only CLI command that changes a remote schema. It prints the pending files, then asks you to type the project ref (even with --yes). --yes skips only the dirty-catalog confirm.\n\n" + + "Live-verify must pass unless --skip-verify. Remote-only versions: migrations pull. Histories aligned but catalog differs: privilege offer, or migrations diff --against linked then migration repair --status applied.", + ), + Command.withShortDescription("Push pending migrations to the linked project"), + Command.withExamples([ + { + command: "supabase migrations push", + description: "Preview pending files, confirm the project ref, then apply", + }, + { + command: "supabase migrations push --yes --project-ref ", + description: "Skip the ref prompt by asserting the linked project", + }, + ]), + Command.withHandler((flags) => + legacyMigrationsPush(flags).pipe( + withLegacyCommandInstrumentation({ flags, config, aliases: { y: "yes" } }), + withJsonErrorHandling, + ), + ), + Command.provide(legacySchemaRuntimeLayer(["migrations", "push"])), +); diff --git a/apps/cli/src/legacy/commands/migrations/push/push.handler.ts b/apps/cli/src/legacy/commands/migrations/push/push.handler.ts new file mode 100644 index 0000000000..3ceaa8f9a6 --- /dev/null +++ b/apps/cli/src/legacy/commands/migrations/push/push.handler.ts @@ -0,0 +1,22 @@ +import { Effect, Option } from "effect"; +import { pushMigrations } from "../../../../shared/migrations/push-migrations.ts"; +import { Output } from "../../../../shared/output/output.service.ts"; +import { renderSchemaResult } from "../../../../shared/schema/schema-render.ts"; +import type { LegacyMigrationsPushFlags } from "./push.command.ts"; + +export const legacyMigrationsPush = Effect.fn("legacy.migrations.push")(function* ( + flags: LegacyMigrationsPushFlags, +) { + const output = yield* Output; + if (output.format === "text") { + yield* output.intro("Push migrations"); + } + const result = yield* pushMigrations({ + yes: flags.yes, + projectRef: Option.getOrUndefined(flags.projectRef), + allowRemote: flags.allowRemote, + dbUrl: Option.getOrUndefined(flags.dbUrl), + skipVerify: flags.skipVerify, + }); + yield* renderSchemaResult("Push migrations", result, { skipIntro: true }); +}); diff --git a/apps/cli/src/legacy/commands/schema/apply/apply.command.ts b/apps/cli/src/legacy/commands/schema/apply/apply.command.ts new file mode 100644 index 0000000000..8982fd2b14 --- /dev/null +++ b/apps/cli/src/legacy/commands/schema/apply/apply.command.ts @@ -0,0 +1,21 @@ +import { Command } from "effect/unstable/cli"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withLegacyCommandInstrumentation } from "../../../telemetry/legacy-command-instrumentation.ts"; +import { legacySchemaRuntimeLayer } from "../../../schema/legacy-schema-runtime.layer.ts"; +import { legacySchemaApply } from "./apply.handler.ts"; + +export const legacySchemaApplyCommand = Command.make("apply").pipe( + Command.withDescription( + "Apply supabase/schemas to the local database without writing migration files.\n\n" + + "Use this while iterating. When the local database looks right, run schema generate --name .\n\n" + + "Only the local stack can be the target. Deploy remotes with migrations push.", + ), + Command.withShortDescription("Apply supabase/schemas to the local database"), + Command.withExamples([ + { command: "supabase schema apply", description: "Apply declarations to the local database" }, + ]), + Command.withHandler(() => + legacySchemaApply().pipe(withLegacyCommandInstrumentation(), withJsonErrorHandling), + ), + Command.provide(legacySchemaRuntimeLayer(["schema", "apply"])), +); diff --git a/apps/cli/src/legacy/commands/schema/apply/apply.handler.ts b/apps/cli/src/legacy/commands/schema/apply/apply.handler.ts new file mode 100644 index 0000000000..e29bbf80f4 --- /dev/null +++ b/apps/cli/src/legacy/commands/schema/apply/apply.handler.ts @@ -0,0 +1,8 @@ +import { Effect } from "effect"; +import { applySchema } from "../../../../shared/schema/apply-schema.ts"; +import { renderSchemaResult } from "../../../../shared/schema/schema-render.ts"; + +export const legacySchemaApply = Effect.fn("legacy.schema.apply")(function* () { + const result = yield* applySchema(); + yield* renderSchemaResult("Apply declarative schema", result); +}); diff --git a/apps/cli/src/legacy/commands/schema/generate/generate.command.ts b/apps/cli/src/legacy/commands/schema/generate/generate.command.ts new file mode 100644 index 0000000000..6c85bc5968 --- /dev/null +++ b/apps/cli/src/legacy/commands/schema/generate/generate.command.ts @@ -0,0 +1,53 @@ +import { Command, Flag } from "effect/unstable/cli"; +import type * as CliCommand from "effect/unstable/cli/Command"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withLegacyCommandInstrumentation } from "../../../telemetry/legacy-command-instrumentation.ts"; +import { legacySchemaRuntimeLayer } from "../../../schema/legacy-schema-runtime.layer.ts"; +import { legacySchemaGenerate } from "./generate.handler.ts"; + +const config = { + name: Flag.string("name").pipe( + Flag.withDescription("Name for the generated migration files."), + Flag.optional, + ), + dryRun: Flag.boolean("dry-run").pipe( + Flag.withDescription("Show the plan without writing migration files."), + ), + baseline: Flag.boolean("baseline").pipe( + Flag.withDescription( + "Write the first migration from supabase/schemas. Refuses if migration files already exist. Use schema pull first if a live database is the source.", + ), + ), +} as const; + +export type LegacySchemaGenerateFlags = CliCommand.Command.Config.Infer; + +export const legacySchemaGenerateCommand = Command.make("generate", config).pipe( + Command.withDescription( + "Turn supabase/schemas into migration files.\n\n" + + "Compares a clean replay of supabase/migrations to supabase/schemas on a local shadow, not the linked project. Does not apply anything to a live database.\n\n" + + "--dry-run prints the plan SQL without writing files. --baseline writes the first migration from supabase/schemas when supabase/migrations is empty.", + ), + Command.withShortDescription("Write migrations from supabase/schemas"), + Command.withExamples([ + { + command: "supabase schema generate --dry-run", + description: "Preview the plan SQL without writing files", + }, + { + command: "supabase schema generate --name add_billing", + description: "Write the migration files", + }, + { + command: "supabase schema generate --baseline --name initial_schema", + description: "Write the first migration from supabase/schemas", + }, + ]), + Command.withHandler((flags) => + legacySchemaGenerate(flags).pipe( + withLegacyCommandInstrumentation({ flags, config }), + withJsonErrorHandling, + ), + ), + Command.provide(legacySchemaRuntimeLayer(["schema", "generate"])), +); diff --git a/apps/cli/src/legacy/commands/schema/generate/generate.handler.ts b/apps/cli/src/legacy/commands/schema/generate/generate.handler.ts new file mode 100644 index 0000000000..7f0d024e5e --- /dev/null +++ b/apps/cli/src/legacy/commands/schema/generate/generate.handler.ts @@ -0,0 +1,15 @@ +import { Effect, Option } from "effect"; +import { generateSchema } from "../../../../shared/schema/generate-schema.ts"; +import { renderSchemaResult } from "../../../../shared/schema/schema-render.ts"; +import type { LegacySchemaGenerateFlags } from "./generate.command.ts"; + +export const legacySchemaGenerate = Effect.fn("legacy.schema.generate")(function* ( + flags: LegacySchemaGenerateFlags, +) { + const result = yield* generateSchema({ + name: Option.getOrUndefined(flags.name), + dryRun: flags.dryRun, + baseline: flags.baseline, + }); + yield* renderSchemaResult("Generate schema migrations", result); +}); diff --git a/apps/cli/src/legacy/commands/schema/pull/pull.command.ts b/apps/cli/src/legacy/commands/schema/pull/pull.command.ts new file mode 100644 index 0000000000..1bf23a29f0 --- /dev/null +++ b/apps/cli/src/legacy/commands/schema/pull/pull.command.ts @@ -0,0 +1,63 @@ +import { Command, Flag } from "effect/unstable/cli"; +import type * as CliCommand from "effect/unstable/cli/Command"; +import { withJsonErrorHandling } from "../../../../shared/output/json-error-handling.ts"; +import { withLegacyCommandInstrumentation } from "../../../telemetry/legacy-command-instrumentation.ts"; +import { legacySchemaRuntimeLayer } from "../../../schema/legacy-schema-runtime.layer.ts"; +import { legacySchemaPull } from "./pull.handler.ts"; + +const config = { + from: Flag.string("from").pipe( + Flag.withDescription( + "Source database. Defaults to local. Also accepts linked or a connection string.", + ), + Flag.withDefault("local"), + ), + output: Flag.string("output").pipe( + Flag.withDescription("Write a side-by-side snapshot instead of replacing supabase/schemas."), + Flag.optional, + ), + force: Flag.boolean("force").pipe( + Flag.withDescription( + "Replace managed files in supabase/schemas. Not a merge. _custom/ is left alone.", + ), + ), + pruneUnmanaged: Flag.boolean("prune-unmanaged").pipe( + Flag.withDescription( + "Delete unmanaged .sql files that are not owned by the export or _custom/.", + ), + ), +} as const; + +export type LegacySchemaPullFlags = CliCommand.Command.Config.Infer; + +export const legacySchemaPullCommand = Command.make("pull", config).pipe( + Command.withDescription( + "Write supabase/schemas from a live database.\n\n" + + "Defaults to the local database. Use --from linked or a connection string to pull a remote instead.\n\n" + + "The database wins: pull replaces managed files and never merges. Files in _custom/ are left alone.\n\n" + + "If supabase/schemas already exists, pass --force to replace it, or --output for a side-by-side copy.", + ), + Command.withShortDescription("Write supabase/schemas from a database"), + Command.withExamples([ + { command: "supabase schema pull", description: "Export the local database" }, + { + command: "supabase schema pull --from linked", + description: "Export the linked project database", + }, + { + command: "supabase schema pull --from linked --output supabase/schemas.remote", + description: "Write a side-by-side remote snapshot", + }, + { + command: "supabase schema pull --force", + description: "Replace the managed schema tree from local", + }, + ]), + Command.withHandler((flags) => + legacySchemaPull(flags).pipe( + withLegacyCommandInstrumentation({ flags, config }), + withJsonErrorHandling, + ), + ), + Command.provide(legacySchemaRuntimeLayer(["schema", "pull"])), +); diff --git a/apps/cli/src/legacy/commands/schema/pull/pull.handler.ts b/apps/cli/src/legacy/commands/schema/pull/pull.handler.ts new file mode 100644 index 0000000000..655f9a582b --- /dev/null +++ b/apps/cli/src/legacy/commands/schema/pull/pull.handler.ts @@ -0,0 +1,16 @@ +import { Effect, Option } from "effect"; +import { pullSchema } from "../../../../shared/schema/pull-schema.ts"; +import { renderSchemaResult } from "../../../../shared/schema/schema-render.ts"; +import type { LegacySchemaPullFlags } from "./pull.command.ts"; + +export const legacySchemaPull = Effect.fn("legacy.schema.pull")(function* ( + flags: LegacySchemaPullFlags, +) { + const result = yield* pullSchema({ + from: flags.from, + output: Option.getOrUndefined(flags.output), + force: flags.force, + pruneUnmanaged: flags.pruneUnmanaged, + }); + yield* renderSchemaResult("Pull declarative schema", result); +}); diff --git a/apps/cli/src/legacy/commands/schema/schema.command.ts b/apps/cli/src/legacy/commands/schema/schema.command.ts new file mode 100644 index 0000000000..ad2a7777cd --- /dev/null +++ b/apps/cli/src/legacy/commands/schema/schema.command.ts @@ -0,0 +1,21 @@ +import { Command } from "effect/unstable/cli"; +import { legacySchemaApplyCommand } from "./apply/apply.command.ts"; +import { legacySchemaGenerateCommand } from "./generate/generate.command.ts"; +import { legacySchemaPullCommand } from "./pull/pull.command.ts"; + +export const legacySchemaCommand = Command.make("schema").pipe( + Command.withDescription( + "Edit database shape as SQL in supabase/schemas.\n\n" + + "Typical loop:\n" + + " schema pull Capture a database into supabase/schemas\n" + + " schema apply Try declaration edits on the local database\n" + + " schema generate Write a reviewable migration when the shape is right\n\n" + + "schema apply never touches a remote. Deploy remotes with migrations push.", + ), + Command.withShortDescription("Edit database shape in supabase/schemas"), + Command.withSubcommands([ + legacySchemaPullCommand, + legacySchemaGenerateCommand, + legacySchemaApplyCommand, + ]), +); diff --git a/apps/cli/src/legacy/docs/legacy-docs-spec.tables.ts b/apps/cli/src/legacy/docs/legacy-docs-spec.tables.ts index d3648e8ad7..5496c6d622 100644 --- a/apps/cli/src/legacy/docs/legacy-docs-spec.tables.ts +++ b/apps/cli/src/legacy/docs/legacy-docs-spec.tables.ts @@ -56,11 +56,13 @@ export const LEGACY_DOCS_TAGS: Readonly>> = "supabase-login": ["local-dev"], "supabase-logout": ["local-dev"], "supabase-migration": ["local-dev"], + "supabase-migrations": ["local-dev"], "supabase-network-bans": ["management-api"], "supabase-network-restrictions": ["management-api"], "supabase-orgs": ["management-api"], "supabase-postgres-config": ["management-api"], "supabase-projects": ["management-api"], + "supabase-schema": ["local-dev"], "supabase-secrets": ["management-api"], "supabase-seed": ["local-dev"], "supabase-services": ["local-dev"], diff --git a/apps/cli/src/legacy/schema/legacy-docker-isolated-shadow.layer.ts b/apps/cli/src/legacy/schema/legacy-docker-isolated-shadow.layer.ts new file mode 100644 index 0000000000..9d312a9bea --- /dev/null +++ b/apps/cli/src/legacy/schema/legacy-docker-isolated-shadow.layer.ts @@ -0,0 +1,101 @@ +import { Effect, FileSystem, Layer, Option, Path } from "effect"; +import { explicitBooleanLongFlag } from "../../shared/cli/cobra-flag-groups.ts"; +import { Output } from "../../shared/output/output.service.ts"; +import { IsolatedShadowProvisioner } from "../../shared/schema/isolated-shadow.service.ts"; +import { SchemaEngineError } from "../../shared/schema/schema-errors.ts"; +import { wrapShadowReplayOutput } from "../../shared/schema/shadow-replay-output.ts"; +import { LegacyCliConfig } from "../config/legacy-cli-config.service.ts"; +import { legacyReadDbToml } from "../shared/legacy-db-config.toml-read.ts"; +import { + legacyResolvePgDeltaProjectId, + type LegacyPgDeltaContext, +} from "../shared/legacy-pgdelta.ts"; +import { LegacyPgDeltaNextShadow } from "../commands/db/shared/legacy-pgdelta-next-shadow.service.ts"; +import type { LegacyPgDeltaNextShadowInput } from "../commands/db/shared/legacy-pgdelta-next-shadow.service.ts"; +import type { LegacyDeclarativeShadowDbError } from "../commands/db/shared/legacy-pgdelta.errors.ts"; + +const LEGACY_SHADOW_DAEMON_HINT = "Start Docker Desktop or Podman, then retry."; +const LEGACY_SHADOW_RESTORE_HINT = + "Retry the command. If it persists, delete ~/.supabase/cache/shadow-baseline and rerun."; +const LEGACY_SHADOW_RETRY_HINT = "Retry the command."; + +/** True only for an existing-tar restore failure — a cold/create miss never uses these phrases. */ +function isShadowBaselineRestoreFailure(message: string): boolean { + return ( + message.includes("failed to restore shadow baseline") || + message.includes("failed to restore archive into container") + ); +} + +export const legacyIsolatedShadowToEngineError = (error: LegacyDeclarativeShadowDbError) => + new SchemaEngineError({ + detail: error.message, + suggestion: + error.suggestion ?? + (error.docker === "daemon" + ? LEGACY_SHADOW_DAEMON_HINT + : isShadowBaselineRestoreFailure(error.message) + ? LEGACY_SHADOW_RESTORE_HINT + : LEGACY_SHADOW_RETRY_HINT), + }); + +const tomlError = (cause: unknown) => + new SchemaEngineError({ + detail: cause instanceof Error ? cause.message : String(cause), + suggestion: "Fix supabase/config.toml, then retry.", + }); + +export const legacyDockerIsolatedShadowLayer = Layer.effect( + IsolatedShadowProvisioner, + Effect.gen(function* () { + const shadows = yield* LegacyPgDeltaNextShadow; + const output = yield* Output; + const config = yield* LegacyCliConfig; + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + + const shadowInput = (): Effect.Effect => + Effect.gen(function* () { + const toml = yield* legacyReadDbToml(fs, path, config.workdir, undefined, { + validate: false, + warnOnUnresolvedEnv: false, + }).pipe(Effect.mapError(tomlError)); + const context: LegacyPgDeltaContext = { + projectId: legacyResolvePgDeltaProjectId(config.projectId, toml, config.workdir), + cwd: config.workdir, + npmVersion: Option.getOrUndefined(toml.pgDelta.npmVersion), + denoVersion: toml.denoVersion, + projectEnv: toml.projectEnv, + }; + return { context, toml } satisfies LegacyPgDeltaNextShadowInput; + }); + + return IsolatedShadowProvisioner.of({ + provision: Effect.gen(function* () { + const opts = yield* shadowInput(); + const shadow = yield* shadows + .provisionDeclarative(opts) + .pipe(Effect.mapError(legacyIsolatedShadowToEngineError)); + return { url: shadow.declarativeUrl }; + }), + provisionPlatform: Effect.gen(function* () { + const opts = yield* shadowInput(); + const shadow = yield* shadows + .provisionPlatform(opts) + .pipe(Effect.mapError(legacyIsolatedShadowToEngineError)); + return { url: shadow.platformUrl }; + }), + provisionMigrations: Effect.gen(function* () { + const opts = yield* shadowInput(); + const wrapped = wrapShadowReplayOutput(output, { + debug: explicitBooleanLongFlag(process.argv, "debug") === true, + }); + // Isolated replay only; live db start / migrations apply stay unprefixed. + const shadow = yield* shadows + .provisionMigrations(opts, wrapped) + .pipe(Effect.mapError(legacyIsolatedShadowToEngineError)); + return { url: shadow.migrationsUrl }; + }), + }); + }), +); diff --git a/apps/cli/src/legacy/schema/legacy-docker-isolated-shadow.layer.unit.test.ts b/apps/cli/src/legacy/schema/legacy-docker-isolated-shadow.layer.unit.test.ts new file mode 100644 index 0000000000..bab16301b8 --- /dev/null +++ b/apps/cli/src/legacy/schema/legacy-docker-isolated-shadow.layer.unit.test.ts @@ -0,0 +1,57 @@ +import { describe, expect, it } from "vitest"; + +import { LegacyDeclarativeShadowDbError } from "../commands/db/shared/legacy-pgdelta.errors.ts"; +import { SchemaEngineError } from "../../shared/schema/schema-errors.ts"; +import { legacyIsolatedShadowToEngineError } from "./legacy-docker-isolated-shadow.layer.ts"; + +const DELETE_CACHE_HINT = "delete ~/.supabase/cache/shadow-baseline"; + +describe("legacyIsolatedShadowToEngineError", () => { + it("keeps a container-exit create-baseline failure in detail and does not suggest deleting the cache", () => { + const mapped = legacyIsolatedShadowToEngineError( + new LegacyDeclarativeShadowDbError({ + message: + "error running container: exit 1:\nStorageBackendError: Migration fix-search-by-timestamp-sqli not found", + }), + ); + expect(mapped).toBeInstanceOf(SchemaEngineError); + expect(mapped.detail).toContain( + "StorageBackendError: Migration fix-search-by-timestamp-sqli not found", + ); + expect(mapped.suggestion).toBe("Retry the command."); + expect(mapped.suggestion).not.toContain(DELETE_CACHE_HINT); + }); + + it.each([ + "failed to restore shadow baseline: docker cp failed", + "failed to create docker container: failed to restore archive into container", + ])("suggests deleting the cache when restoring an existing tar failed: %s", (message) => { + const mapped = legacyIsolatedShadowToEngineError( + new LegacyDeclarativeShadowDbError({ message }), + ); + expect(mapped.suggestion).toContain(DELETE_CACHE_HINT); + }); + + it("keeps the daemon hint when Docker is unreachable", () => { + const mapped = legacyIsolatedShadowToEngineError( + new LegacyDeclarativeShadowDbError({ + message: "Cannot connect to the Docker daemon", + docker: "daemon", + }), + ); + expect(mapped.suggestion).toBe("Start Docker Desktop or Podman, then retry."); + expect(mapped.suggestion).not.toContain(DELETE_CACHE_HINT); + }); + + it("keeps an underlying recovery suggestion when one is already attached", () => { + const mapped = legacyIsolatedShadowToEngineError( + new LegacyDeclarativeShadowDbError({ + message: "container supabase_db_x is not ready: exec format error", + suggestion: "Run `docker image rm public.ecr.aws/supabase/postgres:17.4.1.056` and retry.", + }), + ); + expect(mapped.suggestion).toBe( + "Run `docker image rm public.ecr.aws/supabase/postgres:17.4.1.056` and retry.", + ); + }); +}); diff --git a/apps/cli/src/legacy/schema/legacy-docker-local-database.layer.ts b/apps/cli/src/legacy/schema/legacy-docker-local-database.layer.ts new file mode 100644 index 0000000000..d8c799888f --- /dev/null +++ b/apps/cli/src/legacy/schema/legacy-docker-local-database.layer.ts @@ -0,0 +1,140 @@ +import { isDockerDaemonDownMessage } from "@supabase/stack/effect"; +import { Effect, FileSystem, Layer, Option, Path } from "effect"; +import { ChildProcessSpawner } from "effect/unstable/process"; +import type { DatabaseTarget } from "../../shared/database/database-target.ts"; +import { LocalDatabaseFallback } from "../../shared/database/local-database-fallback.service.ts"; +import { + localPostgresConnectionString, + publishedPostgresHostPort, +} from "../../shared/database/local-postgres-url.ts"; +import { SchemaLocalStackNotRunningError } from "../../shared/schema/schema-errors.ts"; +import { LegacyCliConfig } from "../config/legacy-cli-config.service.ts"; +import { + legacyCollectText, + legacyDescribeContainerCliFailure, + legacyIsContainerNotFoundMessage, + spawnContainerCli, +} from "../shared/legacy-container-cli.ts"; +import { legacyReadDbToml } from "../shared/legacy-db-config.toml-read.ts"; +import { legacyResolveLocalProjectId, localDbContainerId } from "../shared/legacy-docker-ids.ts"; +import { legacyGetHostname } from "../shared/legacy-hostname.ts"; + +type Spawner = ChildProcessSpawner.ChildProcessSpawner["Service"]; + +const inspectPublishedPostgresPort = (spawner: Spawner, containerId: string) => + Effect.scoped( + Effect.gen(function* () { + const spawned = yield* spawnContainerCli( + spawner, + ["container", "inspect", containerId, "--format", "{{json .NetworkSettings.Ports}}"], + { stdin: "ignore", stdout: "pipe", stderr: "pipe" }, + ).pipe( + Effect.map(Option.some), + Effect.catchTag("LegacyContainerRuntimeNotFoundError", () => Effect.succeed(Option.none())), + Effect.mapError((cause) => { + const description = legacyDescribeContainerCliFailure(cause); + return new SchemaLocalStackNotRunningError({ + detail: `failed to inspect local database container: ${description}`, + suggestion: isDockerDaemonDownMessage(description) + ? "Start Docker Desktop or Podman, then run `supabase start`." + : "Run `supabase start`, then retry.", + }); + }), + ); + if (Option.isNone(spawned)) { + return Option.none(); + } + const child = spawned.value; + const [exitCode, stdout, stderr] = yield* Effect.all( + [ + child.exitCode.pipe(Effect.map(Number)), + legacyCollectText(child.stdout), + legacyCollectText(child.stderr), + ], + { concurrency: "unbounded" }, + ).pipe( + Effect.mapError( + () => + new SchemaLocalStackNotRunningError({ + detail: "failed to inspect local database container", + suggestion: "Run `supabase start`, then retry.", + }), + ), + ); + if (exitCode !== 0) { + const message = stderr.trim(); + if (legacyIsContainerNotFoundMessage(message)) { + return Option.none(); + } + return yield* new SchemaLocalStackNotRunningError({ + detail: + message.length > 0 + ? `failed to inspect local database container: ${message}` + : "failed to inspect local database container", + suggestion: isDockerDaemonDownMessage(message) + ? "Start Docker Desktop or Podman, then run `supabase start`." + : "Run `supabase start`, then retry.", + }); + } + let parsed: unknown; + try { + parsed = JSON.parse(stdout.trim() || "null"); + } catch { + return yield* new SchemaLocalStackNotRunningError({ + detail: "failed to parse local database container port map", + suggestion: "Run `supabase start`, then retry.", + }); + } + const port = publishedPostgresHostPort(parsed); + if (port === undefined) { + return yield* new SchemaLocalStackNotRunningError({ + detail: `local database container ${containerId} does not publish 5432/tcp`, + suggestion: "Run `supabase start`, then retry.", + }); + } + return Option.some(port); + }), + ); + +export const legacyDockerLocalDatabaseFallbackLayer = Layer.effect( + LocalDatabaseFallback, + Effect.gen(function* () { + const config = yield* LegacyCliConfig; + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const spawner = yield* ChildProcessSpawner.ChildProcessSpawner; + + return LocalDatabaseFallback.of({ + resolve: Effect.gen(function* () { + const toml = yield* legacyReadDbToml(fs, path, config.workdir, undefined, { + validate: false, + warnOnUnresolvedEnv: false, + }).pipe(Effect.orElseSucceed(() => undefined)); + const projectId = legacyResolveLocalProjectId( + Option.getOrUndefined(config.projectId), + toml === undefined ? undefined : Option.getOrUndefined(toml.projectId), + config.workdir, + ); + const published = yield* inspectPublishedPostgresPort( + spawner, + localDbContainerId(projectId), + ); + if (Option.isNone(published)) { + return Option.none(); + } + return Option.some({ + kind: "local", + identity: "local:default", + connectionString: localPostgresConnectionString( + published.value, + toml?.password ?? "postgres", + legacyGetHostname(), + ), + disposable: true, + durable: false, + connectionVerified: true, + } satisfies DatabaseTarget); + }), + }); + }), +); diff --git a/apps/cli/src/legacy/schema/legacy-linked-remote-connector.layer.ts b/apps/cli/src/legacy/schema/legacy-linked-remote-connector.layer.ts new file mode 100644 index 0000000000..24d043ddf3 --- /dev/null +++ b/apps/cli/src/legacy/schema/legacy-linked-remote-connector.layer.ts @@ -0,0 +1,52 @@ +import { Effect, Layer, Option } from "effect"; +import { LinkedRemoteConnector } from "../../shared/database/linked-remote-connector.service.ts"; +import { SchemaLinkedConnectionError } from "../../shared/schema/schema-errors.ts"; +import { LegacyDnsResolverFlag } from "../../shared/legacy/global-flags.ts"; +import { legacyBuildConnectionUrl } from "../shared/legacy-db-connection.sql-pg.layer.ts"; +import { LegacyDbConfigResolver } from "../shared/legacy-db-config.service.ts"; + +function toLinkedConnectionError(error: unknown): SchemaLinkedConnectionError { + const detail = + error !== null && + typeof error === "object" && + "message" in error && + typeof error.message === "string" && + error.message.length > 0 + ? error.message + : "Failed to connect to the linked project."; + const suggestion = + error !== null && + typeof error === "object" && + "suggestion" in error && + typeof error.suggestion === "string" && + error.suggestion.length > 0 + ? error.suggestion + : "Run `supabase link`, or pass --db-url with a connection string."; + return new SchemaLinkedConnectionError({ detail, suggestion }); +} + +export const legacyLinkedRemoteConnectorLayer = Layer.effect( + LinkedRemoteConnector, + Effect.gen(function* () { + const resolver = yield* LegacyDbConfigResolver; + const dnsFlag = yield* Effect.serviceOption(LegacyDnsResolverFlag); + const dnsResolver = Option.getOrUndefined(dnsFlag) ?? "native"; + + return LinkedRemoteConnector.of({ + connect: (projectRef) => + resolver + .resolve({ + dbUrl: Option.none(), + connType: "linked", + dnsResolver, + linkedProjectRef: Option.some(projectRef), + }) + .pipe( + Effect.map((resolved) => + legacyBuildConnectionUrl(resolved.conn, resolved.conn.host, resolved.conn.port), + ), + Effect.mapError(toLinkedConnectionError), + ), + }); + }), +); diff --git a/apps/cli/src/legacy/schema/legacy-migration-repository.layer.integration.test.ts b/apps/cli/src/legacy/schema/legacy-migration-repository.layer.integration.test.ts new file mode 100644 index 0000000000..6d56a3353b --- /dev/null +++ b/apps/cli/src/legacy/schema/legacy-migration-repository.layer.integration.test.ts @@ -0,0 +1,133 @@ +import { mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { describe, expect, it } from "@effect/vitest"; +import { BunServices } from "@effect/platform-bun"; +import { Cause, Effect, Exit, Layer } from "effect"; +import { SchemaMigrationNameError } from "../../shared/schema/schema-errors.ts"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import { schemaWorkspaceLayer } from "../../shared/schema/schema-workspace.layer.ts"; +import { MigrationRepository } from "../../shared/migrations/migration-repository.service.ts"; +import { formatFetchedMigrationSql } from "../../shared/migrations/pull-migrations.ts"; +import { legacyMigrationRepositoryLayer } from "./legacy-migration-repository.layer.ts"; + +function tempProject() { + const root = mkdtempSync(join(tmpdir(), "fetched-migrations-")); + const supabaseDir = join(root, "supabase"); + const projectHomeDir = join(root, ".supabase"); + return { root, supabaseDir, projectHomeDir }; +} + +describe("legacyMigrationRepositoryLayer writeFetched", () => { + it.live("writes, skips identical SQL, and side-files a mismatch", () => { + const project = tempProject(); + const out = mockOutput({ interactive: false }); + const workspace = schemaWorkspaceLayer({ + projectRoot: project.root, + supabaseDir: project.supabaseDir, + projectHomeDir: project.projectHomeDir, + }).pipe(Layer.provide(BunServices.layer)); + const layer = Layer.mergeAll( + out.layer, + workspace, + legacyMigrationRepositoryLayer.pipe( + Layer.provide(workspace), + Layer.provide(BunServices.layer), + Layer.provide(out.layer), + ), + ); + return Effect.gen(function* () { + const repository = yield* MigrationRepository; + const sql = formatFetchedMigrationSql(["create table t (id int)"]); + const written = yield* repository.writeFetched({ + version: "20260101000000", + name: "init", + sql, + }); + expect(written.outcome).toBe("written"); + expect( + readFileSync(join(project.supabaseDir, "migrations", "20260101000000_init.sql"), "utf8"), + ).toBe(sql); + + const skipped = yield* repository.writeFetched({ + version: "20260101000000", + name: "init", + sql, + }); + expect(skipped.outcome).toBe("skipped"); + + writeFileSync( + join(project.supabaseDir, "migrations", "20260101000000_init.sql"), + "select 1;\n", + ); + const conflict = yield* repository.writeFetched({ + version: "20260101000000", + name: "init", + sql, + }); + expect(conflict.outcome).toBe("conflict"); + if (conflict.outcome === "conflict") { + expect(conflict.remoteCopyDisplay).toBe( + ".supabase/remote-migrations/20260101000000_init.sql", + ); + expect(readFileSync(conflict.remoteCopyPath, "utf8")).toBe(sql); + expect( + readFileSync(join(project.supabaseDir, "migrations", "20260101000000_init.sql"), "utf8"), + ).toBe("select 1;\n"); + } + + const escape = yield* repository + .writeFetched({ version: "../oops", name: "x", sql }) + .pipe(Effect.exit); + expect(Exit.isFailure(escape)).toBe(true); + const failure = Exit.isFailure(escape) ? Cause.findErrorOption(escape.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaMigrationNameError); + } + }).pipe( + Effect.provide(layer), + Effect.ensuring(Effect.sync(() => rmSync(project.root, { recursive: true, force: true }))), + ); + }); + + it.live("skips a same-version file that was renamed locally", () => { + const project = tempProject(); + const out = mockOutput({ interactive: false }); + const workspace = schemaWorkspaceLayer({ + projectRoot: project.root, + supabaseDir: project.supabaseDir, + projectHomeDir: project.projectHomeDir, + }).pipe(Layer.provide(BunServices.layer)); + const layer = Layer.mergeAll( + out.layer, + workspace, + legacyMigrationRepositoryLayer.pipe( + Layer.provide(workspace), + Layer.provide(BunServices.layer), + Layer.provide(out.layer), + ), + ); + const sql = formatFetchedMigrationSql(["create table t (id int)"]); + mkdirSync(join(project.supabaseDir, "migrations"), { recursive: true }); + writeFileSync(join(project.supabaseDir, "migrations", "20260101000000_initial.sql"), sql); + return Effect.gen(function* () { + const repository = yield* MigrationRepository; + const skipped = yield* repository.writeFetched({ + version: "20260101000000", + name: "init", + sql, + }); + expect(skipped.outcome).toBe("skipped"); + if (skipped.outcome === "skipped") { + expect(skipped.file.fileName).toBe("20260101000000_initial.sql"); + } + expect(readdirSync(join(project.supabaseDir, "migrations"))).toEqual([ + "20260101000000_initial.sql", + ]); + }).pipe( + Effect.provide(layer), + Effect.ensuring(Effect.sync(() => rmSync(project.root, { recursive: true, force: true }))), + ); + }); +}); diff --git a/apps/cli/src/legacy/schema/legacy-migration-repository.layer.ts b/apps/cli/src/legacy/schema/legacy-migration-repository.layer.ts new file mode 100644 index 0000000000..694ccc9b8a --- /dev/null +++ b/apps/cli/src/legacy/schema/legacy-migration-repository.layer.ts @@ -0,0 +1,263 @@ +import { Clock, Effect, FileSystem, Layer, Path } from "effect"; +import { Output } from "../../shared/output/output.service.ts"; +import { + SchemaMigrationNameError, + SchemaWorkspaceIoError, +} from "../../shared/schema/schema-errors.ts"; +import { MIGRATION_NO_TRANSACTION_DIRECTIVE } from "../../shared/schema/schema-paths.ts"; +import { SchemaWorkspace } from "../../shared/schema/schema-workspace.service.ts"; +import type { MigrationFile } from "../../shared/migrations/migration-file.ts"; +import { + MigrationRepository, + type FetchedMigrationWrite, +} from "../../shared/migrations/migration-repository.service.ts"; +import { + legacyFormatMigrationTimestamp, + legacyGetMigrationPath, + legacyParseMigrationContent, +} from "../shared/legacy-migration-file.ts"; +import { + legacyListLocalMigrationPaths, + MIGRATE_FILE_PATTERN, +} from "../shared/legacy-migration-history.ts"; + +const NAME_PATTERN = /^[A-Za-z0-9_-]+$/u; +const MAX_VERSION_COLLISION_ATTEMPTS = 100; + +const ioError = (detail: string) => + new SchemaWorkspaceIoError({ + detail, + suggestion: "Check permissions on supabase/migrations and retry.", + }); + +export const legacyMigrationRepositoryLayer = Layer.effect( + MigrationRepository, + Effect.gen(function* () { + const workspace = yield* SchemaWorkspace; + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const output = yield* Output; + const workdir = path.dirname(path.dirname(workspace.migrationsDir)); + + const readLocal = Effect.gen(function* () { + const paths = yield* legacyListLocalMigrationPaths(fs, path, workspace.migrationsDir).pipe( + Effect.provideService(Output, output), + Effect.mapError((error) => ioError(error.message)), + ); + const files: Array = []; + for (const absolutePath of paths) { + const fileName = path.basename(absolutePath); + const parsed = MIGRATE_FILE_PATTERN.exec(fileName); + const version = parsed?.[1]; + const name = parsed?.[2]; + if (version === undefined || name === undefined) continue; + const content = yield* fs + .readFileString(absolutePath) + .pipe( + Effect.mapError((error) => ioError(`Failed to read ${fileName}: ${error.message}`)), + ); + files.push({ + version, + name, + fileName, + absolutePath, + content, + transactional: legacyParseMigrationContent(content).transactionMode === "transactional", + }); + } + return files.sort((left, right) => left.version.localeCompare(right.version)); + }); + + const assertName = (name: string) => { + if (!NAME_PATTERN.test(name)) { + return Effect.fail( + new SchemaMigrationNameError({ + detail: `Invalid migration name "${name}".`, + suggestion: "Use only letters, numbers, underscores, and hyphens.", + }), + ); + } + return Effect.void; + }; + + const assertInsideMigrations = (absolutePath: string, name: string) => { + if (!absolutePath.startsWith(workspace.migrationsDir + path.sep)) { + return Effect.fail( + new SchemaMigrationNameError({ + detail: `Migration name "${name}" escapes supabase/migrations.`, + suggestion: "Use a simple identifier without path separators.", + }), + ); + } + return Effect.void; + }; + + const historySegmentEscapes = (segment: string) => + /[/\\]/u.test(segment) || segment.split(/[/\\]/u).includes(".."); + + const remoteCopiesDir = path.join(path.dirname(workspace.journalPath), "remote-migrations"); + + return MigrationRepository.of({ + listLocal: readLocal, + writeFetched: (input) => + Effect.gen(function* () { + if (historySegmentEscapes(input.version) || historySegmentEscapes(input.name)) { + return yield* new SchemaMigrationNameError({ + detail: `Invalid version/name in history table: ${input.version}_${input.name}`, + suggestion: "Use a simple identifier without path separators.", + }); + } + // Same version, possibly renamed: skip/conflict against that file, never write a second one. + const local = yield* readLocal; + const existing = local.find((file) => file.version === input.version); + const fileName = existing?.fileName ?? `${input.version}_${input.name}.sql`; + const absolutePath = + existing?.absolutePath ?? path.join(workspace.migrationsDir, fileName); + yield* assertInsideMigrations(absolutePath, existing?.name ?? input.name); + yield* fs + .makeDirectory(workspace.migrationsDir, { recursive: true }) + .pipe(Effect.mapError((error) => ioError(error.message))); + const toFile = (content: string): MigrationFile => ({ + version: input.version, + name: existing?.name ?? input.name, + fileName, + absolutePath, + content, + transactional: legacyParseMigrationContent(content).transactionMode === "transactional", + }); + if (existing !== undefined) { + if (existing.content === input.sql) { + return { + outcome: "skipped", + file: toFile(existing.content), + } satisfies FetchedMigrationWrite; + } + const remoteCopyName = `${input.version}_${input.name}.sql`; + const remoteCopyPath = path.join(remoteCopiesDir, remoteCopyName); + yield* fs + .makeDirectory(remoteCopiesDir, { recursive: true }) + .pipe(Effect.mapError((error) => ioError(error.message))); + yield* fs + .writeFileString(remoteCopyPath, input.sql) + .pipe( + Effect.mapError((error) => + ioError(`Failed to write ${remoteCopyName} remote copy: ${error.message}`), + ), + ); + return { + outcome: "conflict", + file: toFile(existing.content), + remoteCopyPath, + remoteCopyDisplay: path.join(".supabase", "remote-migrations", remoteCopyName), + } satisfies FetchedMigrationWrite; + } + yield* fs + .writeFileString(absolutePath, input.sql) + .pipe( + Effect.mapError((error) => ioError(`Failed to write ${fileName}: ${error.message}`)), + ); + return { outcome: "written", file: toFile(input.sql) } satisfies FetchedMigrationWrite; + }), + createEmpty: (name, content = "") => + Effect.gen(function* () { + yield* assertName(name); + const version = legacyFormatMigrationTimestamp(yield* Clock.currentTimeMillis); + const absolutePath = legacyGetMigrationPath(path, workdir, version, name); + const fileName = path.basename(absolutePath); + yield* assertInsideMigrations(absolutePath, name); + yield* fs + .makeDirectory(workspace.migrationsDir, { recursive: true }) + .pipe(Effect.mapError((error) => ioError(error.message))); + yield* fs + .writeFileString(absolutePath, content) + .pipe(Effect.mapError((error) => ioError(error.message))); + return { + version, + name, + fileName, + absolutePath, + content, + transactional: true, + } satisfies MigrationFile; + }), + writeGenerated: (input) => + Effect.gen(function* () { + yield* assertName(input.name); + yield* fs + .makeDirectory(workspace.migrationsDir, { recursive: true }) + .pipe(Effect.mapError((error) => ioError(error.message))); + const existing = yield* readLocal; + const usedVersions = new Set(existing.map((file) => file.version)); + const unitName = (suffix: string | null) => + suffix !== null && suffix !== "" ? `${input.name}_${suffix}` : input.name; + + const build = (baseMillis: number) => + input.files.map((file, index) => { + const version = legacyFormatMigrationTimestamp(baseMillis + index * 1000); + const name = unitName(file.suffix); + const absolutePath = legacyGetMigrationPath(path, workdir, version, name); + return { + version, + name, + fileName: path.basename(absolutePath), + absolutePath, + body: file.transactional + ? file.sql + : `${MIGRATION_NO_TRANSACTION_DIRECTIVE}\n${file.sql}`, + transactional: file.transactional, + }; + }); + + let planned = build(input.baseMillis); + for (let attempt = 0; attempt < MAX_VERSION_COLLISION_ATTEMPTS; attempt++) { + if (!planned.some((file) => usedVersions.has(file.version))) break; + planned = build(input.baseMillis + (attempt + 1) * 1000); + } + if (planned.some((file) => usedVersions.has(file.version))) { + return yield* new SchemaMigrationNameError({ + detail: "Could not allocate unique migration versions.", + suggestion: "Retry schema generate in a moment.", + }); + } + + const written: Array = []; + for (const file of planned) { + yield* assertInsideMigrations(file.absolutePath, file.name); + yield* fs.writeFileString(file.absolutePath, file.body).pipe( + Effect.mapError((error) => + ioError(`Failed to write ${file.fileName}: ${error.message}`), + ), + Effect.tapError(() => + Effect.forEach(written, (created) => + fs.remove(created.absolutePath).pipe(Effect.ignore), + ), + ), + ); + written.push({ + version: file.version, + name: file.name, + fileName: file.fileName, + absolutePath: file.absolutePath, + content: file.body, + transactional: file.transactional, + }); + } + return written; + }), + remove: (files) => + Effect.gen(function* () { + for (const file of files) { + yield* fs + .remove(file.absolutePath) + .pipe( + Effect.catchTag("PlatformError", (error) => + error.reason._tag === "NotFound" + ? Effect.void + : Effect.fail(ioError(`Failed to remove ${file.fileName}: ${error.message}`)), + ), + ); + } + }), + }); + }), +); diff --git a/apps/cli/src/legacy/schema/legacy-migration-runner.layer.integration.test.ts b/apps/cli/src/legacy/schema/legacy-migration-runner.layer.integration.test.ts new file mode 100644 index 0000000000..cddd622e97 --- /dev/null +++ b/apps/cli/src/legacy/schema/legacy-migration-runner.layer.integration.test.ts @@ -0,0 +1,145 @@ +import { describe, expect, it } from "@effect/vitest"; +import { BunServices } from "@effect/platform-bun"; +import { Effect, Exit } from "effect"; +import type { Pool, PoolClient } from "pg"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import { MigrationRunner } from "../../shared/migrations/migration-runner.service.ts"; +import { + SchemaEngineError, + SchemaMigrationsPrivilegeError, +} from "../../shared/schema/schema-errors.ts"; +import { legacyMigrationRunnerLayer } from "./legacy-migration-runner.layer.ts"; + +const local = { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: "select 1;", + transactional: true, +}; + +function queryFailPool(error: Error): Pool { + const client = { + query: async () => { + throw error; + }, + release: () => undefined, + }; + return { + connect: async () => client as unknown as PoolClient, + options: { connectionString: "postgresql://cli_login_abc.ref:pw@127.0.0.1:1/postgres" }, + } as Pool; +} + +function historyPool(versions: ReadonlyArray): Pool { + const client = { + query: async (sql: string) => { + if (sql.includes("statements")) { + return { + rows: versions.map((version) => ({ + version, + name: "init", + statements: ["select 1"], + })), + }; + } + if (sql.includes("SELECT version")) { + return { rows: versions.map((version) => ({ version, name: "" })) }; + } + if (sql.includes("SHOW server_version")) { + return { rows: [{ server_version: "17.6" }] }; + } + return { rows: [] }; + }, + release: () => undefined, + }; + return { + connect: async () => client as unknown as PoolClient, + options: { connectionString: "postgresql://postgres:postgres@127.0.0.1:1/postgres" }, + } as Pool; +} + +describe("legacyMigrationRunnerLayer", () => { + it.live("is a no-op when remote is strictly ahead and nothing is pending", () => { + const out = mockOutput({ interactive: false }); + return Effect.gen(function* () { + const runner = yield* MigrationRunner; + const result = yield* runner.applyPending(historyPool(["19990101000000", local.version]), [ + local, + ]); + expect(result.applied).toEqual([]); + expect(result.skipped).toEqual([local.version]); + }).pipe( + Effect.provide(legacyMigrationRunnerLayer), + Effect.provide(out.layer), + Effect.provide(BunServices.layer), + ); + }); + + it.live("maps 42501 on supabase_migrations to SchemaMigrationsPrivilegeError", () => { + const out = mockOutput({ interactive: false }); + const denied = Object.assign(new Error("permission denied for schema supabase_migrations"), { + code: "42501", + }); + return Effect.gen(function* () { + const runner = yield* MigrationRunner; + const error = yield* runner.listRemote(queryFailPool(denied)).pipe(Effect.flip); + expect(error).toBeInstanceOf(SchemaMigrationsPrivilegeError); + expect(error.message).toContain("permission denied for schema supabase_migrations"); + expect(error.suggestion).toBe("Set SUPABASE_DB_PASSWORD for the postgres role, then retry."); + expect(error.suggestion).not.toContain("--password"); + }).pipe( + Effect.provide(legacyMigrationRunnerLayer), + Effect.provide(out.layer), + Effect.provide(BunServices.layer), + ); + }); + + it.live("keeps unrelated privilege failures as SchemaEngineError", () => { + const out = mockOutput({ interactive: false }); + const denied = Object.assign(new Error('permission denied to set role "postgres"'), { + code: "42501", + }); + return Effect.gen(function* () { + const runner = yield* MigrationRunner; + const error = yield* runner.listRemote(queryFailPool(denied)).pipe(Effect.flip); + expect(error).toBeInstanceOf(SchemaEngineError); + expect(error).not.toBeInstanceOf(SchemaMigrationsPrivilegeError); + }).pipe( + Effect.provide(legacyMigrationRunnerLayer), + Effect.provide(out.layer), + Effect.provide(BunServices.layer), + ); + }); + + it.live("lists remote statements for fetch-pull", () => { + const out = mockOutput({ interactive: false }); + return Effect.gen(function* () { + const runner = yield* MigrationRunner; + const rows = yield* runner.listRemoteStatements(historyPool(["20260101000000"])); + expect(rows).toEqual([{ version: "20260101000000", name: "init", statements: ["select 1"] }]); + expect(yield* runner.showServerVersion(historyPool([]))).toBe("17.6"); + }).pipe( + Effect.provide(legacyMigrationRunnerLayer), + Effect.provide(out.layer), + Effect.provide(BunServices.layer), + ); + }); + + it.live("conflicts when remote-only versions and pending files both exist", () => { + const out = mockOutput({ interactive: false }); + const pending = { ...local, version: "20260101000001", fileName: "20260101000001_next.sql" }; + return Effect.gen(function* () { + const runner = yield* MigrationRunner; + const exit = yield* runner + .applyPending(historyPool(["19990101000000"]), [pending]) + .pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + }).pipe( + Effect.provide(legacyMigrationRunnerLayer), + Effect.provide(out.layer), + Effect.provide(BunServices.layer), + ); + }); +}); diff --git a/apps/cli/src/legacy/schema/legacy-migration-runner.layer.ts b/apps/cli/src/legacy/schema/legacy-migration-runner.layer.ts new file mode 100644 index 0000000000..72fea895da --- /dev/null +++ b/apps/cli/src/legacy/schema/legacy-migration-runner.layer.ts @@ -0,0 +1,333 @@ +import { Effect, FileSystem, Layer, Path } from "effect"; +import type { Pool, PoolClient } from "pg"; +import { Output } from "../../shared/output/output.service.ts"; +import { + SchemaEngineError, + SchemaHistoryConflictError, + SchemaMigrationsPrivilegeError, +} from "../../shared/schema/schema-errors.ts"; +import { formatHistoryConflict } from "../../shared/migrations/migration-repair-suggest.ts"; +import { + MigrationRunner, + type MigrationApplyResult, + type MigrationHistoryRow, + type MigrationHistoryStatementsRow, +} from "../../shared/migrations/migration-runner.service.ts"; +import { PLATFORM_SEARCH_PATH_SQL } from "../../shared/database/database-pool.ts"; +import { LegacyDbConnectError, LegacyDbExecError } from "../shared/legacy-db-connection.errors.ts"; +import { LegacyMigrationsReadError } from "../shared/legacy-migration.errors.ts"; +import type { LegacyDbSession } from "../shared/legacy-db-connection.service.ts"; +import { legacyApplyMigrations } from "../shared/legacy-migration-apply.ts"; +import { + INSERT_MIGRATION_VERSION, + legacyCreateMigrationTable, + legacyListRemoteMigrations, +} from "../shared/legacy-migration-history.ts"; + +const engineError = (detail: string) => + new SchemaEngineError({ + detail, + suggestion: "Check the database connection and migration SQL, then retry.", + }); + +const MIGRATIONS_PRIVILEGE_DENIED = /permission denied for schema supabase_migrations/iu; + +const privilegeError = (detail: string) => + new SchemaMigrationsPrivilegeError({ + detail, + suggestion: "Set SUPABASE_DB_PASSWORD for the postgres role, then retry.", + }); + +const isMigrationsPrivilegeDenied = (message: string, code?: string): boolean => + MIGRATIONS_PRIVILEGE_DENIED.test(message) && (code === undefined || code === "42501"); + +const postgresErrorCode = (cause: unknown): string | undefined => { + if (typeof cause !== "object" || cause === null || !("code" in cause)) return undefined; + const { code } = cause; + return typeof code === "string" ? code : undefined; +}; + +const toRunnerErrorFromCause = (cause: unknown) => { + const message = cause instanceof Error ? cause.message : String(cause); + const code = postgresErrorCode(cause); + if (isMigrationsPrivilegeDenied(message, code)) return privilegeError(message); + return engineError(message); +}; + +type RunnerConnectError = + | SchemaEngineError + | SchemaMigrationsPrivilegeError + | LegacyDbConnectError + | LegacyDbExecError + | LegacyMigrationsReadError; + +const mapConnectError = (error: RunnerConnectError) => { + if (error instanceof SchemaMigrationsPrivilegeError) return error; + const code = error instanceof LegacyDbExecError ? error.code : undefined; + if (isMigrationsPrivilegeDenied(error.message, code)) return privilegeError(error.message); + return error instanceof SchemaEngineError ? error : engineError(error.message); +}; + +const toExecError = (cause: unknown, statementIndex?: number) => + new LegacyDbExecError({ + message: cause instanceof Error ? cause.message : String(cause), + ...(postgresErrorCode(cause) !== undefined ? { code: postgresErrorCode(cause) } : {}), + ...(statementIndex !== undefined ? { statementIndex } : {}), + }); + +const sessionFromClient = (client: Pick): LegacyDbSession => ({ + restoreRoleSql: "SET SESSION ROLE postgres", + restoreSearchPathSql: PLATFORM_SEARCH_PATH_SQL, + exec: (sql) => + Effect.tryPromise({ + try: async () => { + await client.query(sql); + }, + catch: (cause) => toExecError(cause), + }), + execBatch: (statements) => + Effect.gen(function* () { + const run = (sql: string, params?: ReadonlyArray, statementIndex?: number) => + Effect.tryPromise({ + try: async () => { + if (params === undefined) { + await client.query(sql); + return; + } + await client.query(sql, [...params]); + }, + catch: (cause) => toExecError(cause, statementIndex), + }); + yield* run("BEGIN"); + yield* Effect.gen(function* () { + for (const [index, statement] of statements.entries()) { + yield* run(statement.sql, statement.params, index); + } + yield* run("COMMIT"); + }).pipe(Effect.tapError(() => run("ROLLBACK").pipe(Effect.ignore))); + }), + query: (sql, params) => + Effect.tryPromise({ + try: async () => { + const result = + params === undefined + ? await client.query>(sql) + : await client.query>(sql, [...params]); + return result.rows; + }, + catch: (cause) => toExecError(cause), + }), + extensionExists: () => Effect.die("legacy migration runner does not query extensions"), + copyToCsv: () => Effect.die("legacy migration runner does not copy CSV"), + queryRaw: () => Effect.die("legacy migration runner does not query raw rows"), +}); + +const withSession = ( + pool: Pool, + body: (session: LegacyDbSession) => Effect.Effect, +): Effect.Effect => + Effect.scoped( + Effect.gen(function* () { + const client = yield* Effect.acquireRelease( + Effect.tryPromise({ + try: () => pool.connect(), + catch: (cause) => toRunnerErrorFromCause(cause), + }), + (held) => Effect.sync(() => held.release()), + ); + return yield* body(sessionFromClient(client)); + }), + ); + +const LIST_HISTORY = + "SELECT version, coalesce(name, '') AS name FROM supabase_migrations.schema_migrations ORDER BY version"; +const LIST_HISTORY_VERSION_ONLY = + "SELECT version FROM supabase_migrations.schema_migrations ORDER BY version"; +const LIST_HISTORY_STATEMENTS = + "SELECT version, coalesce(name, '') AS name, statements FROM supabase_migrations.schema_migrations ORDER BY version"; + +const toStatements = (value: unknown): ReadonlyArray => + Array.isArray(value) ? value.map((entry) => String(entry)) : []; + +const isMissingMigrationHistory = (error: LegacyDbExecError): boolean => { + if (error.code === "3F000" || error.code === "42P01") return true; + return ( + /relation .* does not exist/iu.test(error.message) && + !/column .* does not exist/iu.test(error.message) + ); +}; + +const isMissingNameColumn = (error: LegacyDbExecError): boolean => + /column ["']?name["']? does not exist/iu.test(error.message); + +const isMissingStatementsColumn = (error: LegacyDbExecError): boolean => + /column ["']?statements["']? does not exist/iu.test(error.message); + +const listHistory = ( + session: LegacyDbSession, +): Effect.Effect< + ReadonlyArray, + SchemaEngineError | SchemaMigrationsPrivilegeError +> => + session.query(LIST_HISTORY).pipe( + Effect.map((rows) => + rows.map((row) => ({ + version: String(row["version"] ?? ""), + name: String(row["name"] ?? ""), + })), + ), + Effect.catch((error: LegacyDbExecError) => { + if (isMissingMigrationHistory(error)) return Effect.succeed([]); + if (isMissingNameColumn(error)) { + return session + .query(LIST_HISTORY_VERSION_ONLY) + .pipe( + Effect.map((rows) => + rows.map((row) => ({ version: String(row["version"] ?? ""), name: "" })), + ), + ); + } + return Effect.fail(error); + }), + Effect.mapError(mapConnectError), + ); + +const listHistoryStatements = ( + session: LegacyDbSession, +): Effect.Effect< + ReadonlyArray, + SchemaEngineError | SchemaMigrationsPrivilegeError +> => + session.query(LIST_HISTORY_STATEMENTS).pipe( + Effect.map((rows) => + rows.map((row) => ({ + version: String(row["version"] ?? ""), + name: String(row["name"] ?? ""), + statements: toStatements(row["statements"]), + })), + ), + Effect.catch((error: LegacyDbExecError) => { + if (isMissingMigrationHistory(error)) return Effect.succeed([]); + if (isMissingStatementsColumn(error) || isMissingNameColumn(error)) { + return session.query(LIST_HISTORY).pipe( + Effect.map((rows) => + rows.map((row) => ({ + version: String(row["version"] ?? ""), + name: String(row["name"] ?? ""), + statements: [], + })), + ), + Effect.catch((inner: LegacyDbExecError) => { + if (isMissingMigrationHistory(inner)) return Effect.succeed([]); + if (isMissingNameColumn(inner)) { + return session.query(LIST_HISTORY_VERSION_ONLY).pipe( + Effect.map((rows) => + rows.map((row) => ({ + version: String(row["version"] ?? ""), + name: "", + statements: [], + })), + ), + ); + } + return Effect.fail(inner); + }), + ); + } + return Effect.fail(error); + }), + Effect.mapError(mapConnectError), + ); + +export const legacyMigrationRunnerLayer = Layer.effect( + MigrationRunner, + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + + const listRemote = (pool: Pool) => withSession(pool, listHistory); + const listRemoteStatements = (pool: Pool) => withSession(pool, listHistoryStatements); + const showServerVersion = (pool: Pool) => + withSession(pool, (session) => + session.query("SHOW server_version").pipe( + Effect.map((rows) => { + const raw = rows[0]?.["server_version"]; + return typeof raw === "string" && raw.length > 0 ? raw : undefined; + }), + Effect.catch(() => Effect.succeed(undefined)), + ), + ).pipe(Effect.catch(() => Effect.succeed(undefined))); + const listInstalledExtensions = (pool: Pool) => + withSession(pool, (session) => + session.query("SELECT extname FROM pg_extension").pipe( + Effect.map((rows) => + rows + .map((row) => String(row["extname"] ?? "").toLowerCase()) + .filter((name) => name.length > 0), + ), + Effect.mapError(mapConnectError), + ), + ); + + return MigrationRunner.of({ + listRemote, + listRemoteStatements, + showServerVersion, + listInstalledExtensions, + applyPending: (pool, local) => + withSession(pool, (session) => + Effect.gen(function* () { + const remote = yield* listHistory(session); + const remoteVersions = new Set(remote.map((row) => row.version)); + const pendingFiles = local.filter((file) => !remoteVersions.has(file.version)); + const remoteOnly = remote + .filter((row) => !local.some((file) => file.version === row.version)) + .map((row) => row.version); + if (remoteOnly.length > 0 && pendingFiles.length > 0) { + return yield* new SchemaHistoryConflictError( + formatHistoryConflict({ + remoteOnly, + pending: pendingFiles.map((file) => file.version), + }), + ); + } + const output = yield* Output; + yield* legacyApplyMigrations( + session, + fs, + path, + pendingFiles.map((file) => file.absolutePath), + (message) => + engineError( + message.startsWith("Failed applying") + ? message + : `Failed applying migration: ${message}`, + ), + ).pipe(Effect.provideService(Output, output), Effect.mapError(mapConnectError)); + return { + applied: pendingFiles.map((file) => file.version), + skipped: local + .filter((file) => remoteVersions.has(file.version)) + .map((file) => file.version), + } satisfies MigrationApplyResult; + }), + ), + markApplied: (pool, files) => + withSession(pool, (session) => + Effect.gen(function* () { + yield* legacyCreateMigrationTable(session).pipe(Effect.mapError(mapConnectError)); + const remote = yield* legacyListRemoteMigrations(session).pipe( + Effect.mapError(mapConnectError), + ); + const present = new Set(remote); + for (const file of files) { + if (present.has(file.version)) continue; + yield* session + .query(INSERT_MIGRATION_VERSION, [file.version, file.name, [file.content]]) + .pipe(Effect.mapError(mapConnectError)); + } + }), + ), + }); + }), +); diff --git a/apps/cli/src/legacy/schema/legacy-schema-database-target.layer.ts b/apps/cli/src/legacy/schema/legacy-schema-database-target.layer.ts new file mode 100644 index 0000000000..fbdcbb0730 --- /dev/null +++ b/apps/cli/src/legacy/schema/legacy-schema-database-target.layer.ts @@ -0,0 +1,104 @@ +import { Effect, FileSystem, Layer, Option, Path } from "effect"; +import { DatabaseTargetResolver } from "../../shared/database/database-target.service.ts"; +import { envDatabaseUrl, type DatabaseTarget } from "../../shared/database/database-target.ts"; +import { LinkedRemoteConnector } from "../../shared/database/linked-remote-connector.service.ts"; +import { LocalDatabaseFallback } from "../../shared/database/local-database-fallback.service.ts"; +import { + SchemaLinkedConnectionError, + SchemaLocalStackNotRunningError, +} from "../../shared/schema/schema-errors.ts"; +import { LegacyCliConfig } from "../config/legacy-cli-config.service.ts"; +import { PROJECT_REF_PATTERN } from "../config/legacy-project-ref.service.ts"; +import { legacyReadProjectRefFile } from "../shared/legacy-temp-paths.ts"; + +export const legacySchemaDatabaseTargetLayer = Layer.effect( + DatabaseTargetResolver, + Effect.gen(function* () { + const localDb = yield* LocalDatabaseFallback; + const linkedRemote = yield* LinkedRemoteConnector; + const config = yield* LegacyCliConfig; + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + + const resolveLocal = Effect.gen(function* () { + const owned = yield* localDb.resolve; + if (Option.isNone(owned)) { + return yield* new SchemaLocalStackNotRunningError({ + detail: "No local Supabase database container is running for this project.", + suggestion: "Run `supabase start` or `supabase db start`, then retry.", + }); + } + return owned.value; + }); + + const resolveLinkedRef = Effect.gen(function* () { + if (Option.isSome(config.projectId) && PROJECT_REF_PATTERN.test(config.projectId.value)) { + return config.projectId.value; + } + const fileRef = yield* legacyReadProjectRefFile(fs, path, config.workdir).pipe( + Effect.mapError( + (error) => + new SchemaLinkedConnectionError({ + detail: error.message, + suggestion: "Fix or remove supabase/.temp/project-ref, then retry.", + }), + ), + ); + if (Option.isNone(fileRef)) { + return yield* new SchemaLinkedConnectionError({ + detail: "This project is not linked to a Supabase project.", + suggestion: "Run `supabase link`, or pass --from / --against with a connection string.", + }); + } + if (!PROJECT_REF_PATTERN.test(fileRef.value)) { + return yield* new SchemaLinkedConnectionError({ + detail: "supabase/.temp/project-ref is not a valid project ref.", + suggestion: "Run `supabase link` again, or remove the invalid project-ref file.", + }); + } + return fileRef.value; + }); + + const resolveLinked = Effect.gen(function* () { + const url = envDatabaseUrl(); + if (url !== undefined) { + return { + kind: "url", + identity: "connection-string", + connectionString: url, + disposable: false, + durable: true, + connectionVerified: false, + connectionSource: "env", + } satisfies DatabaseTarget; + } + const ref = yield* resolveLinkedRef; + const connectionString = yield* linkedRemote.connect(ref); + return { + kind: "linked", + identity: ref, + connectionString, + disposable: false, + durable: true, + connectionVerified: true, + projectRef: ref, + } satisfies DatabaseTarget; + }); + + return DatabaseTargetResolver.of({ + resolve: (selector) => { + if (selector.kind === "local") return resolveLocal; + if (selector.kind === "linked") return resolveLinked; + return Effect.succeed({ + kind: "url", + identity: "connection-string", + connectionString: selector.url, + disposable: false, + durable: true, + connectionVerified: false, + connectionSource: "flag", + } satisfies DatabaseTarget); + }, + }); + }), +); diff --git a/apps/cli/src/legacy/schema/legacy-schema-runtime.layer.ts b/apps/cli/src/legacy/schema/legacy-schema-runtime.layer.ts new file mode 100644 index 0000000000..c590455b26 --- /dev/null +++ b/apps/cli/src/legacy/schema/legacy-schema-runtime.layer.ts @@ -0,0 +1,79 @@ +import { Effect, Layer, Path } from "effect"; +import { commandRuntimeLayer } from "../../shared/runtime/command-runtime.layer.ts"; +import { pgDeltaSchemaEngineLayer } from "../../shared/schema/pg-delta-engine.layer.ts"; +import { schemaStateLayer } from "../../shared/schema/schema-state.layer.ts"; +import { schemaWorkspaceLayer } from "../../shared/schema/schema-workspace.layer.ts"; +import { legacyCliConfigLayer } from "../config/legacy-cli-config.layer.ts"; +import { LegacyCliConfig } from "../config/legacy-cli-config.service.ts"; +import { legacyHttpClientLayer } from "../auth/legacy-http-debug.layer.ts"; +import { legacyDbConfigLayer } from "../shared/legacy-db-config.layer.ts"; +import { legacyDbConnectionLayer } from "../shared/legacy-db-connection.layer.ts"; +import { legacyDebugLoggerLayer } from "../shared/legacy-debug-logger.layer.ts"; +import { legacyDockerRunLayer } from "../shared/legacy-docker-run.layer.ts"; +import { legacyIdentityStitchLayer } from "../shared/legacy-identity-stitch.ts"; +import { legacyPgDeltaNextShadowLayer } from "../commands/db/shared/legacy-pgdelta-next-shadow.layer.ts"; +import { legacyDockerIsolatedShadowLayer } from "./legacy-docker-isolated-shadow.layer.ts"; +import { legacyDockerLocalDatabaseFallbackLayer } from "./legacy-docker-local-database.layer.ts"; +import { legacyLinkedRemoteConnectorLayer } from "./legacy-linked-remote-connector.layer.ts"; +import { legacyMigrationRepositoryLayer } from "./legacy-migration-repository.layer.ts"; +import { legacyMigrationRunnerLayer } from "./legacy-migration-runner.layer.ts"; +import { legacySchemaDatabaseTargetLayer } from "./legacy-schema-database-target.layer.ts"; + +const legacyCliConfig = legacyCliConfigLayer.pipe(Layer.provide(legacyDebugLoggerLayer)); + +const legacySchemaDbConfig = legacyDbConfigLayer.pipe( + Layer.provide(legacyCliConfig), + Layer.provide(legacyDbConnectionLayer), + Layer.provide(legacyDebugLoggerLayer), + Layer.provide(legacyIdentityStitchLayer), +); + +const httpClient = legacyHttpClientLayer.pipe(Layer.provide(legacyDebugLoggerLayer)); + +const nextShadow = legacyPgDeltaNextShadowLayer.pipe( + Layer.provide(legacyDockerRunLayer), + Layer.provide(legacyDbConnectionLayer), + Layer.provide(httpClient), +); + +const dockerShadows = legacyDockerIsolatedShadowLayer.pipe( + Layer.provide(nextShadow), + Layer.provide(legacyCliConfig), +); + +const schemaEngine = pgDeltaSchemaEngineLayer.pipe(Layer.provide(dockerShadows)); + +export const legacySchemaRuntimeLayer = (commandPath: ReadonlyArray) => + Layer.unwrap( + Effect.gen(function* () { + const config = yield* LegacyCliConfig; + const path = yield* Path.Path; + const workspace = schemaWorkspaceLayer({ + projectRoot: config.workdir, + supabaseDir: path.join(config.workdir, "supabase"), + projectHomeDir: path.join(config.workdir, ".supabase"), + }); + const localDatabase = legacyDockerLocalDatabaseFallbackLayer.pipe( + Layer.provide(Layer.succeed(LegacyCliConfig, config)), + ); + const linkedRemote = legacyLinkedRemoteConnectorLayer.pipe( + Layer.provide(legacySchemaDbConfig), + ); + const targets = legacySchemaDatabaseTargetLayer.pipe( + Layer.provide(localDatabase), + Layer.provide(linkedRemote), + Layer.provide(Layer.succeed(LegacyCliConfig, config)), + ); + return Layer.mergeAll( + workspace, + schemaStateLayer.pipe(Layer.provide(workspace)), + legacyMigrationRepositoryLayer.pipe(Layer.provide(workspace)), + legacyMigrationRunnerLayer, + schemaEngine, + targets, + linkedRemote, + localDatabase, + commandRuntimeLayer(commandPath), + ); + }), + ).pipe(Layer.provide(legacyCliConfig), Layer.provide(legacyDebugLoggerLayer)); diff --git a/apps/cli/src/legacy/shared/db-bootstrap/db-setup.ts b/apps/cli/src/legacy/shared/db-bootstrap/db-setup.ts index 85ba9fa136..b722c93505 100644 --- a/apps/cli/src/legacy/shared/db-bootstrap/db-setup.ts +++ b/apps/cli/src/legacy/shared/db-bootstrap/db-setup.ts @@ -120,9 +120,10 @@ import type { ProjectConfig } from "@supabase/config"; import { Data, Effect, type FileSystem, Option, type Path, Schedule, type Scope } from "effect"; import type { ChildProcessSpawner } from "effect/unstable/process/ChildProcessSpawner"; -import type { LocalServiceVersionOverrides } from "../../../shared/services/services.shared.ts"; +import { REVOKE_API_PRIVILEGES_SQL as LEGACY_START_REVOKE_API_PRIVILEGES_SQL } from "../../../shared/migrations/privilege-offer.ts"; import { Output } from "../../../shared/output/output.service.ts"; import { RuntimeInfo } from "../../../shared/runtime/runtime-info.service.ts"; +import type { LocalServiceVersionOverrides } from "../../../shared/services/services.shared.ts"; import { actionability, type CliErrorActionabilityDeclaration, @@ -180,21 +181,8 @@ import { type Spawner = ChildProcessSpawner["Service"]; -/** - * Go's inline `RevokeDefaultDataApiPrivilegesSql` constant (`start.go:405-412`) — - * NOT a `//go:embed` file (unlike the three large SQL templates), so transcribed - * directly here rather than as a sibling `templates/*.sql.ts` module. Exported for the - * shadow baseline cache's embedded-SQL digest (`shadow-cache.ts`), which must re-key - * whenever this text changes across CLI releases. - */ -export const LEGACY_START_REVOKE_API_PRIVILEGES_SQL = ` -alter default privileges for role postgres in schema public - revoke select, insert, update, delete on tables from anon, authenticated, service_role; -alter default privileges for role postgres in schema public - revoke usage, select on sequences from anon, authenticated, service_role; -alter default privileges for role postgres in schema public - revoke execute on functions from anon, authenticated, service_role; -`; +/** Re-export of the shared revoke body so shadow-cache keeps hashing this name. */ +export { LEGACY_START_REVOKE_API_PRIVILEGES_SQL }; /** * Exported for the shadow baseline cache's embedded-SQL digest (`shadow-cache.ts`), same as @@ -212,11 +200,10 @@ const LEGACY_START_REMOVE_DATABASE_WEBHOOKS_SQL = "drop extension if exists pg_n /** * A SQL exec (schema/globals/API-privileges) or one-shot service-migration Docker - * job failed, or the scratch temp directory/file could not be created. The Docker - * job branch's message mirrors Go's `DockerRunOnceWithStream` failure shape - * (`errors.Errorf("error running container: %w", err)`, `apps/cli-go/internal/ - * utils/docker.go:469-487,559-591` — Go discards the container's own stdout/stderr - * outside `--debug`, so only the exit code is meaningful here too). + * job failed, or the scratch temp directory/file could not be created. Container + * exits keep the `error running container: exit N` prefix used by `db dump` and + * the edge-runtime helper; when stderr was captured, the last meaningful line is + * appended (`exit N:\n${line}`) so a migrate miss is visible without `--debug`. */ export class LegacyDbSetupError extends Data.TaggedError("LegacyDbSetupError")<{ readonly message: string; @@ -465,10 +452,9 @@ export interface LegacySetupDatabaseInput { */ readonly projectEnvValues: Readonly> | undefined; /** - * `--debug` — threaded to each PG15+ one-shot migrate job (see - * {@link legacyRunStartMigrateJob}'s own doc comment) so a failed Realtime/Storage/Auth - * migration job's own stderr is visible, matching Go's `initSchema15` passing - * `utils.GetDebugLogger()` as the job's stderr writer (`start.go:349-353`). + * `--debug` — tees each PG15+ one-shot migrate job's full stderr (see + * {@link legacyRunStartMigrateJob}). The last meaningful line is already on + * {@link LegacyDbSetupError} without this flag. */ readonly debug: boolean; /** `toml.baseline.apiAutoExposeNewTables` — Go's `api.auto_expose_new_tables` tri-state, threaded straight into {@link legacyApplyApiPrivileges}. */ @@ -691,17 +677,31 @@ const legacyStartInitSchemaPre15 = Effect.fnUntraced(function* ( yield* legacyInitSchema14(session, fs, path, tmpDir, majorVersion); }); +/** + * Last non-empty container stderr line that is not a V8/Node stack frame or + * `Node.js v…` footer — the migrate miss (`StorageBackendError: …`). + */ +function legacyLastMeaningfulStderrLine(stderr: string): string | undefined { + const lines = stderr.split(/\r?\n/); + for (let i = lines.length - 1; i >= 0; i--) { + const line = lines[i]!.trim(); + if (line.length === 0 || line.startsWith("at ") || /^Node\.js v\d/u.test(line)) continue; + return line; + } + return undefined; +} + /** * Runs one PG15+ one-shot service-migration job to completion (Go's * `utils.DockerRunJob` = `DockerRunOnceWithStream`, `docker.go:457-459,469-487`): * foreground, same Docker network as `db`, no entrypoint override (Go's plain * `Cmd` field), stdout always discarded (Go's own `stdout` writer here is always - * `io.Discard`, `start.go:352`) and stderr teed to the parent process's own stderr ONLY - * under `--debug` — Go passes `logger := utils.GetDebugLogger()` as the job's stderr - * writer (`os.Stderr` under `--debug`, else `io.Discard`, `logger.go:10-15`) — so a - * fresh-volume Realtime/Storage/Auth migration job's own diagnostics are visible when - * `db start --debug`/`supabase start --debug` is used, not just its exit code. A - * non-zero exit fails with the same shape as Go's `error running container: `. + * `io.Discard`, `start.go:352`). Stderr is always captured for the failure + * message (same `result.stderr` as `db dump` / the edge-runtime helper); it is + * teed to the parent process only under `--debug` — Go's `utils.GetDebugLogger()` + * (`os.Stderr` under `--debug`, else `io.Discard`, `logger.go:10-15`). A non-zero + * exit fails with `error running container: exit N` plus the last meaningful + * stderr line when one exists. * * Resolves `opts.image` itself, individually, right here — via `legacyEnsureImagesCached` * (NOT `LegacyDockerRun.runStream`'s own ambient-only resolver, which never sees @@ -781,9 +781,13 @@ const legacyRunStartMigrateJob = Effect.fnUntraced(function* ( ), ); if (result.exitCode !== 0) { + const snippet = legacyLastMeaningfulStderrLine(result.stderr); return yield* Effect.fail( new LegacyDbSetupError({ - message: `error running container: exit ${result.exitCode}`, + message: + snippet === undefined + ? `error running container: exit ${result.exitCode}` + : `error running container: exit ${result.exitCode}:\n${snippet}`, reason: "database", }), ); diff --git a/apps/cli/src/legacy/shared/db-bootstrap/db-setup.unit.test.ts b/apps/cli/src/legacy/shared/db-bootstrap/db-setup.unit.test.ts index ff22342f07..71daba5753 100644 --- a/apps/cli/src/legacy/shared/db-bootstrap/db-setup.unit.test.ts +++ b/apps/cli/src/legacy/shared/db-bootstrap/db-setup.unit.test.ts @@ -83,7 +83,7 @@ function fakeSession() { return { session, calls }; } -function mockDockerRun(opts: { exitCode?: number } = {}) { +function mockDockerRun(opts: { exitCode?: number; stderr?: string } = {}) { const runs: Array = []; const captureOptsCalls: Array<{ readonly teeStderr?: boolean } | undefined> = []; const layer = Layer.succeed(LegacyDockerRun, { @@ -94,7 +94,7 @@ function mockDockerRun(opts: { exitCode?: number } = {}) { return Effect.succeed({ exitCode: opts.exitCode ?? 0, stdout: new Uint8Array(), - stderr: "", + stderr: opts.stderr ?? "", }); }, // `legacyRunStartMigrateJob` (`db-setup.ts`) discards stdout via `runStream` (not @@ -104,7 +104,7 @@ function mockDockerRun(opts: { exitCode?: number } = {}) { runStream: (runOpts, streamOpts) => { runs.push(runOpts); captureOptsCalls.push({ teeStderr: streamOpts.teeStderr }); - return Effect.succeed({ exitCode: opts.exitCode ?? 0, stderr: "" }); + return Effect.succeed({ exitCode: opts.exitCode ?? 0, stderr: opts.stderr ?? "" }); }, }); return { layer, runs, captureOptsCalls }; @@ -622,6 +622,42 @@ describe("legacyRunFreshDbSetup", () => { }), ); }); + + it.effect( + "a non-zero exit includes the last meaningful container stderr line without --debug", + () => { + const workdir = makeWorkdir(); + const { session } = fakeSession(); + const out = mockOutput(); + const docker = mockDockerRun({ + exitCode: 1, + stderr: [ + "ulimit: stack size unlimited", + "StorageBackendError: Migration fix-search-by-timestamp-sqli not found", + " at Object.InternalError (/app/dist/internal/errors/codes.js:278:34)", + " at /app/dist/internal/database/migrations/migrate.js:572:36", + "Node.js v22.14.0", + "", + ].join("\n"), + }); + const config = decodeConfig({ storage: { enabled: false }, auth: { enabled: false } }); + return run( + baseInput(workdir, session, { majorVersion: 15, config, debug: false }), + out, + docker, + ).pipe( + Effect.flip, + Effect.map((error) => { + expect(error).toBeInstanceOf(LegacyDbSetupError); + expect((error as LegacyDbSetupError).message).toBe( + "error running container: exit 1:\nStorageBackendError: Migration fix-search-by-timestamp-sqli not found", + ); + expect(docker.captureOptsCalls).toEqual([{ teeStderr: false }]); + rmSync(workdir, { recursive: true, force: true }); + }), + ); + }, + ); }); describe("ApplyApiPrivileges tri-state", () => { diff --git a/apps/cli/src/legacy/shared/db-bootstrap/reset-local-database.ts b/apps/cli/src/legacy/shared/db-bootstrap/reset-local-database.ts index c486fe8009..2917303997 100644 --- a/apps/cli/src/legacy/shared/db-bootstrap/reset-local-database.ts +++ b/apps/cli/src/legacy/shared/db-bootstrap/reset-local-database.ts @@ -40,6 +40,7 @@ import { Data, Effect, FileSystem, Option, Path } from "effect"; import { ChildProcessSpawner } from "effect/unstable/process"; import { detectGitBranch } from "../../../shared/git/git-branch.ts"; +import { clearDraftJournalFile } from "../../../shared/schema/clear-draft-journal.ts"; import { LegacyDebugFlag, LegacyNetworkIdFlag, @@ -209,6 +210,8 @@ export const legacyResetLocalDatabase = Effect.fnUntraced(function* ( setup: { ...setup, experimental }, }); + yield* clearDraftJournalFile(fs, path, workdir); + // Seed objects from supabase/buckets when storage is up (Go gates buckets on // an existing, healthy storage container). Reuses the ported seed-buckets // local path; its summary is suppressed (reset emits its own result). diff --git a/apps/cli/src/legacy/shared/legacy-db-connection.service.ts b/apps/cli/src/legacy/shared/legacy-db-connection.service.ts index 674876e74c..6837c530a2 100644 --- a/apps/cli/src/legacy/shared/legacy-db-connection.service.ts +++ b/apps/cli/src/legacy/shared/legacy-db-connection.service.ts @@ -105,6 +105,12 @@ export interface LegacyDbSession { * statement: consumers embed it verbatim in batches. */ readonly restoreRoleSql?: string; + /** + * SQL that restores hosted `postgres` search_path after a CLI-owned `RESET ALL`. + * Login-role GUC defaults omit `extensions`; SET ROLE does not adopt postgres + * rolconfig. Absent when the caller does not own that restore. + */ + readonly restoreSearchPathSql?: string; /** Run a single SQL statement, ignoring any returned rows. */ readonly exec: (sql: string) => Effect.Effect; /** diff --git a/apps/cli/src/legacy/shared/legacy-db-target-flags.ts b/apps/cli/src/legacy/shared/legacy-db-target-flags.ts index 964434c3a7..5239854fc1 100644 --- a/apps/cli/src/legacy/shared/legacy-db-target-flags.ts +++ b/apps/cli/src/legacy/shared/legacy-db-target-flags.ts @@ -80,6 +80,8 @@ export const VALUE_CONSUMING_LONG_FLAGS = new Set([ "level", "fail-on", "type", + // schema-first migrations diff/list target override (`Flag.string("against")`) + "against", // migration/db credential flag — `StringVarP(&dbPassword, "password", "p", …)` // consumes the next token as the value. "password", diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index f343028926..966ba4ff1c 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -475,6 +475,7 @@ export const legacyMarkError = (stat: string, pos: number): string => { // PostgreSQL error messages like `type "ltree" does not exist`. Unanchored, so it // matches identically inside the rendered `ERROR: … (SQLSTATE …)` head line. const TYPE_NAME_PATTERN = /type "([^"]+)" does not exist/; +const FUNCTION_NAME_PATTERN = /function "?([^\s"(]+)"?\(/; /** * Mirrors `MigrationFile.ExecBatch` error context: @@ -508,6 +509,14 @@ export const legacyFormatExecBatchError = ( msg.push(` CREATE TABLE example (col extensions.${typeName});`); msg.push(" Learn more: supabase migration new --help"); } + const functionName = FUNCTION_NAME_PATTERN.exec(e.message)?.[1]; + if (functionName !== undefined && e.code === "42883" && !functionName.includes(".")) { + msg.push(""); + msg.push("Hint: This function may be defined in a schema that's not in your search_path."); + msg.push(` Hosted Postgres usually has search_path ("$user", public, extensions).`); + msg.push(` Use a schema-qualified call (for example extensions.${functionName})`); + msg.push(` or SET search_path TO "$user", public, extensions.`); + } msg.push(`At statement: ${index}`, marked); return formattedExecBatchFailure(`${legacyErrorMessage(e)}\n${msg.join("\n")}`, e); }; @@ -805,7 +814,14 @@ const resetConnectionState = ( session: LegacyDbSession, mapError: (message: string) => E, ): Effect.Effect => - session.exec("RESET ALL").pipe(Effect.mapError((e) => mapError(legacyErrorMessage(e)))); + Effect.gen(function* () { + yield* session.exec("RESET ALL").pipe(Effect.mapError((e) => mapError(legacyErrorMessage(e)))); + if (session.restoreSearchPathSql !== undefined) { + yield* session + .exec(session.restoreSearchPathSql) + .pipe(Effect.mapError((e) => mapError(legacyErrorMessage(e)))); + } + }); /** * Applies a single migration file to the connected database and records it in diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts index 0bacc45136..ef735a2ef1 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts @@ -43,6 +43,7 @@ function fakeSession( failAfterBatch?: boolean; failWith?: { message: string; code?: string; detail?: string; position?: number }; restoreRoleSql?: string; + restoreSearchPathSql?: string; } = {}, ) { const calls: Array<{ @@ -53,6 +54,9 @@ function fakeSession( }> = []; const session: LegacyDbSession = { ...(opts.restoreRoleSql === undefined ? {} : { restoreRoleSql: opts.restoreRoleSql }), + ...(opts.restoreSearchPathSql === undefined + ? {} + : { restoreSearchPathSql: opts.restoreSearchPathSql }), exec: (sql) => { calls.push({ kind: "exec", sql }); return opts.failOn !== undefined && sql.includes(opts.failOn) @@ -172,6 +176,28 @@ describe("legacyApplyMigrationFile", () => { }, ); + it.effect("restores platform search_path after RESET ALL on a stepped-down session", () => { + const dir = mkdtempSync(join(tmpdir(), "legacy-apply-")); + const file = join(dir, "20240101120000_add_col.sql"); + writeFileSync(file, "ALTER TABLE a ADD COLUMN b int;"); + const searchPath = `SET search_path TO "$user", public, extensions`; + const { session, calls } = fakeSession({ + restoreRoleSql: "SET SESSION ROLE postgres", + restoreSearchPathSql: searchPath, + }); + return run(session, file).pipe( + Effect.tap(() => + Effect.sync(() => { + const execs = executedSql(calls); + expect(execs[0]).toBe("RESET ALL"); + expect(execs[1]).toBe(searchPath); + expect(execs.indexOf(searchPath)).toBeLessThan(execs.indexOf("BEGIN")); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }); + it.effect("records a versioned empty migration in one batch", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-apply-")); const file = join(dir, "20240101120000_empty.sql"); @@ -982,6 +1008,39 @@ describe("migration failure rendering (Go ExecBatch parity)", () => { ); }); + it.effect("hints search_path when an unqualified function is missing", () => { + const stat = "SELECT gen_random_bytes(16)"; + return failing(stat, { + message: "ERROR: function gen_random_bytes(integer) does not exist (SQLSTATE 42883)", + code: "42883", + }).pipe( + Effect.tap((message) => + Effect.sync(() => { + expect(message).toContain( + "Hint: This function may be defined in a schema that's not in your search_path.", + ); + expect(message).toContain("extensions.gen_random_bytes"); + expect(message).toContain(`SET search_path TO "$user", public, extensions`); + }), + ), + ); + }); + + it.effect("skips the function hint when the name is already schema-qualified", () => { + const stat = "SELECT extensions.gen_random_bytes(16)"; + return failing(stat, { + message: + "ERROR: function extensions.gen_random_bytes(integer) does not exist (SQLSTATE 42883)", + code: "42883", + }).pipe( + Effect.tap((message) => + Effect.sync(() => { + expect(message).not.toContain("Hint: This function may be defined"); + }), + ), + ); + }); + it.effect("skips the 42704 hint when the type is already schema-qualified", () => { const stat = "CREATE TABLE test (path extensions.ltree NOT NULL)"; return failing(stat, { diff --git a/apps/cli/src/shared/database/database-pool.ts b/apps/cli/src/shared/database/database-pool.ts new file mode 100644 index 0000000000..7b2fe0b175 --- /dev/null +++ b/apps/cli/src/shared/database/database-pool.ts @@ -0,0 +1,74 @@ +import { Effect } from "effect"; +import pg from "pg"; +import { parseSslConfig } from "@supabase/pg-delta/frontends"; + +const SUPERUSER_ROLE = "supabase_admin"; +const CLI_LOGIN_PREFIX = "cli_login_"; +const SET_SESSION_ROLE = "SET SESSION ROLE postgres"; + +/** Hosted `postgres` rolconfig. SET ROLE does not adopt it; RESET ALL drops it. */ +export const PLATFORM_SEARCH_PATH_SQL = `SET search_path TO "$user", public, extensions`; + +/** + * Same rule as LegacyDbConnection remote AfterConnect: minted `cli_login_*` + * and `supabase_admin` must become `postgres` so owner-only + * `supabase_migrations` is usable. Strips a Supavisor `.{ref}` suffix first. + */ +export function needsRoleStepDown(user: string): boolean { + const base = user.split(".")[0] ?? user; + return base.toLowerCase() === SUPERUSER_ROLE || base.startsWith(CLI_LOGIN_PREFIX); +} + +function connectionUser(connectionString: string): string | undefined { + try { + const user = decodeURIComponent(new URL(connectionString).username); + return user.length > 0 ? user : undefined; + } catch { + return undefined; + } +} + +type StepDownClient = { + readonly query: (sql: string) => Promise; +}; + +type DatabasePoolConfig = pg.PoolConfig & { + readonly verify?: typeof databasePoolStepDownVerify; +}; + +const toPoolError = (error: unknown): Error => + error instanceof Error ? error : new Error(String(error)); + +/** pg-pool `verify`: every new physical connection steps down before checkout. */ +export function databasePoolStepDownVerify( + client: StepDownClient, + callback: (err?: Error) => void, +): void { + client.query(SET_SESSION_ROLE).then( + () => + client.query(PLATFORM_SEARCH_PATH_SQL).then( + () => callback(), + (error) => callback(toPoolError(error)), + ), + (error) => callback(toPoolError(error)), + ); +} + +export const acquireDatabasePool = (connectionString: string) => + Effect.acquireRelease( + Effect.sync(() => { + const { ssl, cleanedUrl } = parseSslConfig(connectionString); + const user = connectionUser(cleanedUrl); + const stepDownRequired = user !== undefined && needsRoleStepDown(user); + const config: DatabasePoolConfig = { + connectionString: cleanedUrl, + max: 5, + ...(ssl !== undefined ? { ssl } : {}), + ...(stepDownRequired ? { verify: databasePoolStepDownVerify } : {}), + }; + const pool = new pg.Pool(config); + pool.on("error", () => undefined); + return pool; + }), + (pool) => Effect.promise(() => pool.end()), + ); diff --git a/apps/cli/src/shared/database/database-pool.unit.test.ts b/apps/cli/src/shared/database/database-pool.unit.test.ts new file mode 100644 index 0000000000..ab769998f6 --- /dev/null +++ b/apps/cli/src/shared/database/database-pool.unit.test.ts @@ -0,0 +1,95 @@ +import { describe, expect, it } from "@effect/vitest"; +import { Effect } from "effect"; +import { + acquireDatabasePool, + databasePoolStepDownVerify, + needsRoleStepDown, +} from "./database-pool.ts"; + +describe("needsRoleStepDown", () => { + it("steps down minted login roles and supabase_admin", () => { + expect(needsRoleStepDown("cli_login_abc")).toBe(true); + expect(needsRoleStepDown("cli_login_abc.txpqcjhbturcubdkrzoz")).toBe(true); + expect(needsRoleStepDown("supabase_admin")).toBe(true); + expect(needsRoleStepDown("SUPABASE_ADMIN")).toBe(true); + expect(needsRoleStepDown("supabase_admin.ref")).toBe(true); + }); + + it("never re-asserts a role on sessions that did not step down", () => { + expect(needsRoleStepDown("postgres")).toBe(false); + expect(needsRoleStepDown("postgres.ref")).toBe(false); + expect(needsRoleStepDown("repro_writer")).toBe(false); + }); +}); + +describe("databasePoolStepDownVerify", () => { + it("runs SET SESSION ROLE postgres and reports success to the pool", async () => { + const queries: Array = []; + const client = { query: (sql: string) => (queries.push(sql), Promise.resolve()) }; + const done = await new Promise((resolve) => { + databasePoolStepDownVerify(client, resolve); + }); + expect(queries).toEqual([ + "SET SESSION ROLE postgres", + `SET search_path TO "$user", public, extensions`, + ]); + expect(done).toBeUndefined(); + }); + + it("propagates a failing search_path restore after a successful step-down", async () => { + const failure = new Error("cannot set search_path"); + const queries: Array = []; + const client = { + query: (sql: string) => { + queries.push(sql); + return sql.includes("search_path") ? Promise.reject(failure) : Promise.resolve(); + }, + }; + const done = await new Promise((resolve) => { + databasePoolStepDownVerify(client, resolve); + }); + expect(queries).toEqual([ + "SET SESSION ROLE postgres", + `SET search_path TO "$user", public, extensions`, + ]); + expect(done).toBe(failure); + }); + + it("propagates a failing step-down to the pool callback", async () => { + const failure = new Error("permission denied to set role"); + const client = { query: () => Promise.reject(failure) }; + const done = await new Promise((resolve) => { + databasePoolStepDownVerify(client, resolve); + }); + expect(done).toBe(failure); + }); + + it("wraps a non-Error rejection into an Error for the pool callback", async () => { + const client = { query: () => Promise.reject("boom") }; + const done = await new Promise((resolve) => { + databasePoolStepDownVerify(client, resolve); + }); + expect(done).toBeInstanceOf(Error); + expect(String(done)).toContain("boom"); + }); +}); + +describe("acquireDatabasePool", () => { + it.effect("installs SET SESSION ROLE postgres on cli_login_* pools", () => + Effect.gen(function* () { + const pool = yield* acquireDatabasePool( + "postgresql://cli_login_abc.txpqcjhbturcubdkrzoz:secret@127.0.0.1:1/postgres", + ); + expect(Reflect.get(pool.options, "verify")).toBe(databasePoolStepDownVerify); + }).pipe(Effect.scoped), + ); + + it.effect("does not step down local postgres pools", () => + Effect.gen(function* () { + const pool = yield* acquireDatabasePool( + "postgresql://postgres:postgres@127.0.0.1:54322/postgres", + ); + expect(Reflect.get(pool.options, "verify")).toBeUndefined(); + }).pipe(Effect.scoped), + ); +}); diff --git a/apps/cli/src/shared/database/database-target.service.ts b/apps/cli/src/shared/database/database-target.service.ts new file mode 100644 index 0000000000..df4ce5315f --- /dev/null +++ b/apps/cli/src/shared/database/database-target.service.ts @@ -0,0 +1,22 @@ +import type { Effect } from "effect"; +import { Context } from "effect"; +import type { + SchemaLinkedConnectionError, + SchemaLocalStackNotRunningError, + SchemaTargetRequiredError, +} from "../schema/schema-errors.ts"; +import type { DatabaseTarget, DatabaseTargetSelector } from "./database-target.ts"; + +interface DatabaseTargetResolverShape { + readonly resolve: ( + selector: DatabaseTargetSelector, + ) => Effect.Effect< + DatabaseTarget, + SchemaLocalStackNotRunningError | SchemaLinkedConnectionError | SchemaTargetRequiredError + >; +} + +export class DatabaseTargetResolver extends Context.Service< + DatabaseTargetResolver, + DatabaseTargetResolverShape +>()("supabase/database/DatabaseTargetResolver") {} diff --git a/apps/cli/src/shared/database/database-target.ts b/apps/cli/src/shared/database/database-target.ts new file mode 100644 index 0000000000..33162c141f --- /dev/null +++ b/apps/cli/src/shared/database/database-target.ts @@ -0,0 +1,43 @@ +type DatabaseTargetKind = "local" | "linked" | "url"; + +export type DatabaseTargetSelector = + | { readonly kind: "local" } + | { readonly kind: "linked" } + | { readonly kind: "url"; readonly url: string }; + +export type DatabaseTarget = { + readonly kind: DatabaseTargetKind; + readonly identity: string; + readonly connectionString: string; + readonly disposable: boolean; + readonly durable: boolean; + readonly connectionVerified: boolean; + readonly projectRef?: string; + readonly connectionSource?: "env" | "flag"; +}; + +export function envDatabaseUrl(): string | undefined { + return process.env["SUPABASE_DB_URL"] ?? process.env["DATABASE_URL"]; +} + +export function envDatabaseUrlVarName(): "SUPABASE_DB_URL" | "DATABASE_URL" | undefined { + if (process.env["SUPABASE_DB_URL"] !== undefined) return "SUPABASE_DB_URL"; + if (process.env["DATABASE_URL"] !== undefined) return "DATABASE_URL"; + return undefined; +} + +export function parseTargetSelector(value: string): DatabaseTargetSelector { + if (value === "local") return { kind: "local" }; + if (value === "linked") return { kind: "linked" }; + return { kind: "url", url: value }; +} + +export function redactConnectionString(url: string): string { + try { + const parsed = new URL(url); + if (parsed.password) parsed.password = "****"; + return parsed.toString(); + } catch { + return ""; + } +} diff --git a/apps/cli/src/shared/database/database-target.unit.test.ts b/apps/cli/src/shared/database/database-target.unit.test.ts new file mode 100644 index 0000000000..45d4daf6c4 --- /dev/null +++ b/apps/cli/src/shared/database/database-target.unit.test.ts @@ -0,0 +1,20 @@ +import { describe, expect, test } from "vitest"; +import { envDatabaseUrl, envDatabaseUrlVarName } from "./database-target.ts"; + +describe("envDatabaseUrl", () => { + test("prefers SUPABASE_DB_URL over DATABASE_URL", () => { + const previousSupa = process.env["SUPABASE_DB_URL"]; + const previousDb = process.env["DATABASE_URL"]; + process.env["SUPABASE_DB_URL"] = "postgresql://supabase"; + process.env["DATABASE_URL"] = "postgresql://database"; + try { + expect(envDatabaseUrl()).toBe("postgresql://supabase"); + expect(envDatabaseUrlVarName()).toBe("SUPABASE_DB_URL"); + } finally { + if (previousSupa === undefined) delete process.env["SUPABASE_DB_URL"]; + else process.env["SUPABASE_DB_URL"] = previousSupa; + if (previousDb === undefined) delete process.env["DATABASE_URL"]; + else process.env["DATABASE_URL"] = previousDb; + } + }); +}); diff --git a/apps/cli/src/shared/database/destructive-auth.integration.test.ts b/apps/cli/src/shared/database/destructive-auth.integration.test.ts new file mode 100644 index 0000000000..02b8595e81 --- /dev/null +++ b/apps/cli/src/shared/database/destructive-auth.integration.test.ts @@ -0,0 +1,141 @@ +import { describe, expect, it } from "@effect/vitest"; +import { Effect, Exit, Layer } from "effect"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import { authorizeMutation } from "./destructive-auth.ts"; +import type { DatabaseTarget } from "./database-target.ts"; + +const local: DatabaseTarget = { + kind: "local", + identity: "local:default", + connectionString: "postgresql://postgres:postgres@127.0.0.1:54322/postgres", + disposable: true, + durable: false, + connectionVerified: true, +}; + +const linked: DatabaseTarget = { + kind: "linked", + identity: "abcdefghijklmnop", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: true, + projectRef: "abcdefghijklmnop", +}; + +const unverifiedLinked: DatabaseTarget = { + kind: "linked", + identity: "abcdefghijklmnop", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: false, + projectRef: "abcdefghijklmnop", +}; + +const url: DatabaseTarget = { + kind: "url", + identity: "connection-string", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: false, +}; + +describe("authorizeMutation", () => { + it.live("auto-approves disposable local targets", () => { + const out = mockOutput({ interactive: false }); + return Effect.gen(function* () { + yield* authorizeMutation({ + target: local, + flags: { yes: false, allowRemote: false }, + command: "schema apply", + }); + }).pipe(Effect.provide(out.layer)); + }); + + it.live("rejects mismatched --project-ref", () => { + const out = mockOutput({ interactive: false }); + return Effect.gen(function* () { + const exit = yield* authorizeMutation({ + target: linked, + flags: { yes: true, allowRemote: false, projectRef: "otherref" }, + command: "migrations push", + }).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + }).pipe(Effect.provide(out.layer)); + }); + + it.live("accepts a matching --project-ref for linked targets", () => { + const out = mockOutput({ interactive: false }); + return Effect.gen(function* () { + yield* authorizeMutation({ + target: unverifiedLinked, + flags: { yes: true, allowRemote: false, projectRef: "abcdefghijklmnop" }, + command: "migrations push", + }); + }).pipe(Effect.provide(out.layer)); + }); + + it.live("accepts --yes for non-interactive linked pushes", () => { + const out = mockOutput({ interactive: false }); + return Effect.gen(function* () { + yield* authorizeMutation({ + target: unverifiedLinked, + flags: { yes: true, allowRemote: false }, + command: "migrations push", + }); + }).pipe(Effect.provide(out.layer)); + }); + + it.live("requires --yes or --project-ref for non-interactive linked pushes", () => { + const out = mockOutput({ interactive: false }); + return Effect.gen(function* () { + const exit = yield* authorizeMutation({ + target: unverifiedLinked, + flags: { yes: false, allowRemote: false }, + command: "migrations push", + }).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + }).pipe(Effect.provide(out.layer)); + }); + + it.live("requires --allow-remote for URL targets", () => { + const out = mockOutput({ interactive: false }); + return Effect.gen(function* () { + const exit = yield* authorizeMutation({ + target: url, + flags: { yes: true, allowRemote: false }, + command: "migrations push", + }).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(JSON.stringify(exit)).toContain("--allow-remote"); + }).pipe(Effect.provide(Layer.mergeAll(out.layer))); + }); + + it.live("tells the user to unset DATABASE_URL when the URL came from env", () => { + const previous = process.env["DATABASE_URL"]; + process.env["DATABASE_URL"] = "postgresql://postgres:secret@other.example/postgres"; + const out = mockOutput({ interactive: false }); + return Effect.gen(function* () { + const exit = yield* authorizeMutation({ + target: { ...url, connectionSource: "env" }, + flags: { yes: true, allowRemote: false }, + command: "migrations push", + }).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(JSON.stringify(exit)).toContain("Unset DATABASE_URL"); + }).pipe( + Effect.provide(Layer.mergeAll(out.layer)), + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) { + delete process.env["DATABASE_URL"]; + } else { + process.env["DATABASE_URL"] = previous; + } + }), + ), + ); + }); +}); diff --git a/apps/cli/src/shared/database/destructive-auth.ts b/apps/cli/src/shared/database/destructive-auth.ts new file mode 100644 index 0000000000..1d58300e0a --- /dev/null +++ b/apps/cli/src/shared/database/destructive-auth.ts @@ -0,0 +1,81 @@ +import { Effect } from "effect"; +import { + SchemaAllowRemoteRequiredError, + SchemaCancelledError, + SchemaDestructiveAuthError, + SchemaProjectRefMismatchError, +} from "../schema/schema-errors.ts"; +import { envDatabaseUrlVarName, type DatabaseTarget } from "./database-target.ts"; +import { Output } from "../output/output.service.ts"; + +export type MutationAuthFlags = { + readonly yes: boolean; + readonly projectRef?: string; + readonly allowRemote: boolean; +}; + +export const authorizeMutation = Effect.fnUntraced(function* (input: { + readonly target: DatabaseTarget; + readonly flags: MutationAuthFlags; + readonly command: string; +}) { + const { target, flags, command } = input; + + if (target.disposable) { + return; + } + + if (target.kind === "url") { + if (!flags.allowRemote) { + const envVar = target.connectionSource === "env" ? envDatabaseUrlVarName() : undefined; + return yield* new SchemaAllowRemoteRequiredError({ + detail: + envVar !== undefined + ? `This connection string cannot be identity-verified because ${envVar} is set.` + : "This connection string cannot be identity-verified.", + suggestion: + envVar !== undefined + ? `Unset ${envVar} to use the linked project connection, or re-run ${command} with --allow-remote if this URL is the intended durable target.` + : `Re-run ${command} with --allow-remote to acknowledge the unverifiable target.`, + }); + } + return; + } + + const resolvedRef = target.projectRef; + if (resolvedRef === undefined) { + return yield* new SchemaProjectRefMismatchError({ + detail: "Durable target is missing a project ref.", + suggestion: "Link the project or pass --project-ref .", + }); + } + + if (flags.projectRef !== undefined) { + if (flags.projectRef !== resolvedRef) { + return yield* new SchemaProjectRefMismatchError({ + detail: `--project-ref ${flags.projectRef} does not match resolved target ${resolvedRef}.`, + suggestion: `Pass --project-ref ${resolvedRef}.`, + }); + } + return; + } + + const output = yield* Output; + if (output.interactive) { + const typed = yield* output.promptText(`Type the project ref (${resolvedRef}) to continue`); + if (typed.trim() !== resolvedRef) { + return yield* new SchemaCancelledError({ + detail: "Project ref confirmation did not match.", + suggestion: `Type ${resolvedRef} exactly, or pass --project-ref ${resolvedRef}.`, + }); + } + return; + } + + if (!flags.yes) { + return yield* new SchemaDestructiveAuthError({ + detail: "Non-interactive mutation of a durable target requires confirmation.", + suggestion: "Pass --yes, or pass --project-ref to assert the target identity.", + }); + } +}); diff --git a/apps/cli/src/shared/database/linked-remote-connector.service.ts b/apps/cli/src/shared/database/linked-remote-connector.service.ts new file mode 100644 index 0000000000..caba6f7232 --- /dev/null +++ b/apps/cli/src/shared/database/linked-remote-connector.service.ts @@ -0,0 +1,12 @@ +import type { Effect } from "effect"; +import { Context } from "effect"; +import type { SchemaLinkedConnectionError } from "../schema/schema-errors.ts"; + +interface LinkedRemoteConnectorShape { + readonly connect: (projectRef: string) => Effect.Effect; +} + +export class LinkedRemoteConnector extends Context.Service< + LinkedRemoteConnector, + LinkedRemoteConnectorShape +>()("supabase/database/LinkedRemoteConnector") {} diff --git a/apps/cli/src/shared/database/local-database-fallback.service.ts b/apps/cli/src/shared/database/local-database-fallback.service.ts new file mode 100644 index 0000000000..fdc0ca50ee --- /dev/null +++ b/apps/cli/src/shared/database/local-database-fallback.service.ts @@ -0,0 +1,13 @@ +import { Context, type Effect, Option } from "effect"; +import type { SchemaLocalStackNotRunningError } from "../schema/schema-errors.ts"; +import type { DatabaseTarget } from "./database-target.ts"; + +interface LocalDatabaseFallbackShape { + readonly resolve: Effect.Effect, SchemaLocalStackNotRunningError>; +} + +/** Optional project-owned local DB from this project's Docker `supabase start`. */ +export class LocalDatabaseFallback extends Context.Service< + LocalDatabaseFallback, + LocalDatabaseFallbackShape +>()("supabase/database/LocalDatabaseFallback") {} diff --git a/apps/cli/src/shared/database/local-postgres-url.ts b/apps/cli/src/shared/database/local-postgres-url.ts new file mode 100644 index 0000000000..0ad1b1c320 --- /dev/null +++ b/apps/cli/src/shared/database/local-postgres-url.ts @@ -0,0 +1,30 @@ +const formatPostgresHost = (host: string): string => + host.includes(":") && !host.startsWith("[") ? `[${host}]` : host; + +export const localPostgresConnectionString = ( + port: number, + password: string, + host = "127.0.0.1", +): string => + `postgresql://postgres:${encodeURIComponent(password)}@${formatPostgresHost(host)}:${port}/postgres`; + +function isRecord(value: unknown): value is { readonly [key: string]: unknown } { + return typeof value === "object" && value !== null; +} + +/** + * Host port published for container Postgres (`5432/tcp`) from Docker inspect + * `NetworkSettings.Ports`. Ownership is the named container; this only reads + * the port that container actually published (which may not be 54322). + */ +export const publishedPostgresHostPort = (ports: unknown): number | undefined => { + if (!isRecord(ports)) return undefined; + const bindings = ports["5432/tcp"]; + if (!Array.isArray(bindings) || bindings.length === 0) return undefined; + const first = bindings[0]; + if (!isRecord(first)) return undefined; + const hostPort = first["HostPort"]; + if (typeof hostPort !== "string" && typeof hostPort !== "number") return undefined; + const port = typeof hostPort === "number" ? hostPort : Number(hostPort); + return Number.isInteger(port) && port > 0 && port <= 65535 ? port : undefined; +}; diff --git a/apps/cli/src/shared/database/local-postgres-url.unit.test.ts b/apps/cli/src/shared/database/local-postgres-url.unit.test.ts new file mode 100644 index 0000000000..731a0b2c9f --- /dev/null +++ b/apps/cli/src/shared/database/local-postgres-url.unit.test.ts @@ -0,0 +1,35 @@ +import { describe, expect, it } from "vitest"; +import { localPostgresConnectionString, publishedPostgresHostPort } from "./local-postgres-url.ts"; + +describe("publishedPostgresHostPort", () => { + it("reads the first 5432/tcp HostPort", () => { + expect( + publishedPostgresHostPort({ + "5432/tcp": [{ HostIp: "0.0.0.0", HostPort: "55432" }], + }), + ).toBe(55432); + }); + + it("returns undefined when Postgres is not published", () => { + expect(publishedPostgresHostPort({ "5432/tcp": null })).toBeUndefined(); + expect(publishedPostgresHostPort({})).toBeUndefined(); + expect(publishedPostgresHostPort(null)).toBeUndefined(); + }); +}); + +describe("localPostgresConnectionString", () => { + it("percent-encodes the password", () => { + expect(localPostgresConnectionString(55432, "p@ss")).toBe( + "postgresql://postgres:p%40ss@127.0.0.1:55432/postgres", + ); + }); + + it("uses the caller-supplied host and brackets IPv6", () => { + expect(localPostgresConnectionString(55432, "postgres", "docker.internal")).toBe( + "postgresql://postgres:postgres@docker.internal:55432/postgres", + ); + expect(localPostgresConnectionString(55432, "postgres", "::1")).toBe( + "postgresql://postgres:postgres@[::1]:55432/postgres", + ); + }); +}); diff --git a/apps/cli/src/shared/migrations/apply-local-pending.ts b/apps/cli/src/shared/migrations/apply-local-pending.ts new file mode 100644 index 0000000000..bf3ac75995 --- /dev/null +++ b/apps/cli/src/shared/migrations/apply-local-pending.ts @@ -0,0 +1,93 @@ +import { Effect } from "effect"; +import { acquireDatabasePool } from "../database/database-pool.ts"; +import { SchemaHistoryConflictError } from "../schema/schema-errors.ts"; +import { + imageExtensionCatchupAlreadyPresent, + prepareDeclarativeShadow, +} from "../schema/prepare-declarative-shadow.ts"; +import { PgDeltaSchemaEngine } from "../schema/pg-delta-engine.service.ts"; +import type { MigrationFile } from "./migration-file.ts"; +import { findMatchingPendingPrefix } from "./matching-pending-prefix.ts"; +import { formatHistoryConflict } from "./migration-repair-suggest.ts"; +import { emptyPendingMigrationError } from "./privilege-offer.ts"; +import { MigrationRunner, type MigrationApplyResult } from "./migration-runner.service.ts"; +import type { Pool } from "pg"; + +export const applyLocalPending = Effect.fn("migrations.applyLocalPending")(function* ( + pool: Pool, + local: ReadonlyArray, +) { + const runner = yield* MigrationRunner; + const engine = yield* PgDeltaSchemaEngine; + const history = yield* runner.listRemote(pool); + const present = new Set(history.map((row) => row.version)); + const pending = local.filter((file) => !present.has(file.version)); + if (pending.length === 0) { + return { + applied: [], + recorded: [], + skipped: local.map((file) => file.version), + } satisfies MigrationApplyResult; + } + + const remoteOnly = history.filter((row) => !local.some((file) => file.version === row.version)); + if (remoteOnly.length > 0) { + return yield* new SchemaHistoryConflictError( + formatHistoryConflict({ + remoteOnly: remoteOnly.map((row) => row.version), + pending: pending.map((file) => file.version), + flags: { local: true }, + }), + ); + } + + const empty = emptyPendingMigrationError(pending); + if (empty !== undefined) { + return yield* empty; + } + + const already = local.filter((file) => present.has(file.version)); + const installed = new Set(yield* runner.listInstalledExtensions(pool)); + // First-push catchup recreates image extensions already on the local catalog. + const leftoverCatchup = pending.filter((file) => + imageExtensionCatchupAlreadyPresent(file.content, installed), + ); + const leftoverVersions = new Set(leftoverCatchup.map((file) => file.version)); + const scanPending = pending.filter((file) => !leftoverVersions.has(file.version)); + + if (scanPending.length === 0) { + yield* runner.markApplied(pool, leftoverCatchup); + return { + applied: [], + recorded: leftoverCatchup.map((file) => file.version), + skipped: already.map((file) => file.version), + } satisfies MigrationApplyResult; + } + + const shadow = yield* engine.provisionPlatform; + const shadowPool = yield* acquireDatabasePool(shadow.url); + yield* prepareDeclarativeShadow( + shadowPool, + [...already, ...scanPending].map((file) => ({ name: file.fileName, sql: file.content })), + ); + + const recorded = yield* findMatchingPendingPrefix(shadowPool, pool, already, scanPending, { + failClosed: true, + }); + + const toRecord = [...leftoverCatchup, ...recorded]; + if (toRecord.length > 0) { + yield* runner.markApplied(pool, toRecord); + } + + const remaining = scanPending.slice(recorded.length); + // Full local inventory: applyPending treats a partial list as remote-only history. + const result = + remaining.length === 0 + ? { applied: [], skipped: already.map((file) => file.version) } + : yield* runner.applyPending(pool, local); + return { + ...result, + recorded: toRecord.map((file) => file.version), + } satisfies MigrationApplyResult; +}); diff --git a/apps/cli/src/shared/migrations/apply-migrations.integration.test.ts b/apps/cli/src/shared/migrations/apply-migrations.integration.test.ts new file mode 100644 index 0000000000..4c28b2a240 --- /dev/null +++ b/apps/cli/src/shared/migrations/apply-migrations.integration.test.ts @@ -0,0 +1,422 @@ +import { describe, expect, it } from "@effect/vitest"; +import { classifyPlanHazards, type Plan } from "@supabase/pg-delta/plan"; +import { Cause, Effect, Exit, Layer, Option } from "effect"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { PgDeltaSchemaEngine } from "../schema/pg-delta-engine.service.ts"; +import { SchemaStateStore } from "../schema/schema-state.service.ts"; +import type { SchemaDraftJournal, SchemaPlanView } from "../schema/schema-types.ts"; +import { applyMigrations } from "./apply-migrations.ts"; +import { formatHistoryConflict } from "./migration-repair-suggest.ts"; +import { MigrationRepository } from "./migration-repository.service.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; +import { SchemaEngineError, SchemaHistoryConflictError } from "../schema/schema-errors.ts"; + +const ungeneratedAheadJournal: SchemaDraftJournal = { + version: 1, + draftId: "draft", + targetIdentity: "local:default", + startingMigrationHeadDigest: "abc", + sourceFingerprint: "s", + engineVersion: "0.3.0", + declarativelyAhead: true, + generated: false, + plans: [], +}; + +function emptyPlan(): Plan { + return { + formatVersion: 1, + engineVersion: "0.3.0", + planId: "plan-1", + source: { fingerprint: "s" }, + target: { fingerprint: "d" }, + preamble: [], + deltas: [], + filteredDeltas: [], + renameCandidates: [], + actions: [], + safetyReport: { + destructiveActions: 0, + rewriteRiskActions: 0, + nonTransactionalActions: 0, + lockClasses: {}, + }, + }; +} + +function planView(changes: boolean): SchemaPlanView { + const plan = emptyPlan(); + return { + planId: plan.planId, + sourceFingerprint: plan.source.fingerprint, + desiredFingerprint: plan.target.fingerprint, + engineVersion: plan.engineVersion, + profile: "supabase", + changes, + files: [], + hazards: { + kinds: [], + destructive: 0, + rewrite: 0, + coverageGaps: 0, + report: classifyPlanHazards(plan), + }, + destructive: false, + renameCandidates: [], + acceptedRenames: [], + coverageBlocked: false, + renameBlocked: false, + diagnostics: [], + plan, + }; +} + +const laterFile = { + version: "20260101000001", + name: "next", + fileName: "20260101000001_next.sql", + absolutePath: "/tmp/migrations/20260101000001_next.sql", + content: "select 2;", + transactional: true, +}; + +const leftoverCatchupFile = { + version: "20260101000001", + name: "catchup", + fileName: "20260101000001_catchup.sql", + absolutePath: "/tmp/migrations/20260101000001_catchup.sql", + content: 'CREATE EXTENSION "pgjwt" SCHEMA "extensions";', + transactional: true, +}; + +function setup( + journal: SchemaDraftJournal | undefined, + opts: { + history?: ReadonlyArray<{ version: string; name: string }>; + catalogMatch?: boolean; + catalogMatches?: ReadonlyArray; + installedExtensions?: ReadonlyArray; + failApplying?: boolean; + files?: ReadonlyArray<{ + version: string; + name: string; + fileName: string; + absolutePath: string; + content: string; + transactional: boolean; + }>; + } = {}, +) { + const out = mockOutput({ interactive: false }); + let applyPending = 0; + let marked = 0; + let diffCalls = 0; + const liveApplied: string[] = []; + const recorded = new Set((opts.history ?? []).map((row) => row.version)); + return { + get applyPending() { + return applyPending; + }, + get marked() { + return marked; + }, + liveApplied, + layer: Layer.mergeAll( + out.layer, + Layer.succeed( + DatabaseTargetResolver, + DatabaseTargetResolver.of({ + resolve: () => + Effect.succeed({ + kind: "local", + identity: "local:default", + connectionString: "postgresql://postgres:postgres@127.0.0.1:54322/postgres", + disposable: true, + durable: false, + connectionVerified: true, + }), + }), + ), + Layer.succeed( + SchemaStateStore, + SchemaStateStore.of({ + readJournal: Effect.succeed(journal === undefined ? Option.none() : Option.some(journal)), + writeJournal: () => Effect.void, + clearJournal: Effect.void, + withLock: (effect) => effect, + }), + ), + Layer.succeed( + MigrationRepository, + MigrationRepository.of({ + listLocal: Effect.succeed( + opts.files ?? [ + { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: "select 1;", + transactional: true, + }, + ], + ), + createEmpty: () => Effect.die("unused"), + writeFetched: () => Effect.die("unused"), + writeGenerated: () => Effect.die("unused"), + remove: () => Effect.die("unused"), + }), + ), + Layer.succeed( + MigrationRunner, + MigrationRunner.of({ + listRemote: () => Effect.succeed(opts.history ?? []), + listRemoteStatements: () => Effect.succeed([]), + showServerVersion: () => Effect.succeed(undefined), + listInstalledExtensions: () => Effect.succeed(opts.installedExtensions ?? []), + applyPending: (pool, files) => + Effect.gen(function* () { + applyPending += 1; + const conn = pool.options.connectionString ?? ""; + if (opts.failApplying === true && !conn.includes("54322")) { + return yield* new SchemaEngineError({ + detail: 'Failed applying migration: extension "pgjwt" already exists', + suggestion: "Check the database connection and migration SQL, then retry.", + }); + } + const leftover = files.filter((file) => !recorded.has(file.version)); + const remoteOnly = [...recorded].filter( + (version) => !files.some((file) => file.version === version), + ); + if (remoteOnly.length > 0 && leftover.length > 0) { + return yield* new SchemaHistoryConflictError( + formatHistoryConflict({ + remoteOnly, + pending: leftover.map((file) => file.version), + flags: { local: true }, + }), + ); + } + if (conn.includes("54322")) { + liveApplied.splice(0, liveApplied.length, ...leftover.map((file) => file.version)); + } + return { + applied: leftover.map((file) => file.version), + skipped: files + .filter((file) => recorded.has(file.version)) + .map((file) => file.version), + }; + }), + markApplied: (_pool, files) => + Effect.sync(() => { + marked += 1; + for (const file of files) recorded.add(file.version); + }), + }), + ), + Layer.succeed( + PgDeltaSchemaEngine, + PgDeltaSchemaEngine.of({ + exportSchema: () => Effect.die("unused"), + planFiles: () => Effect.die("unused"), + diffPools: () => { + const match = + opts.catalogMatches !== undefined + ? opts.catalogMatches[diffCalls] === true + : opts.catalogMatch === true; + diffCalls += 1; + return Effect.succeed(planView(!match)); + }, + applyPlan: () => Effect.die("unused"), + provisionShadow: Effect.die("unused"), + provisionPlatform: Effect.succeed({ + url: "postgresql://postgres:postgres@127.0.0.1:1/postgres", + }), + provisionMigrations: Effect.die("unused"), + }), + ), + ), + }; +} + +describe("applyMigrations", () => { + it.live("fails closed when an ungenerated draft is active", () => { + const ctx = setup(ungeneratedAheadJournal); + return Effect.gen(function* () { + const exit = yield* applyMigrations().pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(ctx.applyPending).toBe(0); + expect(ctx.marked).toBe(0); + }); + }); + + it.live("runs pending SQL when the live catalog does not match full replay", () => { + const ctx = setup(undefined); + return Effect.gen(function* () { + const result = yield* applyMigrations().pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(result.message).toContain("Applied"); + expect(ctx.applyPending).toBe(2); + expect(ctx.marked).toBe(0); + expect(ctx.liveApplied).toEqual(["20260101000000"]); + }); + }); + + it.live("marks history when the live catalog already matches full replay", () => { + const ctx = setup(undefined, { catalogMatch: true }); + return Effect.gen(function* () { + const result = yield* applyMigrations().pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(result.message).toContain("Recorded"); + expect(ctx.marked).toBe(1); + }); + }); + + it.live("records leftover image-extension catchup already on the live catalog", () => { + const ctx = setup(undefined, { + files: [ + { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: "select 1;", + transactional: true, + }, + leftoverCatchupFile, + ], + history: [{ version: "20260101000000", name: "init" }], + installedExtensions: ["pgjwt"], + }); + return Effect.gen(function* () { + const result = yield* applyMigrations().pipe(Effect.provide(ctx.layer)); + expect(result.message).toContain("Recorded"); + expect(ctx.marked).toBe(1); + expect(ctx.applyPending).toBe(0); + expect(ctx.liveApplied).toEqual([]); + }); + }); + + it.live("applies remaining pending after recording leftover image-extension catchup", () => { + const ctx = setup(undefined, { + files: [ + { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: "select 1;", + transactional: true, + }, + leftoverCatchupFile, + { + version: "20260101000002", + name: "todos", + fileName: "20260101000002_todos.sql", + absolutePath: "/tmp/migrations/20260101000002_todos.sql", + content: "select 3;", + transactional: true, + }, + ], + history: [{ version: "20260101000000", name: "init" }], + installedExtensions: ["pgjwt"], + }); + return Effect.gen(function* () { + const result = yield* applyMigrations().pipe(Effect.provide(ctx.layer)); + expect(result.message).toContain("Recorded"); + expect(result.message).toContain("Applied"); + expect(ctx.marked).toBe(1); + expect(ctx.liveApplied).toEqual(["20260101000002"]); + }); + }); + + it.live("records a matching prefix then runs the remaining pending SQL", () => { + const ctx = setup(undefined, { + files: [ + { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: "select 1;", + transactional: true, + }, + laterFile, + ], + catalogMatches: [true, false], + }); + return Effect.gen(function* () { + const result = yield* applyMigrations().pipe(Effect.provide(ctx.layer)); + expect(result.message).toContain("Recorded"); + expect(ctx.marked).toBe(1); + expect(ctx.applyPending).toBe(3); + expect(ctx.liveApplied).toEqual([laterFile.version]); + }); + }); + + it.live("fails closed when history has remote-only versions and pending files", () => { + const ctx = setup(undefined, { + history: [{ version: "19990101000000", name: "other" }], + }); + return Effect.gen(function* () { + const exit = yield* applyMigrations().pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(JSON.stringify(exit)).toContain("supabase migrations pull --from local"); + expect(JSON.stringify(exit)).not.toContain("repair --status reverted"); + expect(ctx.applyPending).toBe(0); + expect(ctx.marked).toBe(0); + }); + }); + + it.live("fails closed when the prefix scan cannot replay pending SQL", () => { + const ctx = setup(undefined, { failApplying: true }); + return Effect.gen(function* () { + const exit = yield* applyMigrations().pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : Option.none(); + expect(failure._tag).toBe("Some"); + if (failure._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaEngineError); + expect(failure.value.detail).toContain("pgjwt"); + } + expect(ctx.liveApplied).toEqual([]); + expect(ctx.marked).toBe(0); + }); + }); + + it.live("refuses a pending file with no executable SQL", () => { + const ctx = setup(undefined, { + files: [ + { + version: "20260101000000", + name: "sneak", + fileName: "20260101000000_sneak.sql", + absolutePath: "/tmp/migrations/20260101000000_sneak.sql", + content: "-- empty stub\n", + transactional: true, + }, + ], + }); + return Effect.gen(function* () { + const exit = yield* applyMigrations().pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(JSON.stringify(exit)).toContain("20260101000000_sneak.sql"); + expect(JSON.stringify(exit)).toContain("no executable SQL"); + expect(ctx.applyPending).toBe(0); + expect(ctx.liveApplied).toEqual([]); + }); + }); + + it.live("is a no-op when every local version is already in history", () => { + const ctx = setup(undefined, { + history: [{ version: "20260101000000", name: "init" }], + }); + return Effect.gen(function* () { + const result = yield* applyMigrations().pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(false); + expect(ctx.applyPending).toBe(0); + expect(ctx.marked).toBe(0); + }); + }); +}); diff --git a/apps/cli/src/shared/migrations/apply-migrations.ts b/apps/cli/src/shared/migrations/apply-migrations.ts new file mode 100644 index 0000000000..89caaeffe6 --- /dev/null +++ b/apps/cli/src/shared/migrations/apply-migrations.ts @@ -0,0 +1,65 @@ +import { Effect } from "effect"; +import { acquireDatabasePool } from "../database/database-pool.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { SchemaDraftConflictError } from "../schema/schema-errors.ts"; +import { formatNextAction } from "../schema/schema-output.ts"; +import { SchemaStateStore } from "../schema/schema-state.service.ts"; +import type { SchemaCommandResult } from "../schema/schema-types.ts"; +import { applyLocalPending } from "./apply-local-pending.ts"; +import { MigrationRepository } from "./migration-repository.service.ts"; + +export const applyMigrations = Effect.fn("migrations.apply")(function* () { + const targets = yield* DatabaseTargetResolver; + const repository = yield* MigrationRepository; + const state = yield* SchemaStateStore; + const journal = yield* state.readJournal; + if ( + journal._tag === "Some" && + journal.value.declarativelyAhead && + journal.value.generated !== true + ) { + return yield* new SchemaDraftConflictError({ + detail: "A declarative draft is active on the local database.", + suggestion: + "Run `supabase schema generate`, reset the local database, or discard the draft before applying migration files.", + }); + } + const target = yield* targets.resolve({ kind: "local" }); + const local = yield* repository.listLocal; + + return yield* Effect.scoped( + Effect.gen(function* () { + const pool = yield* acquireDatabasePool(target.connectionString); + const result = yield* applyLocalPending(pool, local); + const recorded = result.recorded ?? []; + const mutatedDatabase = result.applied.length > 0 || recorded.length > 0; + const parts = [ + ...(recorded.length > 0 + ? [`Recorded ${recorded.length} already-applied migration(s): ${recorded.join(", ")}`] + : []), + ...(result.applied.length > 0 + ? [`Applied ${result.applied.length} migration(s): ${result.applied.join(", ")}`] + : []), + ]; + const nextActions = + mutatedDatabase === true ? [formatNextAction("to deploy", "supabase migrations push")] : []; + return { + status: "clean", + message: parts.length > 0 ? parts.join(". ") : "No pending migrations.", + data: { + status: "clean", + applied: result.applied, + recorded, + skipped: result.skipped, + target: target.identity, + mutated_database: mutatedDatabase, + mutated_files: false, + next_actions: nextActions, + }, + nextActions, + mutatedDatabase, + mutatedFiles: false, + } satisfies SchemaCommandResult; + }), + ); +}); diff --git a/apps/cli/src/shared/migrations/diff-migrations.ts b/apps/cli/src/shared/migrations/diff-migrations.ts new file mode 100644 index 0000000000..fcd31ccd90 --- /dev/null +++ b/apps/cli/src/shared/migrations/diff-migrations.ts @@ -0,0 +1,115 @@ +import { Effect, FileSystem } from "effect"; +import { acquireDatabasePool } from "../database/database-pool.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { parseTargetSelector } from "../database/database-target.ts"; +import { SchemaEmptyHistoryReplayError, SchemaWorkspaceIoError } from "../schema/schema-errors.ts"; +import { formatPlanSql } from "../schema/schema-body.ts"; +import { formatNextAction, withCoverageMessage, withPlanSummary } from "../schema/schema-output.ts"; +import type { SchemaCommandResult } from "../schema/schema-types.ts"; +import { PgDeltaSchemaEngine } from "../schema/pg-delta-engine.service.ts"; +import { + formatMigrationsDiffFileCommand, + formatMigrationRepairCommand, + formatMigrationsPushCommand, + repairFlagsForTarget, +} from "./migration-repair-suggest.ts"; +import { MigrationRepository } from "./migration-repository.service.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; +import { warnIfRemotePostgresMajorMismatch } from "./remote-postgres.ts"; + +export type DiffMigrationsInput = { + readonly against?: string; + readonly file?: string; +}; + +export const diffMigrations = Effect.fn("migrations.diff")(function* (input: DiffMigrationsInput) { + const targets = yield* DatabaseTargetResolver; + const engine = yield* PgDeltaSchemaEngine; + const repository = yield* MigrationRepository; + const runner = yield* MigrationRunner; + const live = yield* targets.resolve(parseTargetSelector(input.against ?? "local")); + + return yield* Effect.scoped( + Effect.gen(function* () { + const livePool = yield* acquireDatabasePool(live.connectionString); + yield* warnIfRemotePostgresMajorMismatch(livePool, live); + const localFiles = yield* repository.listLocal; + const history = yield* runner.listRemote(livePool); + const flags = repairFlagsForTarget(live); + if (history.length === 0 && localFiles.length > 0) { + return yield* new SchemaEmptyHistoryReplayError({ + detail: + "Remote history is empty, so there are no applied migrations to replay. Local files exist.", + suggestion: `Apply them first: ${formatMigrationsPushCommand(flags)}. migrations diff is for after histories match.`, + }); + } + const remoteVersions = new Set(history.map((row) => row.version)); + const applied = localFiles.filter((file) => remoteVersions.has(file.version)); + const shadow = yield* engine.provisionPlatform; + const sourcePool = yield* acquireDatabasePool(shadow.url); + yield* runner.applyPending(sourcePool, applied); + const plan = yield* engine.diffPools({ + sourcePool, + desiredPool: livePool, + allowDrops: true, + }); + + const sql = formatPlanSql(plan); + if (input.file !== undefined) { + const fs = yield* FileSystem.FileSystem; + yield* fs.writeFileString(input.file, sql).pipe( + Effect.mapError( + (error) => + new SchemaWorkspaceIoError({ + detail: `Failed to write ${input.file}: ${error.message}`, + suggestion: "Check the output path and retry.", + }), + ), + ); + } + + const nextActions = plan.changes ? nextActionsForDiff(live) : []; + + return { + status: plan.changes ? "drift" : "clean", + message: plan.changes + ? withPlanSummary("Preview only; nothing was changed.", plan) + : withCoverageMessage("Live database matches migration replay.", plan), + ...(plan.changes && sql.length > 0 ? { body: sql } : {}), + data: { + status: plan.changes ? "drift" : "clean", + plan_id: plan.planId, + source_fingerprint: plan.sourceFingerprint, + desired_fingerprint: plan.desiredFingerprint, + hazards: plan.hazards, + sql, + files: plan.files, + file: input.file, + mutated_database: false, + mutated_files: input.file !== undefined, + next_actions: nextActions, + }, + nextActions, + mutatedDatabase: false, + mutatedFiles: input.file !== undefined, + } satisfies SchemaCommandResult; + }), + ); +}); + +function nextActionsForDiff( + target: Parameters[0], +): ReadonlyArray { + const flags = repairFlagsForTarget(target); + return [ + formatNextAction("to write a migration file", formatMigrationsDiffFileCommand(flags)), + formatNextAction( + "to record it as applied without running SQL", + formatMigrationRepairCommand({ + status: "applied", + versions: [""], + flags, + }), + ), + ]; +} diff --git a/apps/cli/src/shared/migrations/list-migrations.ts b/apps/cli/src/shared/migrations/list-migrations.ts new file mode 100644 index 0000000000..3246ae0319 --- /dev/null +++ b/apps/cli/src/shared/migrations/list-migrations.ts @@ -0,0 +1,184 @@ +import { Effect } from "effect"; +import { acquireDatabasePool } from "../database/database-pool.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { parseTargetSelector, type DatabaseTarget } from "../database/database-target.ts"; +import { + formatMigrationInventory, + humanTarget, + type MigrationInventoryStatus, +} from "../schema/schema-body.ts"; +import { formatNextAction } from "../schema/schema-output.ts"; +import type { SchemaCommandResult } from "../schema/schema-types.ts"; +import { + formatMigrationsPullCommand, + formatMigrationsPushCommand, + formatSchemaPullCommand, + repairFlagsForTarget, +} from "./migration-repair-suggest.ts"; +import { MigrationRepository } from "./migration-repository.service.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; +import { warnIfRemotePostgresMajorMismatch } from "./remote-postgres.ts"; + +export type ListMigrationsInput = { + readonly against?: string; +}; + +type ListHistory = "matched" | "pending" | "remote_only" | "conflict"; + +type ListRow = { + readonly version: string; + readonly name: string; + readonly local: boolean; + readonly remote: boolean; +}; + +function rowStatus(row: ListRow): MigrationInventoryStatus { + if (row.local && row.remote) return "applied"; + if (row.local) return "pending"; + return "remote-only"; +} + +function historyAlignment(pending: number, remoteOnly: number): ListHistory { + if (pending > 0 && remoteOnly > 0) return "conflict"; + if (pending > 0) return "pending"; + if (remoteOnly > 0) return "remote_only"; + return "matched"; +} + +function plural(count: number, word: string): string { + return `${count} ${word}${count === 1 ? "" : "s"}`; +} + +function listVerdict(input: { + readonly target: DatabaseTarget; + readonly total: number; + readonly applied: number; + readonly pending: number; + readonly remoteOnly: number; + readonly history: ListHistory; +}): string { + const where = humanTarget(input.target); + if (input.total === 0) { + return `No migrations on ${where}.`; + } + switch (input.history) { + case "matched": + return `${plural(input.applied, "migration")} applied on ${where}. History matches files.`; + case "pending": + return `${input.pending} of ${input.total} ${input.total === 1 ? "migration" : "migrations"} pending on ${where}.`; + case "remote_only": + return `${plural(input.remoteOnly, "remote-only migration")} on ${where} (no local file).`; + case "conflict": + return `${plural(input.pending, "pending migration")} and ${plural(input.remoteOnly, "remote-only migration")} on ${where}.`; + } +} + +function listNextActions(input: { + readonly target: DatabaseTarget; + readonly history: ListHistory; +}): ReadonlyArray { + switch (input.history) { + case "matched": + return []; + case "pending": + return [ + input.target.kind === "local" + ? formatNextAction("to apply it locally", "supabase migrations apply") + : formatNextAction( + "to deploy", + formatMigrationsPushCommand(repairFlagsForTarget(input.target)), + ), + ]; + case "remote_only": + case "conflict": + return [ + formatNextAction( + "to fetch missing files", + formatMigrationsPullCommand(repairFlagsForTarget(input.target)), + ), + formatNextAction( + "to refresh declarations", + formatSchemaPullCommand(repairFlagsForTarget(input.target)), + ), + ]; + } +} + +export const listMigrations = Effect.fn("migrations.list")(function* (input: ListMigrationsInput) { + const repository = yield* MigrationRepository; + const runner = yield* MigrationRunner; + const targets = yield* DatabaseTargetResolver; + const local = yield* repository.listLocal; + const selector = parseTargetSelector(input.against ?? "local"); + const target = yield* targets.resolve(selector); + + return yield* Effect.scoped( + Effect.gen(function* () { + const pool = yield* acquireDatabasePool(target.connectionString); + yield* warnIfRemotePostgresMajorMismatch(pool, target); + const remote = yield* runner.listRemote(pool); + const remoteVersions = new Set(remote.map((row) => row.version)); + const localVersions = new Set(local.map((file) => file.version)); + const rows: ListRow[] = [ + ...local.map((file) => ({ + version: file.version, + name: file.name, + local: true, + remote: remoteVersions.has(file.version), + })), + ...remote + .filter((row) => !localVersions.has(row.version)) + .map((row) => ({ + version: row.version, + name: row.name, + local: false, + remote: true, + })), + ].sort((left, right) => left.version.localeCompare(right.version)); + + const applied = rows.filter((row) => row.local && row.remote).length; + const pending = rows.filter((row) => row.local && !row.remote).length; + const remoteOnly = rows.filter((row) => !row.local && row.remote).length; + const history = historyAlignment(pending, remoteOnly); + const body = formatMigrationInventory( + rows.map((row) => ({ + version: row.version, + name: row.name, + status: rowStatus(row), + })), + ); + + return { + status: "clean", + message: listVerdict({ + target, + total: rows.length, + applied, + pending, + remoteOnly, + history, + }), + ...(body.length > 0 ? { body } : {}), + data: { + status: "clean", + target: target.identity, + migrations: rows, + files: rows.map((row) => ({ + name: row.name, + version: row.version, + status: rowStatus(row), + })), + applied, + pending, + remote_only: remoteOnly, + history, + mutated_database: false, + mutated_files: false, + }, + nextActions: listNextActions({ target, history }), + mutatedDatabase: false, + mutatedFiles: false, + } satisfies SchemaCommandResult; + }), + ); +}); diff --git a/apps/cli/src/shared/migrations/matching-pending-prefix.integration.test.ts b/apps/cli/src/shared/migrations/matching-pending-prefix.integration.test.ts new file mode 100644 index 0000000000..ecc37c1c92 --- /dev/null +++ b/apps/cli/src/shared/migrations/matching-pending-prefix.integration.test.ts @@ -0,0 +1,165 @@ +import { describe, expect, it } from "@effect/vitest"; +import { classifyPlanHazards, type Plan } from "@supabase/pg-delta/plan"; +import { Effect, Exit, Layer } from "effect"; +import type { Pool } from "pg"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import { SchemaEngineError } from "../schema/schema-errors.ts"; +import { PgDeltaSchemaEngine } from "../schema/pg-delta-engine.service.ts"; +import type { SchemaPlanView } from "../schema/schema-types.ts"; +import { findMatchingPendingPrefix } from "./matching-pending-prefix.ts"; +import type { MigrationFile } from "./migration-file.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; + +function emptyPlan(): Plan { + return { + formatVersion: 1, + engineVersion: "0.3.0", + planId: "plan", + source: { fingerprint: "s" }, + target: { fingerprint: "d" }, + preamble: [], + deltas: [], + filteredDeltas: [], + renameCandidates: [], + actions: [], + safetyReport: { + destructiveActions: 0, + rewriteRiskActions: 0, + nonTransactionalActions: 0, + lockClasses: {}, + }, + }; +} + +function planView(changes: boolean): SchemaPlanView { + const plan = emptyPlan(); + return { + planId: plan.planId, + sourceFingerprint: plan.source.fingerprint, + desiredFingerprint: plan.target.fingerprint, + engineVersion: plan.engineVersion, + profile: "supabase", + changes, + files: [], + hazards: { + kinds: [], + destructive: 0, + rewrite: 0, + coverageGaps: 0, + report: classifyPlanHazards(plan), + }, + destructive: false, + renameCandidates: [], + acceptedRenames: [], + coverageBlocked: false, + renameBlocked: false, + diagnostics: [], + plan, + }; +} + +const first = { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: "select 1;", + transactional: true, +}; + +const second = { + version: "20260101000001", + name: "next", + fileName: "20260101000001_next.sql", + absolutePath: "/tmp/migrations/20260101000001_next.sql", + content: "select 2;", + transactional: true, +}; + +function historyPool(opts: { readonly failSql?: string } = {}): Pool { + return { + query: async (sql: string) => { + if (opts.failSql !== undefined && sql === opts.failSql) { + throw new Error("relation does not exist"); + } + return { rows: [] }; + }, + } as Pool; +} + +const engine = Layer.succeed( + PgDeltaSchemaEngine, + PgDeltaSchemaEngine.of({ + exportSchema: () => Effect.die("unused"), + planFiles: () => Effect.die("unused"), + diffPools: () => Effect.succeed(planView(false)), + applyPlan: () => Effect.die("unused"), + provisionShadow: Effect.die("unused"), + provisionPlatform: Effect.die("unused"), + provisionMigrations: Effect.die("unused"), + }), +); + +const runner = Layer.succeed( + MigrationRunner, + MigrationRunner.of({ + listRemote: () => Effect.succeed([]), + listRemoteStatements: () => Effect.succeed([]), + showServerVersion: () => Effect.succeed(undefined), + listInstalledExtensions: () => Effect.succeed([]), + applyPending: (pool, files: ReadonlyArray) => + Effect.gen(function* () { + for (const file of files) { + yield* Effect.tryPromise({ + try: () => pool.query(file.content), + catch: (cause) => + new SchemaEngineError({ + detail: `Failed applying ${file.fileName}: ${cause instanceof Error ? cause.message : String(cause)}`, + suggestion: "Check the migration SQL and retry.", + }), + }); + } + return { applied: files.map((file) => file.version), skipped: [] }; + }), + markApplied: () => Effect.void, + }), +); + +describe("findMatchingPendingPrefix", () => { + it.live("applies a later pending file without treating known history as remote-only", () => + Effect.gen(function* () { + const prefix = yield* findMatchingPendingPrefix( + historyPool(), + historyPool(), + [first], + [second], + ); + expect(prefix.map((file) => file.version)).toEqual([second.version]); + }).pipe(Effect.provide(Layer.mergeAll(runner, engine, mockOutput().layer))), + ); + + it.live("stops the prefix scan when a later file cannot replay", () => + Effect.gen(function* () { + const prefix = yield* findMatchingPendingPrefix( + historyPool({ failSql: second.content }), + historyPool(), + [], + [first, second], + ); + expect(prefix.map((file) => file.version)).toEqual([first.version]); + }).pipe(Effect.provide(Layer.mergeAll(runner, engine, mockOutput().layer))), + ); + + it.live("fails closed when local apply cannot swallow a replay error", () => + Effect.gen(function* () { + const exit = yield* findMatchingPendingPrefix( + historyPool({ failSql: second.content }), + historyPool(), + [], + [first, second], + { failClosed: true }, + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + }).pipe(Effect.provide(Layer.mergeAll(runner, engine, mockOutput().layer))), + ); +}); diff --git a/apps/cli/src/shared/migrations/matching-pending-prefix.ts b/apps/cli/src/shared/migrations/matching-pending-prefix.ts new file mode 100644 index 0000000000..a9f1d70b5f --- /dev/null +++ b/apps/cli/src/shared/migrations/matching-pending-prefix.ts @@ -0,0 +1,46 @@ +import { Effect } from "effect"; +import type { Pool } from "pg"; +import { SchemaEngineError } from "../schema/schema-errors.ts"; +import { PgDeltaSchemaEngine } from "../schema/pg-delta-engine.service.ts"; +import type { MigrationFile } from "./migration-file.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; + +export const findMatchingPendingPrefix = Effect.fn("migrations.findMatchingPendingPrefix")( + function* ( + shadowPool: Pool, + livePool: Pool, + known: ReadonlyArray, + pending: ReadonlyArray, + opts: { readonly failClosed?: boolean } = {}, + ) { + const runner = yield* MigrationRunner; + const engine = yield* PgDeltaSchemaEngine; + let recorded: ReadonlyArray = []; + for (const [index] of pending.entries()) { + const prefix = pending.slice(0, index + 1); + const apply = runner.applyPending(shadowPool, [...known, ...prefix]); + const applied = + opts.failClosed === true + ? yield* apply + : yield* apply.pipe( + Effect.catchIf( + (error): error is SchemaEngineError => + error._tag === "SchemaEngineError" && error.detail.startsWith("Failed applying"), + () => Effect.succeed(undefined), + ), + ); + if (applied === undefined) { + break; + } + const drift = yield* engine.diffPools({ + sourcePool: shadowPool, + desiredPool: livePool, + allowDrops: true, + }); + if (!drift.changes) { + recorded = prefix; + } + } + return recorded; + }, +); diff --git a/apps/cli/src/shared/migrations/migration-file.ts b/apps/cli/src/shared/migrations/migration-file.ts new file mode 100644 index 0000000000..8b3ac0cd78 --- /dev/null +++ b/apps/cli/src/shared/migrations/migration-file.ts @@ -0,0 +1,8 @@ +export type MigrationFile = { + readonly version: string; + readonly name: string; + readonly fileName: string; + readonly absolutePath: string; + readonly content: string; + readonly transactional: boolean; +}; diff --git a/apps/cli/src/shared/migrations/migration-repair-suggest.ts b/apps/cli/src/shared/migrations/migration-repair-suggest.ts new file mode 100644 index 0000000000..f7b157c3ca --- /dev/null +++ b/apps/cli/src/shared/migrations/migration-repair-suggest.ts @@ -0,0 +1,108 @@ +import type { DatabaseTarget } from "../database/database-target.ts"; +import { envDatabaseUrlVarName } from "../database/database-target.ts"; + +export type MigrationRepairFlags = { + readonly local?: boolean; + readonly dbUrlEnvVar?: "DATABASE_URL" | "SUPABASE_DB_URL"; + readonly dbUrlSame?: boolean; + readonly projectRef?: string; +}; + +export function repairFlagsForTarget( + target: DatabaseTarget, + opts: { readonly projectRef?: string; readonly dbUrl?: string } = {}, +): MigrationRepairFlags { + if (target.kind === "local") { + return { local: true }; + } + if (target.kind === "url" || opts.dbUrl !== undefined) { + if (target.connectionSource === "env") { + return { dbUrlEnvVar: envDatabaseUrlVarName() ?? "DATABASE_URL" }; + } + return { dbUrlSame: true }; + } + const projectRef = opts.projectRef ?? target.projectRef; + return projectRef !== undefined ? { projectRef } : {}; +} + +function formatAgainstSelector(flags?: MigrationRepairFlags): string { + if (flags?.local === true) return "local"; + if (flags?.dbUrlEnvVar !== undefined) return `"$${flags.dbUrlEnvVar}"`; + if (flags?.dbUrlSame === true) return ""; + return "linked"; +} + +export function formatMigrationsPushCommand(flags?: MigrationRepairFlags): string { + if (flags?.local === true) return "supabase migrations apply"; + const parts = ["supabase", "migrations", "push"]; + if (flags?.dbUrlEnvVar !== undefined) { + parts.push("--db-url", `"$${flags.dbUrlEnvVar}"`, "--allow-remote"); + } else if (flags?.dbUrlSame === true) { + parts.push("--db-url", "", "--allow-remote"); + } + return parts.join(" "); +} + +function formatMigrationsPullFrom(flags?: MigrationRepairFlags): string { + return `--from ${formatAgainstSelector(flags)}`; +} + +export function formatMigrationsPullCommand(flags?: MigrationRepairFlags): string { + return `supabase migrations pull ${formatMigrationsPullFrom(flags)}`; +} + +export function formatSchemaPullCommand(flags?: MigrationRepairFlags): string { + return `supabase schema pull ${formatMigrationsPullFrom(flags)}`; +} + +export function formatMigrationsDiffFileCommand(flags?: MigrationRepairFlags): string { + return `supabase migrations diff --against ${formatAgainstSelector(flags)} --file supabase/migrations/_.sql`; +} + +export function formatLiveEditCommands(flags?: MigrationRepairFlags): string { + return [ + formatMigrationsDiffFileCommand(flags), + formatMigrationRepairCommand({ + status: "applied", + versions: [""], + flags, + }), + ].join("\n"); +} + +export function formatMigrationRepairCommand(input: { + readonly status: "applied" | "reverted"; + readonly versions: ReadonlyArray; + readonly flags?: MigrationRepairFlags; +}): string { + const parts = ["supabase", "migration", "repair"]; + if (input.flags?.local === true) { + parts.push("--local"); + } + if (input.flags?.dbUrlEnvVar !== undefined) { + parts.push("--db-url", `"$${input.flags.dbUrlEnvVar}"`); + } else if (input.flags?.dbUrlSame === true) { + parts.push("--db-url", ""); + } + if (input.flags?.projectRef !== undefined) { + parts.push("--project-ref", input.flags.projectRef); + } + parts.push("--status", input.status, ...input.versions); + return parts.join(" "); +} + +export function formatHistoryConflict(input: { + readonly remoteOnly: ReadonlyArray; + readonly pending: ReadonlyArray; + readonly flags?: MigrationRepairFlags; +}): { readonly detail: string; readonly suggestion: string } { + const remote = input.remoteOnly.join(", "); + const detail = + input.pending.length > 0 + ? `Local and remote migration histories have diverged (remote-only: ${remote}; pending: ${input.pending.join(", ")}).` + : `Remote history has versions with no local files: ${remote}.`; + return { + detail, + suggestion: formatMigrationsPullCommand(input.flags), + }; +} diff --git a/apps/cli/src/shared/migrations/migration-repair-suggest.unit.test.ts b/apps/cli/src/shared/migrations/migration-repair-suggest.unit.test.ts new file mode 100644 index 0000000000..c4d6ecdb89 --- /dev/null +++ b/apps/cli/src/shared/migrations/migration-repair-suggest.unit.test.ts @@ -0,0 +1,178 @@ +import { describe, expect, test } from "vitest"; +import { + formatHistoryConflict, + formatLiveEditCommands, + formatMigrationRepairCommand, + formatMigrationsPullCommand, + formatMigrationsPushCommand, + formatSchemaPullCommand, + repairFlagsForTarget, +} from "./migration-repair-suggest.ts"; + +const linked = { + kind: "linked" as const, + identity: "abcdefghijklmnop", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: true, + projectRef: "abcdefghijklmnop", +}; + +const local = { + kind: "local" as const, + identity: "local:default", + connectionString: "postgresql://postgres:postgres@127.0.0.1:54322/postgres", + disposable: true, + durable: false, + connectionVerified: true, +}; + +const url = { + kind: "url" as const, + identity: "connection-string", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: false, +}; + +describe("formatMigrationRepairCommand", () => { + test("prefills linked applied versions", () => { + expect( + formatMigrationRepairCommand({ + status: "applied", + versions: ["20260819120000"], + }), + ).toBe("supabase migration repair --status applied 20260819120000"); + }); + + test("adds --local and space-separated versions", () => { + expect( + formatMigrationRepairCommand({ + status: "reverted", + versions: ["111", "222"], + flags: { local: true }, + }), + ).toBe("supabase migration repair --local --status reverted 111 222"); + }); + + test("uses a same-url placeholder for flag URL targets", () => { + expect( + formatMigrationRepairCommand({ + status: "applied", + versions: ["20260819120000"], + flags: { dbUrlSame: true }, + }), + ).toBe("supabase migration repair --db-url --status applied 20260819120000"); + }); + + test("uses an env-var placeholder for env URL targets", () => { + expect( + formatMigrationRepairCommand({ + status: "applied", + versions: ["20260819120000"], + flags: { dbUrlEnvVar: "SUPABASE_DB_URL" }, + }), + ).toBe('supabase migration repair --db-url "$SUPABASE_DB_URL" --status applied 20260819120000'); + }); +}); + +describe("repairFlagsForTarget", () => { + test("marks local targets", () => { + expect(repairFlagsForTarget(local)).toEqual({ local: true }); + }); + + test("keeps an explicit --project-ref on linked targets", () => { + expect(repairFlagsForTarget(linked, { projectRef: "abcdefghijklmnop" })).toEqual({ + projectRef: "abcdefghijklmnop", + }); + }); + + test("picks up projectRef from a linked target without opts", () => { + expect(repairFlagsForTarget(linked)).toEqual({ + projectRef: "abcdefghijklmnop", + }); + }); + + test("uses a same-url placeholder for explicit --db-url / --from targets", () => { + expect(repairFlagsForTarget(url)).toEqual({ dbUrlSame: true }); + }); + + test("uses the env var name when the URL came from the environment", () => { + const previousSupa = process.env["SUPABASE_DB_URL"]; + const previousDb = process.env["DATABASE_URL"]; + delete process.env["SUPABASE_DB_URL"]; + delete process.env["DATABASE_URL"]; + try { + expect(repairFlagsForTarget({ ...url, connectionSource: "env" })).toEqual({ + dbUrlEnvVar: "DATABASE_URL", + }); + } finally { + if (previousSupa === undefined) delete process.env["SUPABASE_DB_URL"]; + else process.env["SUPABASE_DB_URL"] = previousSupa; + if (previousDb === undefined) delete process.env["DATABASE_URL"]; + else process.env["DATABASE_URL"] = previousDb; + } + }); +}); + +describe("formatHistoryConflict", () => { + test("points remote-only and pending at migrations pull", () => { + expect( + formatHistoryConflict({ + remoteOnly: ["19990101000000"], + pending: ["20260819120000"], + flags: { dbUrlSame: true }, + }), + ).toEqual({ + detail: + "Local and remote migration histories have diverged (remote-only: 19990101000000; pending: 20260819120000).", + suggestion: "supabase migrations pull --from ", + }); + }); +}); + +describe("formatMigrationsPullCommand", () => { + test("points remote-only history at fetch-pull without echoing secrets", () => { + expect(formatMigrationsPullCommand()).toBe("supabase migrations pull --from linked"); + expect(formatMigrationsPullCommand({ local: true })).toBe( + "supabase migrations pull --from local", + ); + expect(formatMigrationsPullCommand({ dbUrlEnvVar: "DATABASE_URL" })).toBe( + 'supabase migrations pull --from "$DATABASE_URL"', + ); + expect(formatMigrationsPullCommand({ dbUrlSame: true })).toBe( + "supabase migrations pull --from ", + ); + expect(formatSchemaPullCommand()).toBe("supabase schema pull --from linked"); + expect(formatSchemaPullCommand({ local: true })).toBe("supabase schema pull --from local"); + }); +}); + +describe("formatMigrationsPushCommand", () => { + test("keeps linked push bare and preserves URL targets", () => { + expect(formatMigrationsPushCommand()).toBe("supabase migrations push"); + expect(formatMigrationsPushCommand({ projectRef: "abcdefghijklmnop" })).toBe( + "supabase migrations push", + ); + expect(formatMigrationsPushCommand({ local: true })).toBe("supabase migrations apply"); + expect(formatMigrationsPushCommand({ dbUrlSame: true })).toBe( + "supabase migrations push --db-url --allow-remote", + ); + expect(formatMigrationsPushCommand({ dbUrlEnvVar: "SUPABASE_DB_URL" })).toBe( + 'supabase migrations push --db-url "$SUPABASE_DB_URL" --allow-remote', + ); + }); +}); + +describe("formatLiveEditCommands", () => { + test("names diff then repair, not pull", () => { + expect(formatLiveEditCommands()).toBe( + [ + "supabase migrations diff --against linked --file supabase/migrations/_.sql", + "supabase migration repair --status applied ", + ].join("\n"), + ); + }); +}); diff --git a/apps/cli/src/shared/migrations/migration-repository.service.ts b/apps/cli/src/shared/migrations/migration-repository.service.ts new file mode 100644 index 0000000000..3f5fda0987 --- /dev/null +++ b/apps/cli/src/shared/migrations/migration-repository.service.ts @@ -0,0 +1,55 @@ +import type { Effect } from "effect"; +import { Context } from "effect"; +import type { SchemaMigrationNameError, SchemaWorkspaceIoError } from "../schema/schema-errors.ts"; +import type { MigrationFile } from "./migration-file.ts"; + +type GeneratedMigrationUnit = { + readonly suffix: string | null; + readonly sql: string; + readonly transactional: boolean; +}; + +export type FetchedMigrationWrite = + | { + readonly outcome: "written"; + readonly file: MigrationFile; + } + | { + readonly outcome: "skipped"; + readonly file: MigrationFile; + } + | { + readonly outcome: "conflict"; + readonly file: MigrationFile; + readonly remoteCopyPath: string; + readonly remoteCopyDisplay: string; + }; + +interface MigrationRepositoryShape { + readonly listLocal: Effect.Effect, SchemaWorkspaceIoError>; + readonly createEmpty: ( + name: string, + content?: string, + ) => Effect.Effect; + readonly writeFetched: (input: { + readonly version: string; + readonly name: string; + readonly sql: string; + }) => Effect.Effect; + readonly writeGenerated: (input: { + readonly name: string; + readonly baseMillis: number; + readonly files: ReadonlyArray; + }) => Effect.Effect< + ReadonlyArray, + SchemaMigrationNameError | SchemaWorkspaceIoError + >; + readonly remove: ( + files: ReadonlyArray, + ) => Effect.Effect; +} + +export class MigrationRepository extends Context.Service< + MigrationRepository, + MigrationRepositoryShape +>()("supabase/migrations/MigrationRepository") {} diff --git a/apps/cli/src/shared/migrations/migration-runner.service.ts b/apps/cli/src/shared/migrations/migration-runner.service.ts new file mode 100644 index 0000000000..c5622ccbbb --- /dev/null +++ b/apps/cli/src/shared/migrations/migration-runner.service.ts @@ -0,0 +1,62 @@ +import type { Effect } from "effect"; +import { Context } from "effect"; +import type { Pool } from "pg"; +import type { Output } from "../output/output.service.ts"; +import type { + SchemaEngineError, + SchemaHistoryConflictError, + SchemaMigrationsPrivilegeError, +} from "../schema/schema-errors.ts"; +import type { MigrationFile } from "./migration-file.ts"; + +export type MigrationHistoryRow = { + readonly version: string; + readonly name: string; +}; + +export type MigrationHistoryStatementsRow = { + readonly version: string; + readonly name: string; + readonly statements: ReadonlyArray; +}; + +export type MigrationApplyResult = { + readonly applied: ReadonlyArray; + readonly skipped: ReadonlyArray; + readonly recorded?: ReadonlyArray; +}; + +interface MigrationRunnerShape { + readonly listRemote: ( + pool: Pool, + ) => Effect.Effect< + ReadonlyArray, + SchemaEngineError | SchemaMigrationsPrivilegeError + >; + readonly listRemoteStatements: ( + pool: Pool, + ) => Effect.Effect< + ReadonlyArray, + SchemaEngineError | SchemaMigrationsPrivilegeError + >; + readonly showServerVersion: (pool: Pool) => Effect.Effect; + readonly listInstalledExtensions: ( + pool: Pool, + ) => Effect.Effect, SchemaEngineError | SchemaMigrationsPrivilegeError>; + readonly applyPending: ( + pool: Pool, + local: ReadonlyArray, + ) => Effect.Effect< + MigrationApplyResult, + SchemaEngineError | SchemaHistoryConflictError | SchemaMigrationsPrivilegeError, + Output + >; + readonly markApplied: ( + pool: Pool, + files: ReadonlyArray, + ) => Effect.Effect; +} + +export class MigrationRunner extends Context.Service()( + "supabase/migrations/MigrationRunner", +) {} diff --git a/apps/cli/src/shared/migrations/new-list-diff-migrations.integration.test.ts b/apps/cli/src/shared/migrations/new-list-diff-migrations.integration.test.ts new file mode 100644 index 0000000000..b244478b3d --- /dev/null +++ b/apps/cli/src/shared/migrations/new-list-diff-migrations.integration.test.ts @@ -0,0 +1,539 @@ +import { describe, expect, it } from "@effect/vitest"; +import { BunServices } from "@effect/platform-bun"; +import { classifyPlanHazards, type Plan } from "@supabase/pg-delta/plan"; +import { Cause, Effect, Exit, Layer, Option } from "effect"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { PgDeltaSchemaEngine } from "../schema/pg-delta-engine.service.ts"; +import { SchemaEmptyHistoryReplayError } from "../schema/schema-errors.ts"; +import { SchemaStateStore } from "../schema/schema-state.service.ts"; +import type { SchemaPlanView } from "../schema/schema-types.ts"; +import { diffMigrations } from "./diff-migrations.ts"; +import { listMigrations } from "./list-migrations.ts"; +import { MigrationRepository } from "./migration-repository.service.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; +import { newMigration } from "./new-migration.ts"; + +const file = { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: "select 1;", + transactional: true, +}; + +function emptyPlan(): Plan { + return { + formatVersion: 1, + engineVersion: "0.3.0", + planId: "plan", + source: { fingerprint: "s" }, + target: { fingerprint: "d" }, + preamble: [], + deltas: [], + filteredDeltas: [], + renameCandidates: [], + actions: [], + safetyReport: { + destructiveActions: 0, + rewriteRiskActions: 0, + nonTransactionalActions: 0, + lockClasses: {}, + }, + }; +} + +function planView(changes: boolean): SchemaPlanView { + const plan = emptyPlan(); + return { + planId: plan.planId, + sourceFingerprint: plan.source.fingerprint, + desiredFingerprint: plan.target.fingerprint, + engineVersion: plan.engineVersion, + profile: "supabase", + changes, + files: changes + ? [ + { + sequence: 1, + suffix: null, + sql: "create table t (id int);", + transactional: true, + actionCount: 1, + }, + ] + : [], + hazards: { + kinds: [], + destructive: 0, + rewrite: 0, + coverageGaps: 0, + report: classifyPlanHazards(plan), + }, + destructive: false, + renameCandidates: [], + acceptedRenames: [], + coverageBlocked: false, + renameBlocked: false, + diagnostics: [], + plan, + }; +} + +const state = Layer.succeed( + SchemaStateStore, + SchemaStateStore.of({ + readJournal: Effect.succeed(Option.none()), + writeJournal: () => Effect.void, + clearJournal: Effect.void, + withLock: (effect) => effect, + }), +); + +const localTargetValue = { + kind: "local" as const, + identity: "local:default", + connectionString: "postgresql://postgres:postgres@127.0.0.1:1/postgres", + disposable: true, + durable: false, + connectionVerified: true, +}; + +const linkedTargetValue = { + kind: "linked" as const, + identity: "abcdefghijklmnop", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: false, + projectRef: "abcdefghijklmnop", +}; + +const urlTargetValue = { + kind: "url" as const, + identity: "connection-string", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: false, +}; + +function targetLayer( + target: typeof localTargetValue | typeof linkedTargetValue | typeof urlTargetValue, +) { + return Layer.succeed( + DatabaseTargetResolver, + DatabaseTargetResolver.of({ + resolve: () => Effect.succeed(target), + }), + ); +} + +const localTarget = targetLayer(localTargetValue); + +describe("newMigration", () => { + it.live("writes an empty migration file", () => { + const created = { ...file, name: "add_billing", fileName: "20260101000000_add_billing.sql" }; + const layer = Layer.mergeAll( + mockOutput({ interactive: false }).layer, + state, + Layer.succeed( + MigrationRepository, + MigrationRepository.of({ + listLocal: Effect.succeed([]), + createEmpty: (_name, content = "") => Effect.succeed({ ...created, content }), + writeFetched: () => Effect.die("unused"), + writeGenerated: () => Effect.die("unused"), + remove: () => Effect.die("unused"), + }), + ), + ); + return Effect.gen(function* () { + const result = yield* newMigration({ name: "add_billing" }).pipe(Effect.provide(layer)); + expect(result.mutatedFiles).toBe(true); + expect(result.data).toEqual( + expect.objectContaining({ file: created.fileName, version: created.version }), + ); + expect(result.nextActions).toEqual(["to add SQL before apply or push: edit the new file"]); + }); + }); + + it.live("seeds the turn-off revoke template and points at push first", () => { + const created = { + ...file, + name: "revoke_api_privileges", + fileName: "20260101000000_revoke_api_privileges.sql", + content: "", + }; + let written = ""; + const layer = Layer.mergeAll( + mockOutput({ interactive: false }).layer, + state, + Layer.succeed( + MigrationRepository, + MigrationRepository.of({ + listLocal: Effect.succeed([]), + createEmpty: (_name, content = "") => + Effect.sync(() => { + written = content; + return { ...created, content }; + }), + writeFetched: () => Effect.die("unused"), + writeGenerated: () => Effect.die("unused"), + remove: () => Effect.die("unused"), + }), + ), + ); + return Effect.gen(function* () { + const result = yield* newMigration({ + name: "revoke_api_privileges", + template: "revoke-api-privileges", + }).pipe(Effect.provide(layer)); + expect(written).toContain("revoke execute on functions"); + expect(result.nextActions).toEqual([ + "to deploy: supabase migrations push", + "to apply it locally: supabase migrations apply", + ]); + written = ""; + yield* newMigration({ name: "revoke_api_privileges" }).pipe(Effect.provide(layer)); + expect(written).toContain("revoke execute on functions"); + }); + }); +}); + +function listLayer(opts: { + readonly files?: ReadonlyArray; + readonly history?: ReadonlyArray<{ version: string; name: string }>; + readonly target?: typeof localTargetValue | typeof linkedTargetValue | typeof urlTargetValue; +}) { + return Layer.mergeAll( + mockOutput({ interactive: false }).layer, + targetLayer(opts.target ?? localTargetValue), + Layer.succeed( + MigrationRepository, + MigrationRepository.of({ + listLocal: Effect.succeed(opts.files ?? []), + createEmpty: () => Effect.die("unused"), + writeFetched: () => Effect.die("unused"), + writeGenerated: () => Effect.die("unused"), + remove: () => Effect.die("unused"), + }), + ), + Layer.succeed( + MigrationRunner, + MigrationRunner.of({ + listRemote: () => Effect.succeed(opts.history ?? []), + listRemoteStatements: () => Effect.succeed([]), + showServerVersion: () => Effect.succeed(undefined), + listInstalledExtensions: () => Effect.die("unused"), + applyPending: () => Effect.die("unused"), + markApplied: () => Effect.die("unused"), + }), + ), + ); +} + +describe("listMigrations", () => { + it.live("lists nothing when both sides are empty", () => { + return Effect.gen(function* () { + const result = yield* listMigrations({ against: "local" }).pipe( + Effect.provide(listLayer({})), + ); + expect(result.status).toBe("clean"); + expect(result.message).toBe("No migrations on the local database."); + expect(result.body).toBeUndefined(); + expect(result.nextActions).toEqual([]); + expect(result.data).toEqual( + expect.objectContaining({ + status: "clean", + applied: 0, + pending: 0, + remote_only: 0, + history: "matched", + migrations: [], + files: [], + }), + ); + }); + }); + + it.live("lists applied files when history matches", () => { + return Effect.gen(function* () { + const result = yield* listMigrations({ against: "local" }).pipe( + Effect.provide( + listLayer({ files: [file], history: [{ version: file.version, name: file.name }] }), + ), + ); + expect(result.status).toBe("clean"); + expect(result.message).toBe( + "1 migration applied on the local database. History matches files.", + ); + expect(result.body).toContain("20260101000000"); + expect(result.body).toContain("applied"); + expect(result.nextActions).toEqual([]); + expect(result.data).toEqual( + expect.objectContaining({ + status: "clean", + applied: 1, + pending: 0, + remote_only: 0, + history: "matched", + migrations: [{ version: file.version, name: file.name, local: true, remote: true }], + files: [{ name: file.name, version: file.version, status: "applied" }], + }), + ); + }); + }); + + it.live("points pending local files at migrations apply", () => { + return Effect.gen(function* () { + const result = yield* listMigrations({ against: "local" }).pipe( + Effect.provide(listLayer({ files: [file] })), + ); + expect(result.status).toBe("clean"); + expect(result.message).toBe("1 of 1 migration pending on the local database."); + expect(result.body).toContain("pending"); + expect(result.nextActions).toEqual(["to apply it locally: supabase migrations apply"]); + expect(result.data).toEqual( + expect.objectContaining({ + applied: 0, + pending: 1, + remote_only: 0, + history: "pending", + }), + ); + }); + }); + + it.live("points pending linked files at migrations push", () => { + return Effect.gen(function* () { + const result = yield* listMigrations({ against: "linked" }).pipe( + Effect.provide(listLayer({ files: [file], target: linkedTargetValue })), + ); + expect(result.message).toBe("1 of 1 migration pending on the linked project."); + expect(result.nextActions).toEqual(["to deploy: supabase migrations push"]); + expect(result.data).toEqual(expect.objectContaining({ history: "pending", pending: 1 })); + }); + }); + + it.live("points pending URL files at push --db-url, not the linked project", () => { + return Effect.gen(function* () { + const result = yield* listMigrations({ + against: "postgresql://postgres:secret@db.example/postgres", + }).pipe(Effect.provide(listLayer({ files: [file], target: urlTargetValue }))); + expect(result.message).toBe("1 of 1 migration pending on the given database."); + expect(result.nextActions).toEqual([ + "to deploy: supabase migrations push --db-url --allow-remote", + ]); + }); + }); + + it.live("points remote-only history at migrations pull and stays exit 0", () => { + return Effect.gen(function* () { + const result = yield* listMigrations({ against: "linked" }).pipe( + Effect.provide( + listLayer({ + history: [{ version: "19990101000000", name: "from_ci" }], + target: linkedTargetValue, + }), + ), + ); + expect(result.status).toBe("clean"); + expect(result.message).toBe("1 remote-only migration on the linked project (no local file)."); + expect(result.body).toContain("remote-only"); + expect(result.nextActions).toEqual([ + "to fetch missing files: supabase migrations pull --from linked", + "to refresh declarations: supabase schema pull --from linked", + ]); + expect(result.data).toEqual( + expect.objectContaining({ + applied: 0, + pending: 0, + remote_only: 1, + history: "remote_only", + }), + ); + }); + }); + + it.live("treats pending plus remote-only as a conflict and still exits 0", () => { + return Effect.gen(function* () { + const result = yield* listMigrations({ against: "linked" }).pipe( + Effect.provide( + listLayer({ + files: [file], + history: [{ version: "19990101000000", name: "from_ci" }], + target: linkedTargetValue, + }), + ), + ); + expect(result.status).toBe("clean"); + expect(result.message).toContain("pending"); + expect(result.message).toContain("remote-only"); + expect(result.message).not.toMatch(/drift/i); + expect(result.nextActions).toEqual([ + "to fetch missing files: supabase migrations pull --from linked", + "to refresh declarations: supabase schema pull --from linked", + ]); + expect(result.data).toEqual( + expect.objectContaining({ + applied: 0, + pending: 1, + remote_only: 1, + history: "conflict", + }), + ); + }); + }); +}); + +function diffLayer( + opts: { + readonly files?: ReadonlyArray; + readonly history?: ReadonlyArray<{ version: string; name: string }>; + } = {}, +) { + const files = opts.files ?? [file]; + const history = opts.history ?? files.map((item) => ({ version: item.version, name: item.name })); + const replayed: Array> = []; + let platformProvisions = 0; + return { + replayed, + get platformProvisions() { + return platformProvisions; + }, + layer: Layer.mergeAll( + mockOutput({ interactive: false }).layer, + localTarget, + Layer.succeed( + MigrationRepository, + MigrationRepository.of({ + listLocal: Effect.succeed(files), + createEmpty: () => Effect.die("unused"), + writeFetched: () => Effect.die("unused"), + writeGenerated: () => Effect.die("unused"), + remove: () => Effect.die("unused"), + }), + ), + Layer.succeed( + MigrationRunner, + MigrationRunner.of({ + listRemote: () => Effect.succeed(history), + listRemoteStatements: () => Effect.succeed([]), + showServerVersion: () => Effect.succeed(undefined), + listInstalledExtensions: () => Effect.die("unused"), + applyPending: (_pool, applied) => + Effect.sync(() => { + replayed.push(applied.map((item) => item.version)); + return { applied: applied.map((item) => item.version), skipped: [] }; + }), + markApplied: () => Effect.die("unused"), + }), + ), + Layer.succeed( + PgDeltaSchemaEngine, + PgDeltaSchemaEngine.of({ + exportSchema: () => Effect.die("unused"), + planFiles: () => Effect.die("unused"), + diffPools: () => Effect.succeed(planView(true)), + applyPlan: () => Effect.die("unused"), + provisionShadow: Effect.die("unused"), + provisionPlatform: Effect.sync(() => { + platformProvisions += 1; + return { url: "postgresql://postgres:postgres@127.0.0.1:1/postgres" }; + }), + provisionMigrations: Effect.die("provisionMigrations must not replay pending files"), + }), + ), + ), + }; +} + +describe("diffMigrations", () => { + it.live("previews drift against the named target", () => { + const ctx = diffLayer(); + return Effect.gen(function* () { + const result = yield* diffMigrations({ against: "local" }).pipe( + Effect.provide(ctx.layer), + Effect.provide(BunServices.layer), + ); + expect(result.status).toBe("drift"); + expect(result.body).toBe("create table t (id int);"); + expect(result.data["sql"]).toBe("create table t (id int);"); + expect(result.data["files"]).toEqual([ + expect.objectContaining({ sql: "create table t (id int);" }), + ]); + expect(result.mutatedDatabase).toBe(false); + }); + }); + + it.live("defaults --against to local and does not send the next step to linked pull", () => { + const ctx = diffLayer(); + return Effect.gen(function* () { + const result = yield* diffMigrations({}).pipe( + Effect.provide(ctx.layer), + Effect.provide(BunServices.layer), + ); + expect(result.nextActions).toEqual([ + "to write a migration file: supabase migrations diff --against local --file supabase/migrations/_.sql", + "to record it as applied without running SQL: supabase migration repair --local --status applied ", + ]); + }); + }); + + it.live("replays only applied history, not pending local files", () => { + const pending = { + ...file, + version: "20260201000000", + name: "billing", + fileName: "20260201000000_billing.sql", + absolutePath: "/tmp/migrations/20260201000000_billing.sql", + }; + const ctx = diffLayer({ + files: [file, pending], + history: [{ version: file.version, name: file.name }], + }); + return Effect.gen(function* () { + yield* diffMigrations({ against: "linked" }).pipe( + Effect.provide(ctx.layer), + Effect.provide(BunServices.layer), + ); + expect(ctx.replayed).toEqual([[file.version]]); + }); + }); + + it.live("refuses empty history when local files exist, before shadow", () => { + const ctx = diffLayer({ files: [file], history: [] }); + return Effect.gen(function* () { + const exit = yield* diffMigrations({ against: "local" }).pipe( + Effect.provide(ctx.layer), + Effect.provide(BunServices.layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaEmptyHistoryReplayError); + expect(JSON.stringify(exit)).toContain("supabase migrations apply"); + expect(JSON.stringify(exit)).not.toContain("db diff"); + } + expect(ctx.platformProvisions).toBe(0); + expect(ctx.replayed).toEqual([]); + }); + }); + + it.live("captures adopt when history and local files are empty", () => { + const ctx = diffLayer({ files: [], history: [] }); + return Effect.gen(function* () { + const result = yield* diffMigrations({ against: "local" }).pipe( + Effect.provide(ctx.layer), + Effect.provide(BunServices.layer), + ); + expect(result.status).toBe("drift"); + expect(result.data["sql"]).toBe("create table t (id int);"); + expect(ctx.platformProvisions).toBe(1); + }); + }); +}); diff --git a/apps/cli/src/shared/migrations/new-migration.ts b/apps/cli/src/shared/migrations/new-migration.ts new file mode 100644 index 0000000000..a1efc42175 --- /dev/null +++ b/apps/cli/src/shared/migrations/new-migration.ts @@ -0,0 +1,69 @@ +import { Effect } from "effect"; +import { SchemaDraftConflictError, SchemaMigrationNameError } from "../schema/schema-errors.ts"; +import { formatMigrationFilePath, formatNextAction } from "../schema/schema-output.ts"; +import type { SchemaCommandResult } from "../schema/schema-types.ts"; +import { SchemaStateStore } from "../schema/schema-state.service.ts"; +import { MigrationRepository } from "./migration-repository.service.ts"; +import { + REVOKE_API_PRIVILEGES_NAME, + REVOKE_API_PRIVILEGES_TEMPLATE, + revokeApiPrivilegesTemplateSql, +} from "./privilege-offer.ts"; + +export type NewMigrationInput = { + readonly name?: string; + readonly template?: typeof REVOKE_API_PRIVILEGES_TEMPLATE; +}; + +export const newMigration = Effect.fn("migrations.new")(function* (input: NewMigrationInput) { + const name = + input.name !== undefined && input.name.trim() !== "" + ? input.name.trim() + : input.template === REVOKE_API_PRIVILEGES_TEMPLATE + ? REVOKE_API_PRIVILEGES_NAME + : undefined; + if (name === undefined) { + return yield* new SchemaMigrationNameError({ + detail: "Migration name is required.", + suggestion: "Pass a name, for example `supabase migrations new add_billing`.", + }); + } + const repository = yield* MigrationRepository; + const state = yield* SchemaStateStore; + const journal = yield* state.readJournal; + if ( + journal._tag === "Some" && + journal.value.declarativelyAhead && + journal.value.generated !== true + ) { + return yield* new SchemaDraftConflictError({ + detail: "Migration files cannot change while a declarative draft is active.", + suggestion: "Run `supabase schema generate`, reset the local database, or discard the draft.", + }); + } + const seed = + input.template === REVOKE_API_PRIVILEGES_TEMPLATE || name === REVOKE_API_PRIVILEGES_NAME + ? revokeApiPrivilegesTemplateSql() + : ""; + const created = yield* repository.createEmpty(name, seed); + return { + status: "generated", + message: `Created ${formatMigrationFilePath(created.fileName)}`, + data: { + status: "generated", + file: created.fileName, + version: created.version, + mutated_files: true, + mutated_database: false, + }, + nextActions: + seed.length > 0 + ? [ + formatNextAction("to deploy", "supabase migrations push"), + formatNextAction("to apply it locally", "supabase migrations apply"), + ] + : [formatNextAction("to add SQL before apply or push", "edit the new file")], + mutatedDatabase: false, + mutatedFiles: true, + } satisfies SchemaCommandResult; +}); diff --git a/apps/cli/src/shared/migrations/privilege-offer.ts b/apps/cli/src/shared/migrations/privilege-offer.ts new file mode 100644 index 0000000000..816bab0ba8 --- /dev/null +++ b/apps/cli/src/shared/migrations/privilege-offer.ts @@ -0,0 +1,191 @@ +import { Effect } from "effect"; +import { analyzeAndSort } from "@supabase/pg-topo"; +import { + SchemaEmptyMigrationStatementsError, + SchemaPrivilegeOfferError, +} from "../schema/schema-errors.ts"; +import { formatPlanSql } from "../schema/schema-body.ts"; +import type { SchemaScriptFile } from "../schema/schema-body.ts"; +import { + formatMigrationsPushCommand, + type MigrationRepairFlags, +} from "./migration-repair-suggest.ts"; + +/** + * Turn-off default privileges plus leftover hosted object grants. + * Must execute on the remote — do not repair --applied. + */ +export const REVOKE_API_PRIVILEGES_SQL = ` +alter default privileges for role postgres in schema public + revoke select, insert, update, delete on tables from anon, authenticated, service_role; +alter default privileges for role postgres in schema public + revoke usage, select on sequences from anon, authenticated, service_role; +alter default privileges for role postgres in schema public + revoke execute on functions from anon, authenticated, service_role; +revoke all on all tables in schema public from anon, authenticated, service_role; +revoke all on all sequences in schema public from anon, authenticated, service_role; +revoke execute on all functions in schema public from anon, authenticated, service_role; +`; + +export const PRIVILEGE_REFRESH_SUGGESTION = + "Run `supabase db reset` then `supabase schema pull --force` so declarations match. Do not write GRANT ALL or repair."; + +export const REVOKE_API_PRIVILEGES_NAME = "revoke_api_privileges"; +export const REVOKE_API_PRIVILEGES_TEMPLATE = "revoke-api-privileges"; + +export function revokeApiPrivilegesTemplateSql(): string { + return `${REVOKE_API_PRIVILEGES_SQL.trim()}\n`; +} + +const API_ROLES = new Set(["anon", "authenticated", "service_role"]); +const IDENT = String.raw`(?:"([^"]+)"|([A-Za-z_][\w$]*))`; +const ACL_STATEMENT = new RegExp( + String.raw`^ALTER\s+DEFAULT\s+PRIVILEGES\s+FOR\s+ROLE\s+${IDENT}\s+IN\s+SCHEMA\s+${IDENT}\s+(?:GRANT|REVOKE)\s+.+\s+(?:TO|FROM)\s+(.+)$`, + "iu", +); +const GRANT_WORD = /\bGRANT\b/iu; + +export type PrivilegeSqlKind = "grant_present" | "revoke_only" | "not_acl"; + +function identValue(quoted: string | undefined, bare: string | undefined): string { + return (quoted ?? bare ?? "").toLowerCase(); +} + +function targetRoles(list: string): ReadonlyArray { + return list + .split(",") + .map((role) => role.trim().replaceAll('"', "").toLowerCase()) + .filter((role) => role.length > 0); +} + +function normalizeAclStatement(sql: string): string { + // pg-topo attaches a leading `--` header to the first statement's sql. + const stripped = sql.replace(/\/\*[\s\S]*?\*\//gu, "").replace(/--[^\n]*/gu, ""); + const start = stripped.search(/(?:ALTER\s+DEFAULT\s+PRIVILEGES|GRANT\b|REVOKE\b)/iu); + const body = start === -1 ? stripped : stripped.slice(start); + return body.replace(/\s+/gu, " ").trim().replace(/;$/u, ""); +} + +export function isPublicDefaultAclStatement(statement: string): boolean { + const match = ACL_STATEMENT.exec(normalizeAclStatement(statement)); + if (match === null) return false; + if (identValue(match[1], match[2]) !== "postgres") return false; + if (identValue(match[3], match[4]) !== "public") return false; + const roles = targetRoles(match[5] ?? ""); + return roles.length > 0 && roles.every((role) => API_ROLES.has(role)); +} + +export function isPublicObjectAclStatement(statement: string): boolean { + const body = normalizeAclStatement(statement); + const match = + /^(?:GRANT|REVOKE)\s+.+\s+ON\s+(?:ALL\s+)?(FUNCTIONS?|TABLES?|SEQUENCES?)\b(.+)$/iu.exec(body); + if (match === null) return false; + const rest = match[2] ?? ""; + const inPublic = + /\bIN\s+SCHEMA\s+(?:"public"|public)\b/iu.test(rest) || /(?:"public"|public)\./u.test(rest); + if (!inPublic) return false; + const rolesMatch = /\s+(?:TO|FROM)\s+(.+)$/iu.exec(rest); + if (rolesMatch === null) return false; + const roles = targetRoles(rolesMatch[1] ?? ""); + return roles.length > 0 && roles.every((role) => API_ROLES.has(role)); +} + +function isPrivilegeStatement(statementClass: string, sql: string): boolean { + return ( + (statementClass === "ALTER_DEFAULT_PRIVILEGES" && isPublicDefaultAclStatement(sql)) || + isPublicObjectAclStatement(sql) + ); +} + +function classifyPrivilegeStatements( + statements: ReadonlyArray<{ readonly statementClass: string; readonly sql: string }>, +): PrivilegeSqlKind { + if (statements.length === 0) return "not_acl"; + if (!statements.every((node) => isPrivilegeStatement(node.statementClass, node.sql))) { + return "not_acl"; + } + return statements.some((node) => GRANT_WORD.test(normalizeAclStatement(node.sql))) + ? "grant_present" + : "revoke_only"; +} + +function isCommentOnly(sql: string): boolean { + return ( + sql + .replace(/\/\*[\s\S]*?\*\//gu, "") + .replace(/--[^\n]*/gu, "") + .trim().length === 0 + ); +} + +export function migrationHasExecutableSql(content: string): boolean { + return !isCommentOnly(content); +} + +export function emptyPendingMigrationError( + pending: ReadonlyArray<{ readonly fileName: string; readonly content: string }>, +): SchemaEmptyMigrationStatementsError | undefined { + const empty = pending.find((file) => !migrationHasExecutableSql(file.content)); + if (empty === undefined) return undefined; + return new SchemaEmptyMigrationStatementsError({ + detail: `${empty.fileName} has no executable SQL.`, + suggestion: "Put SQL in the file or delete it. Empty migrations cannot be applied or pushed.", + }); +} + +function sqlStatements( + sql: string, +): ReadonlyArray<{ readonly statementClass: string; readonly sql: string }> { + return sql + .split(";") + .map((statement) => statement.trim()) + .filter((statement) => statement.length > 0 && !isCommentOnly(statement)) + .map((statement) => ({ statementClass: "", sql: statement })); +} + +export const classifyPrivilegeSql = (sql: string): Effect.Effect => + Effect.tryPromise({ + try: () => analyzeAndSort([sql]), + catch: () => new Error("privilege-sql-parse"), + }).pipe( + Effect.map((result): PrivilegeSqlKind => { + if (result.diagnostics.some((diagnostic) => diagnostic.code === "PARSE_ERROR")) { + return classifyPrivilegeStatements(sqlStatements(sql)); + } + return classifyPrivilegeStatements(result.ordered.filter((node) => !isCommentOnly(node.sql))); + }), + Effect.orElseSucceed((): PrivilegeSqlKind => "not_acl"), + ); + +export const classifyPrivilegePlan = (plan: { + readonly files: ReadonlyArray<{ readonly sql: string }>; +}): Effect.Effect => classifyPrivilegeSql(formatPlanSql(plan)); + +export const pendingHasPrivilegeSql = ( + pending: ReadonlyArray<{ readonly content: string }>, +): Effect.Effect => + Effect.gen(function* () { + for (const file of pending) { + const kind = yield* classifyPrivilegeSql(file.content); + if (kind !== "not_acl") return true; + } + return false; + }); + +export function privilegeOfferError( + sql: string, + flags?: MigrationRepairFlags, + files?: ReadonlyArray, +): SchemaPrivilegeOfferError { + const push = formatMigrationsPushCommand(flags); + return new SchemaPrivilegeOfferError({ + detail: + "Remote default privileges differ from migration replay. Recommended off for least privilege.", + suggestion: [ + `Turn off: supabase migrations new ${REVOKE_API_PRIVILEGES_NAME} --template ${REVOKE_API_PRIVILEGES_TEMPLATE}, then ${push} (must execute).`, + `Keep on: api.auto_expose_new_tables is deprecated and will be removed on 2026-10-30. If you still want it, set api.auto_expose_new_tables = true in supabase/config.toml, then supabase db reset && supabase schema pull --force so declarations match the grant-kept baseline, then re-run ${push}. Do not write GRANT ALL or repair.`, + ].join("\n"), + sql, + ...(files !== undefined ? { files } : {}), + }); +} diff --git a/apps/cli/src/shared/migrations/privilege-offer.unit.test.ts b/apps/cli/src/shared/migrations/privilege-offer.unit.test.ts new file mode 100644 index 0000000000..96a8d8976b --- /dev/null +++ b/apps/cli/src/shared/migrations/privilege-offer.unit.test.ts @@ -0,0 +1,183 @@ +import { describe, expect, it } from "@effect/vitest"; +import { Effect } from "effect"; +import { + classifyPrivilegePlan, + classifyPrivilegeSql, + emptyPendingMigrationError, + isPublicDefaultAclStatement, + isPublicObjectAclStatement, + migrationHasExecutableSql, + pendingHasPrivilegeSql, + privilegeOfferError, + REVOKE_API_PRIVILEGES_SQL, +} from "./privilege-offer.ts"; + +const PLATFORM_VS_STAGING_SQL = ` +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" REVOKE ALL ON SEQUENCES FROM "anon"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" REVOKE ALL ON SEQUENCES FROM "authenticated"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" REVOKE ALL ON SEQUENCES FROM "service_role"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" REVOKE ALL ON TABLES FROM "anon"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" REVOKE ALL ON TABLES FROM "authenticated"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" REVOKE ALL ON TABLES FROM "service_role"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT SELECT, UPDATE, USAGE ON SEQUENCES TO "anon"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT SELECT, UPDATE, USAGE ON SEQUENCES TO "authenticated"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT SELECT, UPDATE, USAGE ON SEQUENCES TO "service_role"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT EXECUTE ON FUNCTIONS TO "anon"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT EXECUTE ON FUNCTIONS TO "authenticated"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT EXECUTE ON FUNCTIONS TO "service_role"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT DELETE, INSERT, MAINTAIN, REFERENCES, SELECT, TRIGGER, TRUNCATE, UPDATE ON TABLES TO "anon"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT DELETE, INSERT, MAINTAIN, REFERENCES, SELECT, TRIGGER, TRUNCATE, UPDATE ON TABLES TO "authenticated"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT DELETE, INSERT, MAINTAIN, REFERENCES, SELECT, TRIGGER, TRUNCATE, UPDATE ON TABLES TO "service_role"; +`; + +describe("isPublicDefaultAclStatement", () => { + it("accepts postgres/public grants and revokes to Data API roles", () => { + expect( + isPublicDefaultAclStatement( + 'ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" REVOKE ALL ON TABLES FROM "anon"', + ), + ).toBe(true); + expect( + isPublicDefaultAclStatement( + "ALTER DEFAULT PRIVILEGES FOR ROLE postgres IN SCHEMA public GRANT SELECT ON TABLES TO authenticated, service_role", + ), + ).toBe(true); + }); + + it("rejects other roles, schemas, or DDL", () => { + expect( + isPublicDefaultAclStatement( + 'ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "auth" GRANT SELECT ON TABLES TO "anon"', + ), + ).toBe(false); + expect( + isPublicDefaultAclStatement( + 'ALTER DEFAULT PRIVILEGES FOR ROLE "supabase_admin" IN SCHEMA "public" GRANT SELECT ON TABLES TO "anon"', + ), + ).toBe(false); + expect( + isPublicDefaultAclStatement( + 'ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT SELECT ON TABLES TO "PUBLIC"', + ), + ).toBe(false); + expect(isPublicDefaultAclStatement("CREATE TABLE public.t (id int)")).toBe(false); + }); +}); + +describe("isPublicObjectAclStatement", () => { + it("accepts public function grants to Data API roles", () => { + expect( + isPublicObjectAclStatement( + "GRANT EXECUTE ON FUNCTION public.accept_invitation() TO service_role", + ), + ).toBe(true); + expect( + isPublicObjectAclStatement( + "REVOKE EXECUTE ON ALL FUNCTIONS IN SCHEMA public FROM anon, authenticated, service_role", + ), + ).toBe(true); + }); + + it("rejects other schemas or roles", () => { + expect(isPublicObjectAclStatement("GRANT EXECUTE ON FUNCTION auth.uid() TO service_role")).toBe( + false, + ); + expect(isPublicObjectAclStatement("GRANT EXECUTE ON FUNCTION public.foo() TO postgres")).toBe( + false, + ); + }); +}); + +describe("classifyPrivilegeSql", () => { + it.live("treats the hosted vs isolated dump as grant_present", () => + Effect.gen(function* () { + expect(yield* classifyPrivilegeSql(PLATFORM_VS_STAGING_SQL)).toBe("grant_present"); + expect( + yield* classifyPrivilegePlan({ + files: [{ sql: PLATFORM_VS_STAGING_SQL }], + }), + ).toBe("grant_present"); + }), + ); + + it.live("treats the turn-off revoke SQL as revoke_only", () => + Effect.gen(function* () { + expect(yield* classifyPrivilegeSql(REVOKE_API_PRIVILEGES_SQL)).toBe("revoke_only"); + }), + ); + + it.live("treats leftover function grants as grant_present", () => + Effect.gen(function* () { + expect( + yield* classifyPrivilegeSql( + "GRANT EXECUTE ON FUNCTION public.accept_invitation() TO service_role;", + ), + ).toBe("grant_present"); + }), + ); + + it.live("rejects empty or mixed DDL", () => + Effect.gen(function* () { + expect(yield* classifyPrivilegeSql("")).toBe("not_acl"); + expect( + yield* classifyPrivilegeSql(`${PLATFORM_VS_STAGING_SQL}\nCREATE TABLE t (id int);`), + ).toBe("not_acl"); + }), + ); +}); + +describe("migrationHasExecutableSql", () => { + it("treats whitespace and comments as empty", () => { + expect(migrationHasExecutableSql("")).toBe(false); + expect(migrationHasExecutableSql("-- empty stub\n")).toBe(false); + expect(migrationHasExecutableSql("select 1;")).toBe(true); + expect( + emptyPendingMigrationError([{ fileName: "20260101000000_sneak.sql", content: "" }])?._tag, + ).toBe("SchemaEmptyMigrationStatementsError"); + expect( + emptyPendingMigrationError([{ fileName: "ok.sql", content: "select 1;" }]), + ).toBeUndefined(); + }); +}); + +describe("pendingHasPrivilegeSql", () => { + it.live("detects a pending revoke file, including a comment header", () => + Effect.gen(function* () { + expect(yield* pendingHasPrivilegeSql([{ content: "select 1;" }])).toBe(false); + expect(yield* pendingHasPrivilegeSql([{ content: REVOKE_API_PRIVILEGES_SQL }])).toBe(true); + expect( + yield* pendingHasPrivilegeSql([ + { content: `-- write revoke SQL\n${REVOKE_API_PRIVILEGES_SQL}` }, + ]), + ).toBe(true); + expect(yield* pendingHasPrivilegeSql([{ content: "-- empty stub\n" }])).toBe(false); + }), + ); +}); + +describe("privilegeOfferError", () => { + it("keeps a URL next action on the selected database", () => { + const error = privilegeOfferError("ALTER DEFAULT PRIVILEGES", { dbUrlSame: true }); + expect(error.suggestion).toContain( + "supabase migrations push --db-url --allow-remote", + ); + expect(error.suggestion).not.toContain("then supabase migrations push\n"); + }); + + it("recommends turn-off first and keep-on as refresh declarations", () => { + const error = privilegeOfferError("ALTER DEFAULT PRIVILEGES"); + expect(error.suggestion.indexOf("Turn off:")).toBeLessThan( + error.suggestion.indexOf("Keep on:"), + ); + expect(error.suggestion).toContain( + "migrations new revoke_api_privileges --template revoke-api-privileges", + ); + expect(error.suggestion).not.toContain("write the revoke SQL"); + expect(error.suggestion).not.toContain("revoke execute on functions"); + expect(error.suggestion).toContain("db reset"); + expect(error.suggestion).toContain("schema pull --force"); + expect(error.suggestion).toContain("deprecated"); + expect(error.suggestion).toContain("Do not write GRANT ALL"); + expect(error.suggestion).not.toContain("schema generate --name"); + }); +}); diff --git a/apps/cli/src/shared/migrations/pull-migrations.integration.test.ts b/apps/cli/src/shared/migrations/pull-migrations.integration.test.ts new file mode 100644 index 0000000000..e30e81b7ef --- /dev/null +++ b/apps/cli/src/shared/migrations/pull-migrations.integration.test.ts @@ -0,0 +1,303 @@ +import { describe, expect, it } from "@effect/vitest"; +import { Cause, Effect, Exit, Layer } from "effect"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import type { DatabaseTarget } from "../database/database-target.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { SchemaEmptyMigrationStatementsError } from "../schema/schema-errors.ts"; +import type { MigrationFile } from "./migration-file.ts"; +import { formatFetchedMigrationSql, pullMigrations } from "./pull-migrations.ts"; +import { MigrationRepository, type FetchedMigrationWrite } from "./migration-repository.service.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; + +const linkedTarget = { + kind: "linked" as const, + identity: "abcdefghijklmnop", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: true, + projectRef: "abcdefghijklmnop", +} satisfies DatabaseTarget; + +const urlTarget = { + kind: "url" as const, + identity: "connection-string", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: false, +} satisfies DatabaseTarget; + +const pendingFile = { + version: "20260201000000", + name: "billing", + fileName: "20260201000000_billing.sql", + absolutePath: "/tmp/migrations/20260201000000_billing.sql", + content: "select 2;", + transactional: true, +} satisfies MigrationFile; + +const initFile = { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: formatFetchedMigrationSql(["create table t (id int)"]), + transactional: true, +} satisfies MigrationFile; + +function setup( + opts: { + remote?: ReadonlyArray<{ + version: string; + name: string; + statements: ReadonlyArray; + }>; + local?: ReadonlyArray; + target?: DatabaseTarget; + } = {}, +) { + const out = mockOutput({ interactive: false }); + const local = [...(opts.local ?? [])]; + const writes: Array = []; + return { + writes, + layer: Layer.mergeAll( + out.layer, + Layer.succeed( + DatabaseTargetResolver, + DatabaseTargetResolver.of({ + resolve: () => Effect.succeed(opts.target ?? linkedTarget), + }), + ), + Layer.succeed( + MigrationRepository, + MigrationRepository.of({ + listLocal: Effect.sync(() => local), + createEmpty: () => Effect.die("unused"), + writeFetched: (input) => + Effect.sync(() => { + const existing = local.find((file) => file.version === input.version); + const file: MigrationFile = { + version: input.version, + name: input.name, + fileName: `${input.version}_${input.name}.sql`, + absolutePath: `/tmp/migrations/${input.version}_${input.name}.sql`, + content: existing?.content ?? input.sql, + transactional: true, + }; + if (existing === undefined) { + local.push({ ...file, content: input.sql }); + const written = { + outcome: "written" as const, + file: { ...file, content: input.sql }, + }; + writes.push(written); + return written; + } + if (existing.content === input.sql) { + const skipped = { outcome: "skipped" as const, file: existing }; + writes.push(skipped); + return skipped; + } + const conflict = { + outcome: "conflict" as const, + file: existing, + remoteCopyPath: `/tmp/.supabase/remote-migrations/${file.fileName}`, + remoteCopyDisplay: `.supabase/remote-migrations/${file.fileName}`, + }; + writes.push(conflict); + return conflict; + }), + writeGenerated: () => Effect.die("provisionMigrations/writeGenerated must not run"), + remove: () => Effect.die("unused"), + }), + ), + Layer.succeed( + MigrationRunner, + MigrationRunner.of({ + listRemote: () => Effect.die("unused"), + listRemoteStatements: () => Effect.succeed(opts.remote ?? []), + showServerVersion: () => Effect.succeed(undefined), + listInstalledExtensions: () => Effect.die("unused"), + applyPending: () => Effect.die("unused"), + markApplied: () => Effect.die("unused"), + }), + ), + ), + }; +} + +describe("pullMigrations", () => { + it.live("writes nothing when remote history is empty", () => { + const ctx = setup({ remote: [], local: [initFile] }); + return Effect.gen(function* () { + const result = yield* pullMigrations({}).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedFiles).toBe(false); + expect(result.message).toBe("Nothing to fetch."); + expect(result.data).toEqual(expect.objectContaining({ files: [] })); + expect(ctx.writes).toEqual([]); + }); + }); + + it.live("writes a missing file at the remote version", () => { + const ctx = setup({ + remote: [ + { version: "20260101000000", name: "init", statements: ["create table t (id int)"] }, + ], + }); + return Effect.gen(function* () { + const result = yield* pullMigrations({}).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedFiles).toBe(true); + expect(result.data).toEqual( + expect.objectContaining({ + files: [ + { name: "20260101000000_init.sql", version: "20260101000000", status: "fetched" }, + ], + }), + ); + expect(ctx.writes).toEqual([ + expect.objectContaining({ + outcome: "written", + file: expect.objectContaining({ + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + }), + }), + ]); + }); + }); + + it.live("skips an identical local file", () => { + const ctx = setup({ + local: [initFile], + remote: [ + { version: initFile.version, name: initFile.name, statements: ["create table t (id int)"] }, + ], + }); + return Effect.gen(function* () { + const result = yield* pullMigrations({}).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedFiles).toBe(false); + expect(ctx.writes[0]?.outcome).toBe("skipped"); + }); + }); + + it.live("writes a side file when SQL differs", () => { + const ctx = setup({ + local: [initFile], + remote: [ + { + version: initFile.version, + name: initFile.name, + statements: ["create table t (id int, n int)"], + }, + ], + }); + return Effect.gen(function* () { + const result = yield* pullMigrations({}).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedFiles).toBe(true); + expect(ctx.writes[0]).toEqual( + expect.objectContaining({ + outcome: "conflict", + remoteCopyDisplay: ".supabase/remote-migrations/20260101000000_init.sql", + }), + ); + expect(result.message).toContain("statements[] join can differ in formatting"); + }); + }); + + it.live("keeps empty-statements recovery on the selected URL", () => { + const ctx = setup({ + target: urlTarget, + remote: [{ version: "20260101000000", name: "init", statements: [] }], + }); + return Effect.gen(function* () { + const exit = yield* pullMigrations({ + from: "postgresql://postgres:secret@db.example/postgres", + }).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value.suggestion).toContain("migrations diff --against "); + expect(failure.value.suggestion).toContain( + "migration repair --db-url --status applied 20260101000000", + ); + } + }); + }); + + it.live("fails named when statements are empty", () => { + const ctx = setup({ + remote: [{ version: "20260101000000", name: "init", statements: [] }], + }); + return Effect.gen(function* () { + const exit = yield* pullMigrations({}).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaEmptyMigrationStatementsError); + expect(failure.value.suggestion).toContain("migrations diff --against linked"); + expect(failure.value.suggestion).toContain( + "migration repair --project-ref abcdefghijklmnop --status applied 20260101000000", + ); + } + expect(ctx.writes).toEqual([]); + }); + }); + + it.live("skips an existing local file when remote statements are empty", () => { + const ctx = setup({ + local: [initFile], + remote: [ + { version: initFile.version, name: initFile.name, statements: [] }, + { version: "20260102000000", name: "users", statements: ["create table u (id int)"] }, + ], + }); + return Effect.gen(function* () { + const result = yield* pullMigrations({}).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedFiles).toBe(true); + expect(ctx.writes.map((write) => write.outcome)).toEqual(["written"]); + expect(result.data).toEqual( + expect.objectContaining({ + skipped: [initFile.fileName], + fetched: ["20260102000000_users.sql"], + }), + ); + }); + }); + + it.live("fails empty remote-only statements before writing earlier rows", () => { + const ctx = setup({ + remote: [ + { version: "20260101000000", name: "init", statements: ["create table t (id int)"] }, + { version: "20260102000000", name: "legacy", statements: [] }, + ], + }); + return Effect.gen(function* () { + const exit = yield* pullMigrations({}).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(ctx.writes).toEqual([]); + }); + }); + + it.live("points leftover pending files at the selected URL", () => { + const ctx = setup({ + target: urlTarget, + local: [pendingFile], + remote: [ + { version: "20260101000000", name: "init", statements: ["create table t (id int)"] }, + ], + }); + return Effect.gen(function* () { + const result = yield* pullMigrations({ + from: "postgresql://postgres:secret@db.example/postgres", + }).pipe(Effect.provide(ctx.layer)); + expect(result.nextActions).toEqual([ + "to deploy your pending files: supabase migrations push --db-url --allow-remote", + ]); + }); + }); +}); diff --git a/apps/cli/src/shared/migrations/pull-migrations.ts b/apps/cli/src/shared/migrations/pull-migrations.ts new file mode 100644 index 0000000000..6ea523ba43 --- /dev/null +++ b/apps/cli/src/shared/migrations/pull-migrations.ts @@ -0,0 +1,163 @@ +import { Effect } from "effect"; +import { acquireDatabasePool } from "../database/database-pool.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { parseTargetSelector } from "../database/database-target.ts"; +import { formatNextAction } from "../schema/schema-output.ts"; +import type { SchemaScriptFile } from "../schema/schema-body.ts"; +import { SchemaEmptyMigrationStatementsError } from "../schema/schema-errors.ts"; +import type { SchemaCommandResult } from "../schema/schema-types.ts"; +import { + formatMigrationRepairCommand, + formatMigrationsDiffFileCommand, + formatMigrationsPushCommand, + repairFlagsForTarget, +} from "./migration-repair-suggest.ts"; +import { MigrationRepository } from "./migration-repository.service.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; + +export type PullMigrationsInput = { + readonly from?: string; +}; + +export function formatFetchedMigrationSql(statements: ReadonlyArray): string { + return `${statements.join(";\n")};\n`; +} + +export const pullMigrations = Effect.fn("migrations.pull")(function* (input: PullMigrationsInput) { + const targets = yield* DatabaseTargetResolver; + const repository = yield* MigrationRepository; + const runner = yield* MigrationRunner; + const remote = yield* targets.resolve(parseTargetSelector(input.from ?? "linked")); + const flags = repairFlagsForTarget(remote); + + return yield* Effect.scoped( + Effect.gen(function* () { + const remotePool = yield* acquireDatabasePool(remote.connectionString); + const history = yield* runner.listRemoteStatements(remotePool); + if (history.length === 0) { + return { + status: "clean", + message: "Nothing to fetch.", + data: { + status: "clean", + fetched: [], + skipped: [], + conflicts: [], + files: [], + mutated_files: false, + mutated_database: false, + }, + nextActions: [], + mutatedDatabase: false, + mutatedFiles: false, + } satisfies SchemaCommandResult; + } + + const fetched: Array = []; + const skipped: Array = []; + const conflicts: Array<{ readonly version: string; readonly remoteCopy: string }> = []; + const files: Array = []; + const messages: Array = []; + const localBefore = yield* repository.listLocal; + const localByVersion = new Map(localBefore.map((file) => [file.version, file])); + const emptyRemoteOnly = history.filter( + (row) => row.statements.length === 0 && !localByVersion.has(row.version), + ); + if (emptyRemoteOnly[0] !== undefined) { + const row = emptyRemoteOnly[0]; + return yield* new SchemaEmptyMigrationStatementsError({ + detail: `Remote history row ${row.version} (${row.name}) has no statements.`, + suggestion: `${formatMigrationsDiffFileCommand(flags)} then ${formatMigrationRepairCommand( + { + status: "applied", + versions: [row.version], + flags, + }, + )}. Or restore the file from git.`, + }); + } + + for (const row of history) { + if (row.statements.length === 0) { + const existing = localByVersion.get(row.version); + if (existing !== undefined) { + skipped.push(existing.fileName); + files.push({ + name: existing.fileName, + version: row.version, + status: "skipped", + }); + } + continue; + } + const sql = formatFetchedMigrationSql(row.statements); + const result = yield* repository.writeFetched({ + version: row.version, + name: row.name, + sql, + }); + if (result.outcome === "written") { + fetched.push(result.file.fileName); + files.push({ name: result.file.fileName, version: row.version, status: "fetched" }); + } else if (result.outcome === "skipped") { + skipped.push(result.file.fileName); + files.push({ name: result.file.fileName, version: row.version, status: "skipped" }); + } else { + conflicts.push({ + version: row.version, + remoteCopy: result.remoteCopyDisplay, + }); + files.push({ name: result.file.fileName, version: row.version, status: "conflict" }); + messages.push( + `Left local ${result.file.fileName}; wrote remote bytes to ${result.remoteCopyDisplay}. statements[] join can differ in formatting.`, + ); + } + } + + const local = yield* repository.listLocal; + const remoteVersions = new Set(history.map((row) => row.version)); + const pending = local.filter((file) => !remoteVersions.has(file.version)); + if (pending.length > 0) { + messages.push( + `Local pending remain (${pending.map((file) => file.version).join(", ")}). Pull does not merge them.`, + ); + } + + const mutatedFiles = fetched.length > 0 || conflicts.length > 0; + const headline = + fetched.length === 0 && conflicts.length === 0 + ? "Remote history already matches local files." + : fetched.length > 0 + ? `Fetched ${fetched.length} migration(s).` + : "Remote history differs from local files."; + + return { + status: conflicts.length > 0 ? "conflict" : mutatedFiles ? "generated" : "clean", + message: [headline, ...messages].join("\n"), + data: { + status: conflicts.length > 0 ? "conflict" : mutatedFiles ? "generated" : "clean", + fetched, + skipped, + conflicts, + files, + pending: pending.map((file) => file.version), + mutated_files: mutatedFiles, + mutated_database: false, + }, + nextActions: + pending.length > 0 + ? [ + formatNextAction( + remote.kind === "local" + ? "to apply your pending files" + : "to deploy your pending files", + formatMigrationsPushCommand(flags), + ), + ] + : [], + mutatedDatabase: false, + mutatedFiles, + } satisfies SchemaCommandResult; + }), + ); +}); diff --git a/apps/cli/src/shared/migrations/push-migrations.integration.test.ts b/apps/cli/src/shared/migrations/push-migrations.integration.test.ts new file mode 100644 index 0000000000..bc01dca57b --- /dev/null +++ b/apps/cli/src/shared/migrations/push-migrations.integration.test.ts @@ -0,0 +1,1124 @@ +import { describe, expect, it } from "@effect/vitest"; +import { classifyPlanHazards, type Plan } from "@supabase/pg-delta/plan"; +import { Cause, Effect, Exit, Layer, Option } from "effect"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import type { DatabaseTarget } from "../database/database-target.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { + SchemaCancelledError, + SchemaCatalogAdoptError, + SchemaDeclarationsAheadError, + SchemaDestructiveAuthError, + SchemaEmptyMigrationStatementsError, + SchemaEngineError, + SchemaLocalStackNotRunningError, + SchemaAllowRemoteRequiredError, + SchemaHistoryConflictError, + SchemaPrivilegeOfferError, + SchemaRemoteDriftError, +} from "../schema/schema-errors.ts"; +import { SchemaStateStore } from "../schema/schema-state.service.ts"; +import type { SchemaDraftJournal, SchemaPlanView } from "../schema/schema-types.ts"; +import { SchemaWorkspace } from "../schema/schema-workspace.service.ts"; +import { PgDeltaSchemaEngine } from "../schema/pg-delta-engine.service.ts"; +import { formatHistoryConflict } from "./migration-repair-suggest.ts"; +import { MigrationRepository } from "./migration-repository.service.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; +import { REVOKE_API_PRIVILEGES_SQL } from "./privilege-offer.ts"; +import { pushMigrations } from "./push-migrations.ts"; + +const linked = { + kind: "linked" as const, + identity: "abcdefghijklmnop", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: false, + projectRef: "abcdefghijklmnop", +}; + +const ACL_SQL = ` +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" REVOKE ALL ON SEQUENCES FROM "anon"; +ALTER DEFAULT PRIVILEGES FOR ROLE "postgres" IN SCHEMA "public" GRANT SELECT ON TABLES TO "anon"; +`; + +function emptyPlan(): Plan { + return { + formatVersion: 1, + engineVersion: "0.3.0", + planId: "plan", + source: { fingerprint: "s" }, + target: { fingerprint: "d" }, + preamble: [], + deltas: [], + filteredDeltas: [], + renameCandidates: [], + actions: [], + safetyReport: { + destructiveActions: 0, + rewriteRiskActions: 0, + nonTransactionalActions: 0, + lockClasses: {}, + }, + }; +} + +function planView(changes: boolean, sql = ""): SchemaPlanView { + const plan = emptyPlan(); + return { + planId: plan.planId, + sourceFingerprint: plan.source.fingerprint, + desiredFingerprint: plan.target.fingerprint, + engineVersion: plan.engineVersion, + profile: "supabase", + changes, + files: + changes && sql.length > 0 + ? [ + { + sequence: 1, + suffix: null, + sql, + transactional: true, + actionCount: 1, + }, + ] + : [], + hazards: { + kinds: [], + destructive: 0, + rewrite: 0, + coverageGaps: 0, + report: classifyPlanHazards(plan), + }, + destructive: false, + renameCandidates: [], + acceptedRenames: [], + coverageBlocked: false, + renameBlocked: false, + diagnostics: [], + plan, + }; +} + +const ungeneratedJournal: SchemaDraftJournal = { + version: 1, + draftId: "draft", + targetIdentity: "local:default", + startingMigrationHeadDigest: "abc", + sourceFingerprint: "s", + engineVersion: "0.3.0", + declarativelyAhead: true, + generated: false, + plans: [], +}; + +const pendingFile = { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: "select 1;", + transactional: true, +}; + +const revokeFile = { + version: "20260102000000", + name: "revoke_api_privileges", + fileName: "20260102000000_revoke_api_privileges.sql", + absolutePath: "/tmp/migrations/20260102000000_revoke_api_privileges.sql", + content: REVOKE_API_PRIVILEGES_SQL, + transactional: true, +}; + +function setup( + opts: { + declarations?: boolean; + ahead?: boolean; + localRunning?: boolean; + drift?: boolean; + driftResults?: ReadonlyArray; + driftSql?: string; + journal?: SchemaDraftJournal; + aheadSql?: string; + applyFailPending?: boolean; + files?: ReadonlyArray; + history?: ReadonlyArray<{ version: string; name: string }>; + localHistory?: ReadonlyArray<{ version: string; name: string }>; + localHistoryFail?: boolean; + target?: DatabaseTarget; + interactive?: boolean; + confirm?: boolean; + promptTextResponses?: ReadonlyArray; + } = {}, +) { + const out = mockOutput({ + interactive: opts.interactive ?? false, + ...(opts.confirm !== undefined ? { promptConfirmResponses: [opts.confirm] } : {}), + ...(opts.promptTextResponses !== undefined + ? { promptTextResponses: opts.promptTextResponses } + : {}), + }); + let shadowProvisions = 0; + let platformProvisions = 0; + let appliedVersions: ReadonlyArray = []; + let applyCalls = 0; + let remoteApplyCalls = 0; + let diffCalls = 0; + const remoteUrl = (opts.target ?? linked).connectionString; + const layer = Layer.mergeAll( + out.layer, + Layer.succeed( + SchemaWorkspace, + SchemaWorkspace.of({ + schemasDir: "/tmp/schemas", + schemasDirDisplay: "supabase/schemas", + migrationsDir: "/tmp/migrations", + migrationsDirDisplay: "supabase/migrations", + customDir: "/tmp/schemas/_custom", + journalPath: "/tmp/.supabase/schema-draft.json", + lockPath: "/tmp/.supabase/schema.lock", + readDeclarationFiles: Effect.succeed( + opts.declarations === false ? [] : [{ name: "a.sql", sql: "create table a (id int);" }], + ), + readExistingSql: () => Effect.succeed(new Map()), + classifyProposed: () => Effect.die("unused"), + installExport: () => Effect.die("unused"), + }), + ), + Layer.succeed( + SchemaStateStore, + SchemaStateStore.of({ + readJournal: Effect.succeed( + opts.journal === undefined ? Option.none() : Option.some(opts.journal), + ), + writeJournal: () => Effect.void, + clearJournal: Effect.void, + withLock: (effect) => effect, + }), + ), + Layer.succeed( + DatabaseTargetResolver, + DatabaseTargetResolver.of({ + resolve: (selector) => { + if (selector.kind === "local") { + if (opts.localRunning === false) { + return Effect.fail( + new SchemaLocalStackNotRunningError({ + detail: "No local Supabase stack is running for this project.", + suggestion: "Run `supabase start`, then retry.", + }), + ); + } + return Effect.succeed({ + kind: "local" as const, + identity: "local:default", + connectionString: "postgresql://postgres:postgres@127.0.0.1:54322/postgres", + disposable: true, + durable: false, + connectionVerified: true, + }); + } + return Effect.succeed(opts.target ?? linked); + }, + }), + ), + Layer.succeed( + MigrationRepository, + MigrationRepository.of({ + listLocal: Effect.succeed(opts.files ?? []), + createEmpty: () => Effect.die("unused"), + writeFetched: () => Effect.die("unused"), + writeGenerated: () => Effect.die("unused"), + remove: () => Effect.die("unused"), + }), + ), + Layer.succeed( + MigrationRunner, + MigrationRunner.of({ + listRemote: (pool) => { + const conn = pool.options.connectionString ?? ""; + if (conn.includes("54322")) { + if (opts.localHistoryFail === true) { + return Effect.fail( + new SchemaEngineError({ + detail: "local history unavailable", + suggestion: "Retry when the local database is reachable.", + }), + ); + } + return Effect.succeed(opts.localHistory ?? []); + } + return Effect.succeed(opts.history ?? []); + }, + listRemoteStatements: () => Effect.succeed([]), + showServerVersion: () => Effect.succeed(undefined), + listInstalledExtensions: () => Effect.die("unused"), + applyPending: (pool, files) => + Effect.gen(function* () { + applyCalls += 1; + const recorded = new Set((opts.history ?? []).map((row) => row.version)); + const pendingOnly = files.filter((file) => !recorded.has(file.version)); + const remoteOnly = [...recorded].filter( + (version) => !files.some((file) => file.version === version), + ); + if (remoteOnly.length > 0 && pendingOnly.length > 0) { + return yield* new SchemaHistoryConflictError( + formatHistoryConflict({ + remoteOnly, + pending: pendingOnly.map((file) => file.version), + }), + ); + } + if (opts.applyFailPending === true && pendingOnly.length > 0) { + return yield* new SchemaEngineError({ + detail: + "Failed applying migration: function gen_random_bytes(integer) does not exist", + suggestion: "Check the database connection and migration SQL, then retry.", + }); + } + const conn = pool.options.connectionString ?? ""; + const isRemote = conn.includes("db.example") || conn === remoteUrl; + if (isRemote) { + remoteApplyCalls += 1; + appliedVersions = files.map((file) => file.version); + } + return { + applied: pendingOnly.map((file) => file.version), + skipped: files + .filter((file) => (opts.history ?? []).some((row) => row.version === file.version)) + .map((file) => file.version), + }; + }), + markApplied: () => Effect.die("unused"), + }), + ), + Layer.succeed( + PgDeltaSchemaEngine, + PgDeltaSchemaEngine.of({ + exportSchema: () => Effect.die("unused"), + planFiles: () => Effect.succeed(planView(opts.ahead === true, opts.aheadSql ?? "")), + diffPools: () => { + const changes = + opts.driftResults !== undefined + ? opts.driftResults[diffCalls] === true + : opts.drift === true; + diffCalls += 1; + return Effect.succeed(planView(changes, opts.driftSql ?? "")); + }, + applyPlan: () => Effect.die("unused"), + provisionShadow: Effect.sync(() => { + shadowProvisions += 1; + return { url: "postgresql://postgres:postgres@127.0.0.1:1/postgres" }; + }), + provisionPlatform: Effect.sync(() => { + platformProvisions += 1; + return { url: "postgresql://postgres:postgres@127.0.0.1:1/postgres" }; + }), + provisionMigrations: Effect.sync(() => { + shadowProvisions += 1; + return { url: "postgresql://postgres:postgres@127.0.0.1:1/postgres" }; + }), + }), + ), + ); + return { + layer, + out, + get shadowProvisions() { + return shadowProvisions; + }, + get platformProvisions() { + return platformProvisions; + }, + get appliedVersions() { + return appliedVersions; + }, + get applyCalls() { + return applyCalls; + }, + get remoteApplyCalls() { + return remoteApplyCalls; + }, + }; +} + +const pushFlags = { + yes: true, + allowRemote: false, + projectRef: "abcdefghijklmnop", + skipVerify: false, +} as const; + +describe("pushMigrations", () => { + it.live("refuses DATABASE_URL before listing history", () => { + const previous = process.env["DATABASE_URL"]; + process.env["DATABASE_URL"] = "postgresql://postgres:secret@other.example/postgres"; + const ctx = setup({ + declarations: false, + files: [pendingFile], + history: [{ version: "20260826095358", name: "notes" }], + target: { + kind: "url", + identity: "connection-string", + connectionString: "postgresql://postgres:secret@other.example/postgres", + disposable: false, + durable: true, + connectionVerified: false, + connectionSource: "env", + }, + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations({ + yes: true, + allowRemote: false, + skipVerify: false, + }).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaAllowRemoteRequiredError); + expect(JSON.stringify(exit)).toContain("Unset DATABASE_URL"); + expect(JSON.stringify(exit)).not.toContain("migrations pull --from"); + } + expect(ctx.applyCalls).toBe(0); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["DATABASE_URL"]; + else process.env["DATABASE_URL"] = previous; + }), + ), + ); + }); + + it.live("fails closed when an ungenerated draft is active", () => { + const { layer } = setup({ journal: ungeneratedJournal }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + }); + }); + + it.live("fails closed when live M to D still has changes", () => { + const ctx = setup({ + declarations: true, + ahead: true, + aheadSql: "CREATE EXTENSION IF NOT EXISTS pgjwt WITH SCHEMA extensions;", + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaDeclarationsAheadError); + expect(JSON.stringify(exit)).toContain("pgjwt"); + expect(JSON.stringify(exit)).toContain("schema generate --name "); + } + expect(ctx.out.stdoutText).toContain("CREATE EXTENSION"); + expect(ctx.remoteApplyCalls).toBe(0); + }); + }); + + it.live("privilege-only decls-ahead suggests refresh, not generate", () => { + const ctx = setup({ + declarations: true, + ahead: true, + aheadSql: ACL_SQL, + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaDeclarationsAheadError); + expect(JSON.stringify(exit)).toContain("schema pull --force"); + expect(JSON.stringify(exit)).toContain("db reset"); + expect(JSON.stringify(exit)).not.toContain("schema generate --name"); + } + expect(ctx.remoteApplyCalls).toBe(0); + }); + }); + + it.live("refuses catalog gap with diff then repair, not pull", () => { + const ctx = setup({ + declarations: false, + localRunning: false, + drift: true, + driftSql: "CREATE TABLE extra (id int);", + files: [pendingFile], + history: [{ version: pendingFile.version, name: pendingFile.name }], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaRemoteDriftError); + expect(JSON.stringify(exit)).toContain("supabase migrations diff --against linked --file"); + expect(JSON.stringify(exit)).toContain("migration repair"); + expect(JSON.stringify(exit)).not.toContain("supabase migrations pull"); + } + expect(ctx.shadowProvisions).toBe(0); + expect(ctx.platformProvisions).toBe(1); + expect(ctx.applyCalls).toBe(1); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "Ensuring declarations and migrations match before push.", + }), + ); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "Declarations and migrations are not in sync.", + }), + ); + }); + }); + + it.live("first-push matching prefix suggests repair, not apply", () => { + const ctx = setup({ + declarations: false, + driftResults: [true, false], + driftSql: "CREATE TABLE extra (id int);", + files: [pendingFile], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaRemoteDriftError); + expect(JSON.stringify(exit)).toContain( + "supabase migration repair --project-ref abcdefghijklmnop --status applied 20260101000000", + ); + expect(JSON.stringify(exit)).not.toContain("supabase migrations pull"); + expect(JSON.stringify(exit)).not.toContain("migrations diff --against"); + } + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: + "Checking whether pending files already match the remote (shadow probe, not a live apply).", + }), + ); + }); + }); + + it.live("first-push dirty with --yes prints SQL then applies", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: "CREATE TABLE extra (id int);", + files: [pendingFile], + }); + return Effect.gen(function* () { + const result = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "No remote migration history yet. This is the first push.", + }), + ); + expect(ctx.out.stdoutText).toContain("CREATE TABLE extra"); + expect(result.data).toEqual( + expect.objectContaining({ + sql: "CREATE TABLE extra (id int);", + files: [{ name: pendingFile.fileName, version: pendingFile.version }], + }), + ); + expect(JSON.stringify(result)).not.toContain("supabase migrations pull"); + }); + }); + + it.live("TTY --yes still types the project ref and skips catalog confirm", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: "CREATE TABLE extra (id int);", + files: [pendingFile], + interactive: true, + promptTextResponses: ["abcdefghijklmnop"], + }); + return Effect.gen(function* () { + const result = yield* pushMigrations({ + yes: true, + allowRemote: false, + skipVerify: false, + }).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(ctx.out.promptConfirmCalls).toEqual([]); + }); + }); + + it.live("TTY --yes still fails when the typed project ref does not match", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: "CREATE TABLE extra (id int);", + files: [pendingFile], + interactive: true, + promptTextResponses: ["wrong-ref"], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations({ + yes: true, + allowRemote: false, + skipVerify: false, + }).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaCancelledError); + } + expect(ctx.applyCalls).toBe(0); + }); + }); + + it.live("TTY first-push dirty cancel leaves the remote unchanged", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: "CREATE TABLE extra (id int);", + files: [pendingFile], + interactive: true, + confirm: false, + promptTextResponses: ["abcdefghijklmnop"], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations({ + yes: false, + allowRemote: false, + skipVerify: false, + }).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaCancelledError); + } + expect(ctx.applyCalls).toBe(2); + }); + }); + + it.live("non-interactive dirty first-push without --yes requires confirmation", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: "CREATE TABLE extra (id int);", + files: [pendingFile], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations({ + yes: false, + allowRemote: false, + projectRef: "abcdefghijklmnop", + skipVerify: false, + }).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaDestructiveAuthError); + expect(failure.value.suggestion).toContain("--yes"); + } + expect(ctx.applyCalls).toBe(2); + }); + }); + + it.live("refuses empty history with no files and a dirty catalog as adopt", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: "CREATE TABLE extra (id int);", + files: [], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaCatalogAdoptError); + expect(JSON.stringify(exit)).toContain("supabase migrations diff --against linked --file"); + expect(JSON.stringify(exit)).toContain("migration repair"); + expect(JSON.stringify(exit)).not.toContain("db diff"); + expect(JSON.stringify(exit)).not.toContain("up to date"); + } + expect(ctx.applyCalls).toBe(1); + }); + }); + + it.live("refuses remote-only versions with migrations pull", () => { + const { layer } = setup({ + declarations: false, + files: [pendingFile], + history: [{ version: "19990101000000", name: "other" }], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(JSON.stringify(exit)).toContain("supabase migrations pull --from linked"); + expect(JSON.stringify(exit)).not.toContain("repair --status reverted"); + }); + }); + + it.live("pushes when replay matches declarations and the remote", () => { + const ctx = setup({ + declarations: true, + ahead: false, + localRunning: false, + drift: false, + }); + return Effect.gen(function* () { + const result = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(false); + expect(ctx.shadowProvisions).toBe(2); + expect(ctx.platformProvisions).toBe(1); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "No pending migrations. History matches files on the linked project.", + }), + ); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "Ensuring declarations and migrations match before push.", + }), + ); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "Declarations and migrations are in sync.", + }), + ); + }); + }); + + it.live("previews pending files before the ref prompt", () => { + const ctx = setup({ + declarations: false, + drift: false, + files: [pendingFile], + interactive: true, + promptTextResponses: ["abcdefghijklmnop"], + }); + return Effect.gen(function* () { + const result = yield* pushMigrations({ + yes: true, + allowRemote: false, + skipVerify: false, + }).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(ctx.out.stdoutText).toContain("20260101000000 init"); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "1 pending migration will be applied on the linked project.", + }), + ); + }); + }); + + it.live("says the remote is in sync before the ref prompt when nothing is pending", () => { + const ctx = setup({ + declarations: false, + drift: false, + interactive: true, + promptTextResponses: ["abcdefghijklmnop"], + }); + return Effect.gen(function* () { + const result = yield* pushMigrations({ + yes: true, + allowRemote: false, + skipVerify: false, + }).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(false); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "No pending migrations. History matches files on the linked project.", + }), + ); + }); + }); + + it.live("still refuses an ungenerated draft when --skip-verify is set", () => { + const { layer } = setup({ journal: ungeneratedJournal }); + return Effect.gen(function* () { + const exit = yield* pushMigrations({ ...pushFlags, skipVerify: true }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + }); + }); + + it.live("points skip-verify remote-only history at pull", () => { + const { layer } = setup({ + files: [pendingFile], + history: [{ version: "19990101000000", name: "other" }], + target: { + kind: "url", + identity: "connection-string", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: false, + connectionSource: "flag", + }, + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations({ + yes: true, + allowRemote: true, + skipVerify: true, + dbUrl: "postgresql://postgres:secret@db.example/postgres", + }).pipe(Effect.provide(layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(JSON.stringify(exit)).toContain("supabase migrations pull --from "); + expect(JSON.stringify(exit)).not.toContain("repair --status reverted"); + }); + }); + + it.live("skips shadow verify when --skip-verify is set", () => { + const ctx = setup({ + declarations: true, + ahead: true, + drift: true, + }); + return Effect.gen(function* () { + const result = yield* pushMigrations({ ...pushFlags, skipVerify: true }).pipe( + Effect.provide(ctx.layer), + ); + expect(result.mutatedDatabase).toBe(false); + expect(ctx.shadowProvisions).toBe(0); + expect(ctx.platformProvisions).toBe(0); + }); + }); + + it.live("offers keep-on vs turn-off for ACL-only first push", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: ACL_SQL, + files: [pendingFile], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaPrivilegeOfferError); + expect(JSON.stringify(exit)).toContain("api.auto_expose_new_tables"); + expect(JSON.stringify(exit)).toContain( + "migrations new revoke_api_privileges --template revoke-api-privileges", + ); + expect(JSON.stringify(exit)).toContain("schema pull --force"); + expect(JSON.stringify(exit)).not.toContain("supabase migrations pull"); + } + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "Remote default privileges differ from migration replay.", + }), + ); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "Detected host-vs-replay privilege SQL (will not run).", + }), + ); + expect(ctx.out.messages).not.toContainEqual( + expect.objectContaining({ + type: "info", + message: "Declarations and migrations are not in sync.", + }), + ); + expect(ctx.out.stdoutText).toContain("ALTER DEFAULT PRIVILEGES"); + expect(ctx.applyCalls).toBe(1); + expect(ctx.remoteApplyCalls).toBe(0); + expect(ctx.appliedVersions).toEqual([]); + }); + }); + + it.live("applies a pending revoke instead of offering turn-off again", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: ACL_SQL, + files: [pendingFile, revokeFile], + localRunning: false, + }); + return Effect.gen(function* () { + const result = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(ctx.appliedVersions).toEqual([pendingFile.version, revokeFile.version]); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "Pending privilege migration will run on the remote.", + }), + ); + expect(ctx.out.messages).not.toContainEqual( + expect.objectContaining({ + type: "info", + message: "Declarations and migrations are not in sync.", + }), + ); + expect(ctx.out.stdoutText).not.toContain("ALTER DEFAULT PRIVILEGES"); + expect(ctx.out.promptConfirmCalls).toEqual([]); + }); + }); + + it.live("refuses live-edit when a pending revoke sits next to unrelated catalog drift", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: "CREATE TABLE extra (id int);", + files: [pendingFile, revokeFile], + history: [{ version: pendingFile.version, name: pendingFile.name }], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaRemoteDriftError); + expect(JSON.stringify(exit)).toContain("supabase migrations diff --against linked --file"); + expect(JSON.stringify(exit)).not.toContain("api.auto_expose_new_tables"); + } + expect(ctx.applyCalls).toBe(2); + }); + }); + + it.live("applies a pending revoke before declarations-ahead", () => { + const ctx = setup({ + declarations: true, + ahead: true, + aheadSql: "GRANT EXECUTE ON FUNCTION public.accept_invitation() TO service_role;", + drift: true, + driftSql: ACL_SQL, + files: [pendingFile, revokeFile], + history: [{ version: pendingFile.version, name: pendingFile.name }], + }); + return Effect.gen(function* () { + const result = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(ctx.appliedVersions).toEqual([pendingFile.version, revokeFile.version]); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "Pending privilege migration will run on the remote.", + }), + ); + expect(JSON.stringify(ctx.out.messages)).not.toContain("schema generate --name"); + }); + }); + + it.live("refuses a pending file with no executable SQL", () => { + const ctx = setup({ + declarations: false, + files: [ + { + ...pendingFile, + name: "sneak", + fileName: "20260101000000_sneak.sql", + content: "-- empty stub\n", + }, + ], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaEmptyMigrationStatementsError); + expect(JSON.stringify(exit)).toContain("20260101000000_sneak.sql"); + expect(JSON.stringify(exit)).not.toContain("schema generate --name"); + } + expect(ctx.remoteApplyCalls).toBe(0); + }); + }); + + it.live("applies a pending revoke when remote history already exists", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: ACL_SQL, + files: [pendingFile, revokeFile], + history: [{ version: pendingFile.version, name: pendingFile.name }], + }); + return Effect.gen(function* () { + const result = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(ctx.appliedVersions).toEqual([pendingFile.version, revokeFile.version]); + }); + }); + + it.live("refreshes declarations instead of live-edit for leftover function grants", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: `GRANT EXECUTE ON FUNCTION public.accept_invitation() TO service_role;`, + files: [revokeFile, pendingFile], + history: [{ version: revokeFile.version, name: revokeFile.name }], + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaRemoteDriftError); + expect(JSON.stringify(exit)).toContain("schema pull --force"); + expect(JSON.stringify(exit)).not.toContain("migrations diff --against"); + expect(JSON.stringify(exit)).not.toContain("repair --status applied"); + } + expect(ctx.remoteApplyCalls).toBe(0); + }); + }); + + it.live("keeps a URL privilege offer on the selected database", () => { + const ctx = setup({ + declarations: false, + drift: true, + driftSql: ACL_SQL, + files: [pendingFile], + target: { + kind: "url", + identity: "connection-string", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: false, + connectionSource: "flag", + }, + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations({ + yes: true, + allowRemote: true, + skipVerify: false, + dbUrl: "postgresql://postgres:secret@db.example/postgres", + }).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(JSON.stringify(exit)).toContain( + "supabase migrations push --db-url --allow-remote", + ); + }); + }); + + it.live("first-push clean applies pending files", () => { + const ctx = setup({ + declarations: false, + drift: false, + files: [pendingFile], + }); + return Effect.gen(function* () { + const result = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "No remote migration history yet. This is the first push.", + }), + ); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "Replaying pending files on a shadow before live apply.", + }), + ); + expect(ctx.out.messages).toContainEqual( + expect.objectContaining({ + type: "info", + message: "Catalog matches migration replay; pending files were verified on a shadow.", + }), + ); + expect(ctx.out.messages).not.toContainEqual( + expect.objectContaining({ + type: "info", + message: "Declarations and migrations are in sync.", + }), + ); + expect(ctx.remoteApplyCalls).toBe(1); + expect(result.message).toContain("1 of 1 migration pending on the local database."); + expect(result.nextActions).toEqual(["to apply it locally: supabase migrations apply"]); + }); + }); + + it.live("does not hint local apply when local history cannot be listed", () => { + const ctx = setup({ + declarations: false, + drift: false, + files: [pendingFile], + localHistoryFail: true, + }); + return Effect.gen(function* () { + const result = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(result.message).not.toContain("pending on the local database"); + expect(result.nextActions).toEqual([]); + }); + }); + + it.live("does not hint local apply when the local database is down", () => { + const ctx = setup({ + declarations: false, + drift: false, + files: [pendingFile], + localRunning: false, + }); + return Effect.gen(function* () { + const result = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer)); + expect(result.mutatedDatabase).toBe(true); + expect(result.message).not.toContain("pending on the local database"); + expect(result.nextActions).toEqual([]); + }); + }); + + it.live("pending shadow probe failure leaves the remote unchanged", () => { + const ctx = setup({ + declarations: false, + drift: false, + files: [pendingFile], + applyFailPending: true, + }); + return Effect.gen(function* () { + const exit = yield* pushMigrations(pushFlags).pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : undefined; + expect(failure?._tag).toBe("Some"); + if (failure?._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaEngineError); + expect(JSON.stringify(exit)).toContain("gen_random_bytes"); + } + expect(ctx.out.messages).not.toContainEqual( + expect.objectContaining({ + type: "info", + message: "Declarations and migrations are in sync.", + }), + ); + expect(ctx.out.messages).not.toContainEqual( + expect.objectContaining({ + type: "info", + message: "Catalog matches migration replay; pending files were verified on a shadow.", + }), + ); + expect(ctx.remoteApplyCalls).toBe(0); + }); + }); +}); diff --git a/apps/cli/src/shared/migrations/push-migrations.ts b/apps/cli/src/shared/migrations/push-migrations.ts new file mode 100644 index 0000000000..09b4400104 --- /dev/null +++ b/apps/cli/src/shared/migrations/push-migrations.ts @@ -0,0 +1,447 @@ +import { Effect } from "effect"; +import type { Pool } from "pg"; +import { explicitBooleanLongFlag } from "../cli/cobra-flag-groups.ts"; +import { acquireDatabasePool } from "../database/database-pool.ts"; +import { authorizeMutation } from "../database/destructive-auth.ts"; +import type { DatabaseTarget } from "../database/database-target.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { Output } from "../output/output.service.ts"; +import { assertNoUngeneratedDraft } from "../schema/declarations-ahead.ts"; +import { + formatMigrationInventory, + formatPlanSql, + humanTarget, + planStatementCount, + type SchemaScriptFile, +} from "../schema/schema-body.ts"; +import { + SchemaCancelledError, + SchemaCatalogAdoptError, + SchemaDeclarationsAheadError, + SchemaDestructiveAuthError, + SchemaHistoryConflictError, + SchemaRemoteDriftError, +} from "../schema/schema-errors.ts"; +import { formatNextAction } from "../schema/schema-output.ts"; +import type { SchemaCommandResult, SchemaPlanView } from "../schema/schema-types.ts"; +import { wrapShadowReplayOutput } from "../schema/shadow-replay-output.ts"; +import { SchemaWorkspace } from "../schema/schema-workspace.service.ts"; +import { PgDeltaSchemaEngine } from "../schema/pg-delta-engine.service.ts"; +import type { MigrationFile } from "./migration-file.ts"; +import { findMatchingPendingPrefix } from "./matching-pending-prefix.ts"; +import { + formatHistoryConflict, + formatLiveEditCommands, + formatMigrationRepairCommand, + formatMigrationsPushCommand, + repairFlagsForTarget, + type MigrationRepairFlags, +} from "./migration-repair-suggest.ts"; +import { MigrationRepository } from "./migration-repository.service.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; +import { + classifyPrivilegePlan, + emptyPendingMigrationError, + pendingHasPrivilegeSql, + privilegeOfferError, + PRIVILEGE_REFRESH_SUGGESTION, +} from "./privilege-offer.ts"; +import { warnIfRemotePostgresMajorMismatch } from "./remote-postgres.ts"; + +export type PushMigrationsInput = { + readonly yes: boolean; + readonly projectRef?: string; + readonly allowRemote: boolean; + readonly dbUrl?: string; + readonly skipVerify: boolean; +}; + +const MATCHING_PREFIX_BANNER = + "Checking whether pending files already match the remote (shadow probe, not a live apply)."; +const VERIFY_BANNER = "Ensuring declarations and migrations match before push."; +const VERIFY_IN_SYNC = "Declarations and migrations are in sync."; +const VERIFY_NOT_IN_SYNC = "Declarations and migrations are not in sync."; +const VERIFY_PENDING_SHADOW = "Replaying pending files on a shadow before live apply."; +const VERIFY_PENDING_OK = + "Catalog matches migration replay; pending files were verified on a shadow."; +const PRIVILEGE_BANNER = "Remote default privileges differ from migration replay."; +const PRIVILEGE_DUMP_HEADER = "Detected host-vs-replay privilege SQL (will not run)."; +const PRIVILEGE_PENDING_BANNER = "Pending privilege migration will run on the remote."; +const DECLARATIONS_AHEAD_GENERATE = + "Update `supabase/schemas` to include hand-written migration changes, or run `supabase schema generate --name ` if declarations are the intended state."; +const DECLARATIONS_AHEAD_REFRESH = + "Declarations differ from migration replay by default privileges only. Run `supabase db reset` then `supabase schema pull --force` so declarations match the grant-kept baseline. Do not write GRANT ALL."; + +const pendingScriptFiles = ( + pending: ReadonlyArray<{ readonly fileName: string; readonly version: string }>, +): ReadonlyArray => + pending.map((file) => ({ name: file.fileName, version: file.version })); + +const previewPending = Effect.fnUntraced(function* ( + pending: ReadonlyArray<{ readonly version: string; readonly name: string }>, + target: DatabaseTarget, +) { + const output = yield* Output; + if (output.format !== "text") return; + if (pending.length === 0) { + yield* output.info(`No pending migrations. History matches files on ${humanTarget(target)}.`); + return; + } + const inventory = formatMigrationInventory( + pending.map((file) => ({ version: file.version, name: file.name })), + ); + if (inventory.length > 0) { + yield* output.raw(inventory.endsWith("\n") ? inventory : `${inventory}\n`); + } + const noun = pending.length === 1 ? "migration" : "migrations"; + yield* output.info( + `${pending.length} pending ${noun} will be applied on ${humanTarget(target)}.`, + ); +}); + +const emitPlanSql = Effect.fnUntraced(function* (plan: SchemaPlanView) { + const output = yield* Output; + const sql = formatPlanSql(plan); + if (output.format === "text") { + const count = planStatementCount(plan); + if (count > 0) { + yield* output.info(`${count} ${count === 1 ? "statement" : "statements"}`); + } + if (sql.length > 0) { + yield* output.raw(sql.endsWith("\n") ? sql : `${sql}\n`); + } + } + return sql; +}); + +const refusePrivilegeOffer = Effect.fnUntraced(function* ( + plan: SchemaPlanView, + flags: MigrationRepairFlags | undefined, + files: ReadonlyArray, +) { + const output = yield* Output; + yield* output.info(PRIVILEGE_BANNER); + yield* output.info(PRIVILEGE_DUMP_HEADER); + const sql = yield* emitPlanSql(plan); + return yield* privilegeOfferError(sql, flags, files); +}); + +const refuseLiveEdit = Effect.fnUntraced(function* ( + plan: SchemaPlanView, + flags: MigrationRepairFlags, + files: ReadonlyArray, +) { + const sql = yield* emitPlanSql(plan); + return yield* new SchemaRemoteDriftError({ + detail: "Remote database shape has drifted from migration replay.", + suggestion: formatLiveEditCommands(flags), + sql, + files, + }); +}); + +const refusePrivilegeRefresh = (files: ReadonlyArray) => + new SchemaRemoteDriftError({ + detail: "Remote privileges differ from migration replay.", + suggestion: PRIVILEGE_REFRESH_SUGGESTION, + files, + }); + +const refuseCatalogAdopt = Effect.fnUntraced(function* ( + plan: SchemaPlanView, + flags: MigrationRepairFlags, + files: ReadonlyArray, +) { + const sql = yield* emitPlanSql(plan); + return yield* new SchemaCatalogAdoptError({ + detail: "Remote catalog has objects but there is no migration history and no local files.", + suggestion: formatLiveEditCommands(flags), + sql, + files, + }); +}); + +const refuseMatchingPrefix = (input: { + readonly matching: ReadonlyArray<{ readonly version: string }>; + readonly flags: MigrationRepairFlags; + readonly files: ReadonlyArray; +}) => + new SchemaRemoteDriftError({ + detail: "Remote catalog already matches these pending migration files.", + suggestion: formatMigrationRepairCommand({ + status: "applied", + versions: input.matching.map((file) => file.version), + flags: input.flags, + }), + files: input.files, + }); + +const probeMatchingPrefix = Effect.fnUntraced(function* ( + replayPool: Pool, + remotePool: Pool, + replayed: ReadonlyArray, + pending: ReadonlyArray, + flags: MigrationRepairFlags, +) { + const output = yield* Output; + yield* output.info(MATCHING_PREFIX_BANNER); + const matching = yield* findMatchingPendingPrefix(replayPool, remotePool, replayed, pending); + if (matching.length > 0) { + return yield* refuseMatchingPrefix({ + matching, + flags, + files: pendingScriptFiles(pending), + }); + } +}); + +const noteLocalPending = Effect.fnUntraced(function* (localFiles: ReadonlyArray) { + if (localFiles.length === 0) return undefined; + const targets = yield* DatabaseTargetResolver; + const runner = yield* MigrationRunner; + const local = yield* targets + .resolve({ kind: "local" }) + .pipe(Effect.catch(() => Effect.succeed(undefined))); + if (local === undefined) return undefined; + return yield* Effect.scoped( + Effect.gen(function* () { + const pool = yield* acquireDatabasePool(local.connectionString); + const history = yield* runner + .listRemote(pool) + .pipe(Effect.catch(() => Effect.succeed(undefined))); + if (history === undefined) return undefined; + const versions = new Set(history.map((row) => row.version)); + const pending = localFiles.filter((file) => !versions.has(file.version)).length; + if (pending === 0) return undefined; + const total = localFiles.length; + return `${pending} of ${total} ${total === 1 ? "migration" : "migrations"} pending on the local database.`; + }), + ).pipe(Effect.catch(() => Effect.succeed(undefined))); +}); + +const confirmFirstPushDirty = Effect.fnUntraced(function* ( + yes: boolean, + flags?: MigrationRepairFlags, +) { + if (yes) return; + const output = yield* Output; + if (output.interactive) { + const ok = yield* output.promptConfirm( + "Apply pending migrations on the remote despite this catalog difference?", + ); + if (!ok) { + return yield* new SchemaCancelledError({ + detail: "First push cancelled.", + suggestion: `Re-run ${formatMigrationsPushCommand(flags)} after reviewing the SQL.`, + }); + } + return; + } + return yield* new SchemaDestructiveAuthError({ + detail: "Non-interactive first push of a catalog difference requires confirmation.", + suggestion: "Re-run with --yes after reviewing the SQL, or abort.", + }); +}); + +export const pushMigrations = Effect.fn("migrations.push")(function* (input: PushMigrationsInput) { + const targets = yield* DatabaseTargetResolver; + const repository = yield* MigrationRepository; + const runner = yield* MigrationRunner; + const engine = yield* PgDeltaSchemaEngine; + const workspace = yield* SchemaWorkspace; + const output = yield* Output; + + yield* assertNoUngeneratedDraft(); + + const remote = yield* targets.resolve( + input.dbUrl !== undefined ? { kind: "url", url: input.dbUrl } : { kind: "linked" }, + ); + yield* authorizeMutation({ + target: remote, + flags: { + yes: input.yes, + allowRemote: input.allowRemote, + ...(input.projectRef !== undefined ? { projectRef: input.projectRef } : {}), + }, + command: "migrations push", + }); + const localFiles = yield* repository.listLocal; + const declarations = yield* workspace.readDeclarationFiles; + + return yield* Effect.scoped( + Effect.gen(function* () { + const remotePool = yield* acquireDatabasePool(remote.connectionString); + const remoteHistory = yield* runner.listRemote(remotePool); + const remoteVersions = new Set(remoteHistory.map((row) => row.version)); + const pending = localFiles.filter((file) => !remoteVersions.has(file.version)); + const remoteOnly = remoteHistory.filter( + (row) => !localFiles.some((file) => file.version === row.version), + ); + const flags = repairFlagsForTarget(remote, { + ...(input.projectRef !== undefined ? { projectRef: input.projectRef } : {}), + ...(input.dbUrl !== undefined ? { dbUrl: input.dbUrl } : {}), + }); + const emptyHistory = remoteHistory.length === 0; + const files = pendingScriptFiles(pending); + + yield* warnIfRemotePostgresMajorMismatch(remotePool, remote); + + if (remoteOnly.length > 0) { + return yield* new SchemaHistoryConflictError( + formatHistoryConflict({ + remoteOnly: remoteOnly.map((row) => row.version), + pending: pending.map((file) => file.version), + flags, + }), + ); + } + + const emptyPending = emptyPendingMigrationError(pending); + if (emptyPending !== undefined) { + return yield* emptyPending; + } + + yield* previewPending(pending, remote); + + const pendingPrivilege = yield* pendingHasPrivilegeSql(pending); + + let planSql: string | undefined; + if (!input.skipVerify) { + yield* output.info(VERIFY_BANNER); + if (declarations.length > 0 && !pendingPrivilege) { + const sourceShadow = yield* engine.provisionMigrations; + const desiredShadow = yield* engine.provisionShadow; + const sourcePool = yield* acquireDatabasePool(sourceShadow.url); + const desiredPool = yield* acquireDatabasePool(desiredShadow.url); + const ahead = yield* engine.planFiles({ + targetPool: sourcePool, + shadowPool: desiredPool, + files: declarations, + allowDrops: true, + }); + if (ahead.changes) { + yield* output.info(VERIFY_NOT_IN_SYNC); + const sql = yield* emitPlanSql(ahead); + const aheadKind = yield* classifyPrivilegePlan(ahead); + return yield* new SchemaDeclarationsAheadError({ + detail: "Declarations and local migration files have diverged.", + suggestion: + aheadKind === "not_acl" ? DECLARATIONS_AHEAD_GENERATE : DECLARATIONS_AHEAD_REFRESH, + ...(sql.length > 0 ? { sql } : {}), + files: ahead.files.map((file) => ({ + name: file.suffix ?? `schema-${file.sequence}.sql`, + sql: file.sql, + })), + }); + } + } + + const driftShadow = yield* engine.provisionPlatform; + const replayPool = yield* acquireDatabasePool(driftShadow.url); + const replayed = localFiles.filter((file) => remoteVersions.has(file.version)); + const replayOutput = wrapShadowReplayOutput(output, { + debug: explicitBooleanLongFlag(process.argv, "debug") === true, + }); + yield* runner + .applyPending(replayPool, replayed) + .pipe(Effect.provideService(Output, replayOutput)); + const drift = yield* engine.diffPools({ + sourcePool: replayPool, + desiredPool: remotePool, + allowDrops: true, + }); + + const privilegeKind = yield* classifyPrivilegePlan(drift); + const privilegeOffer = + emptyHistory && drift.changes && privilegeKind === "grant_present" && !pendingPrivilege; + + if (emptyHistory) { + yield* output.info("No remote migration history yet. This is the first push."); + } + if (privilegeOffer) { + return yield* refusePrivilegeOffer(drift, flags, files); + } + + if (emptyHistory && drift.changes && pending.length === 0) { + yield* output.info(VERIFY_NOT_IN_SYNC); + return yield* refuseCatalogAdopt(drift, flags, files); + } + + const pendingPrivilegeApply = + drift.changes && pendingPrivilege && privilegeKind !== "not_acl"; + + if (pendingPrivilegeApply) { + yield* output.info(PRIVILEGE_PENDING_BANNER); + } else if (emptyHistory && drift.changes && pending.length > 0) { + yield* output.info(VERIFY_NOT_IN_SYNC); + yield* probeMatchingPrefix(replayPool, remotePool, replayed, pending, flags).pipe( + Effect.provideService(Output, replayOutput), + ); + planSql = yield* emitPlanSql(drift); + yield* confirmFirstPushDirty(input.yes, flags); + } else if ( + !emptyHistory && + drift.changes && + privilegeKind !== "not_acl" && + !pendingPrivilege + ) { + return yield* refusePrivilegeRefresh(files); + } else if ( + !emptyHistory && + drift.changes && + !(pendingPrivilege && privilegeKind !== "not_acl") + ) { + yield* output.info(VERIFY_NOT_IN_SYNC); + yield* probeMatchingPrefix(replayPool, remotePool, replayed, pending, flags).pipe( + Effect.provideService(Output, replayOutput), + ); + return yield* refuseLiveEdit(drift, flags, files); + } else if (drift.changes) { + yield* output.info(VERIFY_NOT_IN_SYNC); + } + + if (pending.length > 0) { + yield* output.info(VERIFY_PENDING_SHADOW); + // Full local inventory: applyPending treats a pending-only list as remote-only history. + yield* runner + .applyPending(replayPool, localFiles) + .pipe(Effect.provideService(Output, replayOutput)); + } + + if (!drift.changes) { + yield* output.info(pending.length > 0 ? VERIFY_PENDING_OK : VERIFY_IN_SYNC); + } + } + + const result = yield* runner.applyPending(remotePool, localFiles); + const localPendingLine = + result.applied.length > 0 ? yield* noteLocalPending(localFiles) : undefined; + const nextActions = + localPendingLine !== undefined + ? [formatNextAction("to apply it locally", "supabase migrations apply")] + : []; + const pushed = + result.applied.length === 0 + ? "Remote database is up to date." + : `Pushed ${result.applied.length} migration(s) to ${remote.identity}.`; + return { + status: "clean", + message: localPendingLine !== undefined ? `${pushed}\n${localPendingLine}` : pushed, + data: { + status: "clean", + target: remote.identity, + applied: result.applied, + skipped: result.skipped, + files, + ...(planSql !== undefined ? { sql: planSql } : {}), + mutated_database: result.applied.length > 0, + mutated_files: false, + next_actions: nextActions, + }, + nextActions, + mutatedDatabase: result.applied.length > 0, + mutatedFiles: false, + } satisfies SchemaCommandResult; + }), + ); +}); diff --git a/apps/cli/src/shared/migrations/remote-postgres.ts b/apps/cli/src/shared/migrations/remote-postgres.ts new file mode 100644 index 0000000000..c805bcd809 --- /dev/null +++ b/apps/cli/src/shared/migrations/remote-postgres.ts @@ -0,0 +1,94 @@ +import { Effect, FileSystem, Option, Path } from "effect"; +import type { Pool } from "pg"; +import type { DatabaseTarget } from "../database/database-target.ts"; +import { Output } from "../output/output.service.ts"; +import { SchemaEngineError } from "../schema/schema-errors.ts"; +import { SchemaWorkspace } from "../schema/schema-workspace.service.ts"; +import { MigrationRunner } from "./migration-runner.service.ts"; + +const LOCAL_MAJOR_RESET_NEXT = "supabase db reset"; + +export function formatShadowMajorAlignedMessage( + remoteMajor: number, + previousMajor: number, +): string { + return `Shadow major is now ${remoteMajor} (was ${previousMajor}). The running local database is still ${previousMajor}. Next: ${LOCAL_MAJOR_RESET_NEXT}`; +} + +export function parsePostgresMajor(serverVersion: string | undefined): number | undefined { + if (serverVersion === undefined) return undefined; + const match = /^(\d+)/u.exec(serverVersion.trim()); + if (match?.[1] === undefined) return undefined; + const major = Number(match[1]); + return Number.isInteger(major) ? major : undefined; +} + +export function parseConfigPostgresMajor(toml: string): number | undefined { + const match = /^\s*major_version\s*=\s*(\d+)/mu.exec(toml); + if (match?.[1] === undefined) return undefined; + const major = Number(match[1]); + return Number.isInteger(major) ? major : undefined; +} + +export function alignConfigPostgresMajor( + toml: string, + remoteMajor: number, +): { readonly toml: string; readonly previousMajor: number } | undefined { + const previousMajor = parseConfigPostgresMajor(toml); + if (previousMajor === undefined || previousMajor === remoteMajor) return undefined; + const next = toml.replace(/^(\s*major_version\s*=\s*)\d+/mu, `$1${remoteMajor}`); + if (next === toml) return undefined; + return { toml: next, previousMajor }; +} + +export function generateLocalShadowBanner(major: number | undefined): string { + const pg = major === undefined ? "Postgres" : `PG ${major}`; + return `Compared declarations vs migration replay on a local ${pg} shadow, not the linked project.`; +} + +export const readConfigPostgresMajor = Effect.fnUntraced(function* () { + const workspace = yield* Effect.serviceOption(SchemaWorkspace); + const fs = yield* Effect.serviceOption(FileSystem.FileSystem); + const path = yield* Effect.serviceOption(Path.Path); + if (Option.isNone(workspace) || Option.isNone(fs) || Option.isNone(path)) { + return undefined; + } + const configPath = path.value.join( + path.value.dirname(workspace.value.migrationsDir), + "config.toml", + ); + const toml = yield* fs.value.readFileString(configPath).pipe(Effect.orElseSucceed(() => "")); + return parseConfigPostgresMajor(toml); +}); + +export const assertLocalPostgresMajorMatchesConfig = Effect.fnUntraced(function* (pool: Pool) { + const runner = yield* MigrationRunner; + const liveMajor = parsePostgresMajor(yield* runner.showServerVersion(pool)); + const configMajor = yield* readConfigPostgresMajor(); + if (liveMajor === undefined || configMajor === undefined || liveMajor === configMajor) { + return; + } + return yield* new SchemaEngineError({ + detail: `Local database is PostgreSQL ${liveMajor}; config.toml major_version is ${configMajor}.`, + suggestion: `Run \`${LOCAL_MAJOR_RESET_NEXT}\` so the local container matches config.toml, then retry.`, + }); +}); + +export const warnIfRemotePostgresMajorMismatch = Effect.fnUntraced(function* ( + pool: Pool, + target: DatabaseTarget, +) { + if (target.kind === "local") return; + const runner = yield* Effect.serviceOption(MigrationRunner); + const output = yield* Effect.serviceOption(Output); + if (Option.isNone(runner) || Option.isNone(output)) return; + const remoteVersion = yield* runner.value.showServerVersion(pool); + const remoteMajor = parsePostgresMajor(remoteVersion); + const localMajor = yield* readConfigPostgresMajor(); + if (remoteMajor === undefined || localMajor === undefined || remoteMajor === localMajor) { + return; + } + yield* output.value.warn( + `config.toml major_version is ${localMajor}; the remote database is PostgreSQL ${remoteMajor} (${remoteVersion}). Continuing.`, + ); +}); diff --git a/apps/cli/src/shared/migrations/remote-postgres.unit.test.ts b/apps/cli/src/shared/migrations/remote-postgres.unit.test.ts new file mode 100644 index 0000000000..0f42c37e1d --- /dev/null +++ b/apps/cli/src/shared/migrations/remote-postgres.unit.test.ts @@ -0,0 +1,54 @@ +import { describe, expect, it } from "vitest"; +import { + alignConfigPostgresMajor, + formatShadowMajorAlignedMessage, + generateLocalShadowBanner, + parseConfigPostgresMajor, + parsePostgresMajor, +} from "./remote-postgres.ts"; + +describe("parsePostgresMajor", () => { + it("reads the leading major from SHOW server_version", () => { + expect(parsePostgresMajor("17.6")).toBe(17); + expect(parsePostgresMajor("15.8 (Ubuntu 15.8-1)")).toBe(15); + expect(parsePostgresMajor(undefined)).toBeUndefined(); + expect(parsePostgresMajor("")).toBeUndefined(); + }); +}); + +describe("parseConfigPostgresMajor", () => { + it("reads the first major_version assignment", () => { + expect(parseConfigPostgresMajor("[db]\nmajor_version = 15\n")).toBe(15); + expect(parseConfigPostgresMajor('project_id = "x"\n')).toBeUndefined(); + }); +}); + +describe("alignConfigPostgresMajor", () => { + it("rewrites a differing major_version and leaves matching config alone", () => { + const aligned = alignConfigPostgresMajor("[db]\nmajor_version = 15\n", 17); + expect(aligned).toEqual({ + previousMajor: 15, + toml: "[db]\nmajor_version = 17\n", + }); + expect(alignConfigPostgresMajor("[db]\nmajor_version = 17\n", 17)).toBeUndefined(); + expect(alignConfigPostgresMajor('project_id = "x"\n', 17)).toBeUndefined(); + }); +}); + +describe("formatShadowMajorAlignedMessage", () => { + it("sends the running local container to db reset", () => { + expect(formatShadowMajorAlignedMessage(17, 15)).toBe( + "Shadow major is now 17 (was 15). The running local database is still 15. Next: supabase db reset", + ); + }); +}); + +describe("generateLocalShadowBanner", () => { + it("names a local shadow, not the linked project", () => { + expect(generateLocalShadowBanner(15)).toBe( + "Compared declarations vs migration replay on a local PG 15 shadow, not the linked project.", + ); + expect(generateLocalShadowBanner(undefined)).toContain("local Postgres shadow"); + expect(generateLocalShadowBanner(undefined)).toContain("not the linked project"); + }); +}); diff --git a/apps/cli/src/shared/output/normalize-error.ts b/apps/cli/src/shared/output/normalize-error.ts index bd89a4c1d5..3e3b306c38 100644 --- a/apps/cli/src/shared/output/normalize-error.ts +++ b/apps/cli/src/shared/output/normalize-error.ts @@ -9,6 +9,8 @@ type NormalizedCliError = { readonly message: string; readonly detail?: string; readonly suggestion?: string; + readonly sql?: string; + readonly files?: ReadonlyArray; }; type ErrorRecord = Record; @@ -180,11 +182,15 @@ export function normalizeCliError( // `Fprintln(os.Stderr, CmdSuggestion)`). `readString` would trim exactly // that away. const suggestion = readRawString(error, "suggestion"); + const sql = readRawString(error, "sql"); + const files = error["files"]; return { code, message, ...(detail && detail !== message ? { detail } : {}), ...(suggestion !== undefined && suggestion.length > 0 ? { suggestion } : {}), + ...(sql !== undefined && sql.length > 0 ? { sql } : {}), + ...(Array.isArray(files) ? { files } : {}), }; } diff --git a/apps/cli/src/shared/schema/apply-schema.integration.test.ts b/apps/cli/src/shared/schema/apply-schema.integration.test.ts new file mode 100644 index 0000000000..21ecd81054 --- /dev/null +++ b/apps/cli/src/shared/schema/apply-schema.integration.test.ts @@ -0,0 +1,458 @@ +import { describe, expect, it } from "@effect/vitest"; +import { classifyPlanHazards, type Plan } from "@supabase/pg-delta/plan"; +import { Cause, Effect, Exit, FileSystem, Layer, Option, Path } from "effect"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { formatHistoryConflict } from "../migrations/migration-repair-suggest.ts"; +import { MigrationRepository } from "../migrations/migration-repository.service.ts"; +import { MigrationRunner } from "../migrations/migration-runner.service.ts"; +import { applySchema } from "./apply-schema.ts"; +import { SchemaEngineError, SchemaHistoryConflictError } from "./schema-errors.ts"; +import { digestVersions } from "./schema-digest.ts"; +import { PgDeltaSchemaEngine } from "./pg-delta-engine.service.ts"; +import { SchemaStateStore } from "./schema-state.service.ts"; +import type { SchemaApplyOutcome, SchemaDraftJournal, SchemaPlanView } from "./schema-types.ts"; +import { SchemaWorkspace } from "./schema-workspace.service.ts"; + +function emptyPlan(): Plan { + return { + formatVersion: 1, + engineVersion: "0.3.0", + planId: "plan-1", + source: { fingerprint: "source-fingerprint" }, + target: { fingerprint: "desired-fingerprint" }, + preamble: [], + deltas: [], + filteredDeltas: [], + renameCandidates: [], + actions: [], + safetyReport: { + destructiveActions: 0, + rewriteRiskActions: 0, + nonTransactionalActions: 0, + lockClasses: {}, + }, + }; +} + +function planView( + changes: boolean, + extras: Partial> = {}, +): SchemaPlanView { + const plan = emptyPlan(); + return { + planId: plan.planId, + sourceFingerprint: plan.source.fingerprint, + desiredFingerprint: plan.target.fingerprint, + engineVersion: plan.engineVersion, + profile: "supabase", + changes, + files: [], + hazards: { + kinds: [], + destructive: 0, + rewrite: 0, + coverageGaps: 0, + report: classifyPlanHazards(plan), + }, + destructive: false, + renameCandidates: [], + acceptedRenames: [], + coverageBlocked: extras.coverageBlocked ?? false, + renameBlocked: extras.renameBlocked ?? false, + diagnostics: extras.diagnostics ?? [], + plan, + }; +} + +const localFile = { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: "select 1;", + transactional: true, +}; + +const ungeneratedJournal: SchemaDraftJournal = { + version: 1, + draftId: "draft", + targetIdentity: "local:default", + startingMigrationHeadDigest: digestVersions([localFile.version]), + sourceFingerprint: "s", + engineVersion: "0.3.0", + declarativelyAhead: true, + generated: false, + plans: [], +}; + +function setup( + opts: { + journal?: SchemaDraftJournal; + history?: ReadonlyArray<{ version: string; name: string }>; + files?: ReadonlyArray; + catalogMatch?: boolean; + installedExtensions?: ReadonlyArray; + liveServerVersion?: string; + configMajor?: number; + failApplying?: boolean; + planChanges?: boolean; + plan?: Partial>; + applyPlan?: SchemaApplyOutcome; + } = {}, +) { + const out = mockOutput({ interactive: false }); + let applyPending = 0; + let marked = 0; + let journaled = false; + const liveApplied: string[] = []; + const recorded = new Set((opts.history ?? []).map((row) => row.version)); + const configLayers = + opts.configMajor === undefined + ? Layer.empty + : Layer.mergeAll( + Path.layer, + FileSystem.layerNoop({ + readFileString: () => Effect.succeed(`[db]\nmajor_version = ${opts.configMajor}\n`), + }), + ); + return { + get applyPending() { + return applyPending; + }, + liveApplied, + get marked() { + return marked; + }, + get journaled() { + return journaled; + }, + layer: Layer.mergeAll( + configLayers, + out.layer, + Layer.succeed( + DatabaseTargetResolver, + DatabaseTargetResolver.of({ + resolve: () => + Effect.succeed({ + kind: "local", + identity: "local:default", + connectionString: "postgresql://postgres:postgres@127.0.0.1:54322/postgres", + disposable: true, + durable: false, + connectionVerified: true, + }), + }), + ), + Layer.succeed( + SchemaWorkspace, + SchemaWorkspace.of({ + schemasDir: "/tmp/schemas", + schemasDirDisplay: "supabase/schemas", + migrationsDir: "/tmp/migrations", + migrationsDirDisplay: "supabase/migrations", + customDir: "/tmp/schemas/_custom", + journalPath: "/tmp/j", + lockPath: "/tmp/l", + readDeclarationFiles: Effect.succeed([]), + readExistingSql: () => Effect.succeed(new Map()), + classifyProposed: () => Effect.die("unused"), + installExport: () => Effect.die("unused"), + }), + ), + Layer.succeed( + SchemaStateStore, + SchemaStateStore.of({ + readJournal: Effect.succeed( + opts.journal === undefined ? Option.none() : Option.some(opts.journal), + ), + writeJournal: () => + Effect.sync(() => { + journaled = true; + }), + clearJournal: Effect.void, + withLock: (effect) => effect, + }), + ), + Layer.succeed( + MigrationRepository, + MigrationRepository.of({ + listLocal: Effect.succeed(opts.files ?? [localFile]), + createEmpty: () => Effect.die("unused"), + writeFetched: () => Effect.die("unused"), + writeGenerated: () => Effect.die("unused"), + remove: () => Effect.die("unused"), + }), + ), + Layer.succeed( + MigrationRunner, + MigrationRunner.of({ + listRemote: () => Effect.succeed(opts.history ?? []), + listRemoteStatements: () => Effect.succeed([]), + showServerVersion: () => Effect.succeed(opts.liveServerVersion), + listInstalledExtensions: () => Effect.succeed(opts.installedExtensions ?? []), + applyPending: (pool, files) => + Effect.gen(function* () { + applyPending += 1; + const conn = pool.options.connectionString ?? ""; + if (opts.failApplying === true && !conn.includes("54322")) { + return yield* new SchemaEngineError({ + detail: 'Failed applying migration: extension "pgjwt" already exists', + suggestion: "Check the database connection and migration SQL, then retry.", + }); + } + const leftover = files.filter((file) => !recorded.has(file.version)); + const remoteOnly = [...recorded].filter( + (version) => !files.some((file) => file.version === version), + ); + if (remoteOnly.length > 0 && leftover.length > 0) { + return yield* new SchemaHistoryConflictError( + formatHistoryConflict({ + remoteOnly, + pending: leftover.map((file) => file.version), + flags: { local: true }, + }), + ); + } + if (conn.includes("54322")) { + liveApplied.splice(0, liveApplied.length, ...leftover.map((file) => file.version)); + } + return { + applied: leftover.map((file) => file.version), + skipped: files + .filter((file) => recorded.has(file.version)) + .map((file) => file.version), + }; + }), + markApplied: (_pool, files) => + Effect.sync(() => { + marked += 1; + for (const file of files) recorded.add(file.version); + }), + }), + ), + Layer.succeed( + PgDeltaSchemaEngine, + PgDeltaSchemaEngine.of({ + exportSchema: () => Effect.die("unused"), + planFiles: () => Effect.succeed(planView(opts.planChanges === true, opts.plan ?? {})), + diffPools: () => Effect.succeed(planView(opts.catalogMatch !== true)), + applyPlan: () => + Effect.succeed( + opts.applyPlan ?? { + partial: false, + report: { + status: "applied", + appliedActions: 0, + actionStatuses: [], + }, + }, + ), + provisionPlatform: Effect.succeed({ + url: "postgresql://postgres:postgres@127.0.0.1:1/postgres", + }), + provisionShadow: Effect.succeed({ + url: "postgresql://postgres:postgres@127.0.0.1:1/postgres", + }), + provisionMigrations: Effect.die("unused"), + }), + ), + ), + }; +} + +describe("applySchema", () => { + it.live("runs pending SQL when the live catalog does not match full replay", () => { + const ctx = setup(); + return Effect.gen(function* () { + const result = yield* applySchema().pipe(Effect.provide(ctx.layer)); + expect(result.status).toBe("clean"); + expect(ctx.applyPending).toBe(2); + expect(ctx.marked).toBe(0); + expect(ctx.journaled).toBe(false); + expect(ctx.liveApplied).toEqual([localFile.version]); + }); + }); + + it.live("marks history when the live catalog already matches full replay", () => { + const ctx = setup({ catalogMatch: true }); + return Effect.gen(function* () { + const result = yield* applySchema().pipe(Effect.provide(ctx.layer)); + expect(result.status).toBe("clean"); + expect(result.message).toContain("Recorded"); + expect(ctx.marked).toBe(1); + }); + }); + + it.live("skips file apply while an ungenerated draft is active", () => { + const ctx = setup({ journal: ungeneratedJournal }); + return Effect.gen(function* () { + const result = yield* applySchema().pipe(Effect.provide(ctx.layer)); + expect(result.status).toBe("clean"); + expect(ctx.applyPending).toBe(0); + expect(ctx.marked).toBe(0); + }); + }); + + it.live("fails closed when migration files change during an ungenerated draft", () => { + const ctx = setup({ + journal: { + ...ungeneratedJournal, + startingMigrationHeadDigest: "not-the-current-head", + }, + }); + return Effect.gen(function* () { + const exit = yield* applySchema().pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(ctx.applyPending).toBe(0); + }); + }); + + it.live("names the unmodeled object when planning is blocked", () => { + const ctx = setup({ + history: [{ version: localFile.version, name: localFile.name }], + plan: { + coverageBlocked: true, + diagnostics: [ + { + code: "unmodeled_kind", + severity: "warning", + message: + '1 unmodeled "cast" object not managed by this engine (e.g. public.widget AS integer)', + context: { kind: "cast", count: 1, samples: ["public.widget AS integer"] }, + }, + ], + }, + }); + return Effect.gen(function* () { + const exit = yield* applySchema().pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : Option.none(); + expect(failure._tag).toBe("Some"); + if (failure._tag === "Some") { + expect(failure.value).toMatchObject({ + _tag: "SchemaPlanningBlockedError", + detail: expect.stringContaining("public.widget AS integer"), + suggestion: expect.stringContaining("--debug"), + }); + } + expect(ctx.journaled).toBe(false); + }); + }); + + it.live("names the failing SQL when apply stops partway", () => { + const ctx = setup({ + history: [{ version: localFile.version, name: localFile.name }], + planChanges: true, + applyPlan: { + partial: true, + report: { + status: "failed", + appliedActions: 0, + actionStatuses: ["unapplied"], + error: { + actionIndex: 7, + sql: 'DROP EXTENSION "pgcrypto"', + message: "cannot drop extension pgcrypto because other objects depend on it", + }, + }, + }, + }); + return Effect.gen(function* () { + const exit = yield* applySchema().pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : Option.none(); + expect(failure._tag).toBe("Some"); + if (failure._tag === "Some") { + expect(failure.value).toMatchObject({ + _tag: "SchemaPartialApplyError", + detail: expect.stringMatching( + /cannot drop extension pgcrypto because other objects depend on it[\s\S]*DROP EXTENSION "pgcrypto"/, + ), + suggestion: expect.stringContaining("supabase db reset"), + }); + expect(failure.value).toMatchObject({ + detail: expect.not.stringMatching(/plan|segment|in-doubt/i), + suggestion: expect.not.stringMatching(/plan|segment|in-doubt|repair/i), + }); + } + expect(ctx.journaled).toBe(true); + }); + }); + + it.live("fails closed when the prefix scan cannot replay pending SQL", () => { + const ctx = setup({ failApplying: true, planChanges: true }); + return Effect.gen(function* () { + const exit = yield* applySchema().pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : Option.none(); + expect(failure._tag).toBe("Some"); + if (failure._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaEngineError); + expect(failure.value.detail).toContain("pgjwt"); + } + expect(ctx.journaled).toBe(false); + expect(ctx.liveApplied).toEqual([]); + }); + }); + + it.live("refuses when the local Postgres major does not match config.toml", () => { + const ctx = setup({ + liveServerVersion: "15.8", + configMajor: 17, + planChanges: true, + }); + return Effect.gen(function* () { + const exit = yield* applySchema().pipe(Effect.provide(ctx.layer), Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + expect(JSON.stringify(exit)).toContain("PostgreSQL 15"); + expect(JSON.stringify(exit)).toContain("major_version is 17"); + expect(JSON.stringify(exit)).toContain("supabase db reset"); + expect(ctx.journaled).toBe(false); + expect(ctx.applyPending).toBe(0); + }); + }); + + it.live("journals a draft after recording leftover pending that already matches", () => { + const catchupFile = { + version: "20260101000001", + name: "catchup", + fileName: "20260101000001_catchup.sql", + absolutePath: "/tmp/migrations/20260101000001_catchup.sql", + content: 'CREATE EXTENSION "pgjwt" SCHEMA "extensions";', + transactional: true, + }; + const ctx = setup({ + files: [localFile, catchupFile], + history: [{ version: localFile.version, name: localFile.name }], + installedExtensions: ["pgjwt"], + planChanges: true, + }); + return Effect.gen(function* () { + const result = yield* applySchema().pipe(Effect.provide(ctx.layer)); + expect(result.status).toBe("draft"); + expect(ctx.journaled).toBe(true); + expect(ctx.marked).toBe(1); + expect(ctx.liveApplied).toEqual([]); + }); + }); + + it.live("journals a draft after applying declarations to the local database", () => { + const ctx = setup({ + history: [{ version: localFile.version, name: localFile.name }], + planChanges: true, + }); + return Effect.gen(function* () { + const result = yield* applySchema().pipe(Effect.provide(ctx.layer)); + expect(result.status).toBe("draft"); + expect(result.mutatedDatabase).toBe(true); + expect(ctx.journaled).toBe(true); + expect(result.message).toContain( + "Applied locally and journaled. No migration files were written.", + ); + expect(result.nextActions).toEqual([ + "to generate a migration: supabase schema generate --name ", + ]); + }); + }); +}); diff --git a/apps/cli/src/shared/schema/apply-schema.ts b/apps/cli/src/shared/schema/apply-schema.ts new file mode 100644 index 0000000000..7784f6ae9d --- /dev/null +++ b/apps/cli/src/shared/schema/apply-schema.ts @@ -0,0 +1,193 @@ +import { Effect } from "effect"; +import { readExportManifest } from "@supabase/pg-delta/frontends"; +import { acquireDatabasePool } from "../database/database-pool.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { applyLocalPending } from "../migrations/apply-local-pending.ts"; +import { assertLocalPostgresMajorMatchesConfig } from "../migrations/remote-postgres.ts"; +import { MigrationRepository } from "../migrations/migration-repository.service.ts"; +import type { MigrationApplyResult } from "../migrations/migration-runner.service.ts"; +import { digestUtf8, digestVersions } from "./schema-digest.ts"; +import { + SchemaDraftConflictError, + SchemaDurableTargetError, + SchemaPartialApplyError, + SchemaWorkspaceIoError, +} from "./schema-errors.ts"; +import { formatNextAction, formatShadowLoadAssist, withPlanSummary } from "./schema-output.ts"; +import { assertPlanActionable } from "./schema-plan-gate.ts"; +import { SchemaStateStore } from "./schema-state.service.ts"; +import type { SchemaCommandResult, SchemaDraftJournal } from "./schema-types.ts"; +import { SchemaWorkspace } from "./schema-workspace.service.ts"; +import { PgDeltaSchemaEngine } from "./pg-delta-engine.service.ts"; + +export const applySchema = Effect.fn("schema.apply")(function* () { + const workspace = yield* SchemaWorkspace; + const state = yield* SchemaStateStore; + const engine = yield* PgDeltaSchemaEngine; + const targets = yield* DatabaseTargetResolver; + const migrations = yield* MigrationRepository; + + const target = yield* targets.resolve({ kind: "local" }); + if (!target.disposable) { + return yield* new SchemaDurableTargetError({ + detail: "schema apply can only mutate a verified local disposable database.", + suggestion: + "Start the local stack and rerun, or use schema generate + migrations push for durable targets.", + }); + } + + const declarations = yield* workspace.readDeclarationFiles; + const localMigrations = yield* migrations.listLocal; + + return yield* state.withLock( + Effect.scoped( + Effect.gen(function* () { + const pool = yield* acquireDatabasePool(target.connectionString); + yield* assertLocalPostgresMajorMatchesConfig(pool); + const existingJournal = yield* state.readJournal; + const ungeneratedDraft = + existingJournal._tag === "Some" && + existingJournal.value.declarativelyAhead && + existingJournal.value.generated !== true; + const pendingResult: MigrationApplyResult = + ungeneratedDraft === true + ? { applied: [], recorded: [], skipped: [] } + : yield* applyLocalPending(pool, localMigrations); + if (ungeneratedDraft) { + const currentHead = digestVersions(localMigrations.map((file) => file.version)); + if (currentHead !== existingJournal.value.startingMigrationHeadDigest) { + return yield* new SchemaDraftConflictError({ + detail: "Migration files changed while a declarative draft is active.", + suggestion: + "Run `supabase schema generate`, reset the local database, or discard the draft.", + }); + } + } + + const shadow = yield* engine.provisionShadow; + const shadowPool = yield* acquireDatabasePool(shadow.url); + const manifest = yield* Effect.try({ + try: () => readExportManifest(workspace.schemasDir), + catch: (cause) => + new SchemaWorkspaceIoError({ + detail: cause instanceof Error ? cause.message : String(cause), + suggestion: "Fix supabase/schemas/.pgdelta-export.json or remove it and retry.", + }), + }); + const plan = yield* engine.planFiles({ + targetPool: pool, + shadowPool, + files: declarations, + allowDrops: true, + ...(manifest !== undefined ? { manifest } : {}), + }); + + yield* assertPlanActionable(plan); + + if (!plan.changes) { + const recorded = pendingResult.recorded ?? []; + const mutatedDatabase = pendingResult.applied.length > 0 || recorded.length > 0; + const parts = [ + ...(recorded.length > 0 + ? [`Recorded ${recorded.length} already-applied migration(s): ${recorded.join(", ")}`] + : []), + ...(pendingResult.applied.length > 0 + ? [ + `Applied ${pendingResult.applied.length} migration(s): ${pendingResult.applied.join(", ")}`, + ] + : []), + ]; + const loadAssist = formatShadowLoadAssist(plan); + const base = + parts.length > 0 ? parts.join(". ") : "Local database already matches declarations."; + return { + status: "clean", + message: loadAssist.length > 0 ? `${base}\n${loadAssist}` : base, + data: { + status: "clean", + target: target.identity, + plan_id: plan.planId, + source_fingerprint: plan.sourceFingerprint, + desired_fingerprint: plan.desiredFingerprint, + applied: pendingResult.applied, + recorded, + mutated_database: mutatedDatabase, + mutated_files: false, + }, + nextActions: [], + mutatedDatabase, + mutatedFiles: false, + } satisfies SchemaCommandResult; + } + + const outcome = yield* engine.applyPlan({ pool, plan }); + const journal: SchemaDraftJournal = { + version: 1, + draftId: crypto.randomUUID(), + targetIdentity: target.identity, + startingMigrationHeadDigest: digestVersions(localMigrations.map((file) => file.version)), + sourceFingerprint: plan.sourceFingerprint, + engineVersion: plan.engineVersion, + declarativelyAhead: true, + generated: false, + plans: [ + { + planId: plan.planId, + targetFingerprint: plan.desiredFingerprint, + acceptedRenames: plan.acceptedRenames, + segmentDigests: plan.files.map((file) => digestUtf8(file.sql)), + hazards: { + kinds: plan.hazards.kinds, + destructive: plan.hazards.destructive, + rewrite: plan.hazards.rewrite, + coverageGaps: plan.hazards.coverageGaps, + }, + actionStatuses: outcome.report.actionStatuses, + outcome: outcome.partial ? "partial" : "applied", + }, + ], + }; + yield* state.writeJournal(journal); + + if (outcome.partial) { + const failed = outcome.report.error; + return yield* new SchemaPartialApplyError({ + detail: + failed === undefined + ? "Could not apply schema changes to the local database." + : `Could not apply schema changes to the local database.\n${failed.message}\n${failed.sql}`, + suggestion: + "The local database may be only partly updated. Run `supabase db reset`, fix the failing change in supabase/schemas, then retry `supabase schema apply`.", + }); + } + + const nextActions = [ + formatNextAction("to generate a migration", "supabase schema generate --name "), + ]; + + return { + status: "draft", + message: withPlanSummary( + "Applied locally and journaled. No migration files were written.", + plan, + ), + data: { + status: "draft", + plan_id: plan.planId, + source_fingerprint: plan.sourceFingerprint, + desired_fingerprint: plan.desiredFingerprint, + hazards: plan.hazards, + target: target.identity, + journaled: true, + mutated_database: true, + mutated_files: false, + next_actions: nextActions, + }, + nextActions, + mutatedDatabase: true, + mutatedFiles: false, + } satisfies SchemaCommandResult; + }), + ), + ); +}); diff --git a/apps/cli/src/shared/schema/clear-draft-journal.ts b/apps/cli/src/shared/schema/clear-draft-journal.ts new file mode 100644 index 0000000000..f23fc82585 --- /dev/null +++ b/apps/cli/src/shared/schema/clear-draft-journal.ts @@ -0,0 +1,16 @@ +import { Effect, FileSystem, Path } from "effect"; +import { SCHEMA_DRAFT_JOURNAL_FILE_NAME } from "./schema-paths.ts"; + +/** Unlink `.supabase/schema-draft.json`. Missing file is success. */ +export const clearDraftJournalFile = ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, +) => + fs + .remove(path.join(workdir, ".supabase", SCHEMA_DRAFT_JOURNAL_FILE_NAME)) + .pipe( + Effect.catchTag("PlatformError", (error) => + error.reason._tag === "NotFound" ? Effect.void : Effect.fail(error), + ), + ); diff --git a/apps/cli/src/shared/schema/clear-draft-journal.unit.test.ts b/apps/cli/src/shared/schema/clear-draft-journal.unit.test.ts new file mode 100644 index 0000000000..0567398a0e --- /dev/null +++ b/apps/cli/src/shared/schema/clear-draft-journal.unit.test.ts @@ -0,0 +1,26 @@ +import { existsSync, mkdirSync, mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { Effect, FileSystem, Path } from "effect"; +import { clearDraftJournalFile } from "./clear-draft-journal.ts"; +import { SCHEMA_DRAFT_JOURNAL_FILE_NAME } from "./schema-paths.ts"; + +describe("clearDraftJournalFile", () => { + it.live("unlinks the draft journal and treats a missing file as success", () => { + const workdir = mkdtempSync(join(tmpdir(), "clear-draft-")); + const journalDir = join(workdir, ".supabase"); + const journalPath = join(journalDir, SCHEMA_DRAFT_JOURNAL_FILE_NAME); + mkdirSync(journalDir, { recursive: true }); + writeFileSync(journalPath, "{}\n"); + + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* clearDraftJournalFile(fs, path, workdir); + expect(existsSync(journalPath)).toBe(false); + yield* clearDraftJournalFile(fs, path, workdir); + }).pipe(Effect.provide(BunServices.layer)); + }); +}); diff --git a/apps/cli/src/shared/schema/declarations-ahead.ts b/apps/cli/src/shared/schema/declarations-ahead.ts new file mode 100644 index 0000000000..053beb7f76 --- /dev/null +++ b/apps/cli/src/shared/schema/declarations-ahead.ts @@ -0,0 +1,19 @@ +import { Effect } from "effect"; +import { SchemaDeclarationsAheadError } from "./schema-errors.ts"; +import { SchemaStateStore } from "./schema-state.service.ts"; + +export const assertNoUngeneratedDraft = Effect.fnUntraced(function* () { + const state = yield* SchemaStateStore; + const journal = yield* state.readJournal; + if ( + journal._tag === "Some" && + journal.value.declarativelyAhead && + journal.value.generated !== true + ) { + return yield* new SchemaDeclarationsAheadError({ + detail: "A declarative draft is ahead of the local migration head.", + suggestion: + "Run `supabase schema generate --name `, reset the local database, or discard the draft before `supabase migrations push`.", + }); + } +}); diff --git a/apps/cli/src/shared/schema/generate-schema.integration.test.ts b/apps/cli/src/shared/schema/generate-schema.integration.test.ts new file mode 100644 index 0000000000..2d7651b6f9 --- /dev/null +++ b/apps/cli/src/shared/schema/generate-schema.integration.test.ts @@ -0,0 +1,528 @@ +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { describe, expect, it } from "@effect/vitest"; +import { classifyPlanHazards, type Plan } from "@supabase/pg-delta/plan"; +import { Cause, Effect, Exit, Layer, Option } from "effect"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import type { SchemaPlanFilesInput } from "./pg-delta-engine.service.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { MigrationRepository } from "../migrations/migration-repository.service.ts"; +import { MigrationRunner } from "../migrations/migration-runner.service.ts"; +import { digestVersions } from "./schema-digest.ts"; +import { generateSchema } from "./generate-schema.ts"; +import { renderSchemaResult } from "./schema-render.ts"; +import { PgDeltaSchemaEngine } from "./pg-delta-engine.service.ts"; +import { + SchemaBaselineMigrationsExistError, + SchemaEngineError, + SchemaLinkedConnectionError, +} from "./schema-errors.ts"; +import { SchemaStateStore } from "./schema-state.service.ts"; +import type { SchemaDraftJournal, SchemaPlanView } from "./schema-types.ts"; +import { SchemaWorkspace } from "./schema-workspace.service.ts"; + +function emptyPlan(): Plan { + return { + formatVersion: 1, + engineVersion: "0.3.0", + planId: "plan-1", + source: { fingerprint: "source-fingerprint" }, + target: { fingerprint: "desired-fingerprint" }, + preamble: [], + deltas: [], + filteredDeltas: [], + renameCandidates: [], + actions: [], + safetyReport: { + destructiveActions: 0, + rewriteRiskActions: 0, + nonTransactionalActions: 0, + lockClasses: {}, + }, + }; +} + +function planView( + changes: boolean, + extras: Partial> & { + readonly hazards?: Partial; + } = {}, +): SchemaPlanView { + const plan = emptyPlan(); + return { + planId: plan.planId, + sourceFingerprint: plan.source.fingerprint, + desiredFingerprint: plan.target.fingerprint, + engineVersion: plan.engineVersion, + profile: "supabase", + changes, + files: changes + ? [ + { + sequence: 1, + suffix: null, + sql: "create table t (id int);", + transactional: true, + actionCount: 1, + }, + ] + : [], + hazards: { + kinds: extras.hazards?.kinds ?? [], + destructive: extras.hazards?.destructive ?? 0, + rewrite: extras.hazards?.rewrite ?? 0, + coverageGaps: extras.hazards?.coverageGaps ?? 0, + report: extras.hazards?.report ?? classifyPlanHazards(plan), + }, + destructive: false, + renameCandidates: [], + acceptedRenames: [], + coverageBlocked: extras.coverageBlocked ?? false, + renameBlocked: extras.renameBlocked ?? false, + diagnostics: extras.diagnostics ?? [], + plan, + }; +} + +const draftJournal: SchemaDraftJournal = { + version: 1, + draftId: "draft", + targetIdentity: "local:default", + startingMigrationHeadDigest: digestVersions(["20260101000000"]), + sourceFingerprint: "s", + engineVersion: "0.3.0", + declarativelyAhead: true, + generated: false, + plans: [], +}; + +function setup( + opts: { + changes?: boolean; + journal?: SchemaDraftJournal; + write?: boolean; + localMigrations?: "seeded" | "empty"; + remoteHistory?: ReadonlyArray<{ version: string; name: string }>; + verifySql?: string; + linked?: boolean; + schemasDir?: string; + plan?: Partial> & { + readonly hazards?: Partial; + }; + } = {}, +) { + const out = mockOutput({ interactive: false }); + let cleared = false; + let planCalls = 0; + let wrote = false; + let shadowProvisions = 0; + const planInputs: SchemaPlanFilesInput[] = []; + const layer = Layer.mergeAll( + out.layer, + Layer.succeed( + SchemaWorkspace, + SchemaWorkspace.of({ + schemasDir: opts.schemasDir ?? "/tmp/schemas", + schemasDirDisplay: "supabase/schemas", + migrationsDir: "/tmp/migrations", + migrationsDirDisplay: "supabase/migrations", + customDir: "/tmp/schemas/_custom", + journalPath: "/tmp/j", + lockPath: "/tmp/l", + readDeclarationFiles: Effect.succeed([{ name: "a.sql", sql: "create table a (id int);" }]), + readExistingSql: () => Effect.succeed(new Map()), + classifyProposed: () => Effect.die("unused"), + installExport: () => Effect.die("unused"), + }), + ), + Layer.succeed( + SchemaStateStore, + SchemaStateStore.of({ + readJournal: Effect.succeed( + opts.journal === undefined ? Option.none() : Option.some(opts.journal), + ), + writeJournal: () => Effect.void, + clearJournal: Effect.sync(() => { + cleared = true; + }), + withLock: (effect) => effect, + }), + ), + Layer.succeed( + MigrationRepository, + MigrationRepository.of({ + listLocal: Effect.succeed( + opts.localMigrations === "empty" + ? [] + : [ + { + version: "20260101000000", + name: "init", + fileName: "20260101000000_init.sql", + absolutePath: "/tmp/migrations/20260101000000_init.sql", + content: "select 1;", + transactional: true, + }, + ], + ), + createEmpty: () => Effect.die("unused"), + writeFetched: () => Effect.die("unused"), + writeGenerated: () => + Effect.sync(() => { + wrote = true; + }).pipe( + Effect.andThen( + opts.write === true + ? Effect.succeed([ + { + version: "20260101000001", + name: "schema", + fileName: "20260101000001_schema.sql", + absolutePath: "/tmp/migrations/20260101000001_schema.sql", + content: "create table t (id int);", + transactional: true, + }, + ]) + : Effect.die("unused"), + ), + ), + remove: () => Effect.void, + }), + ), + Layer.succeed( + DatabaseTargetResolver, + DatabaseTargetResolver.of({ + resolve: () => + opts.linked === true || opts.remoteHistory !== undefined + ? Effect.succeed({ + kind: "linked" as const, + identity: "abcdefghijklmnop", + connectionString: "postgresql://postgres:secret@db.example/postgres", + disposable: false, + durable: true, + connectionVerified: true, + projectRef: "abcdefghijklmnop", + }) + : Effect.fail( + new SchemaLinkedConnectionError({ + detail: "This project is not linked to a Supabase project.", + suggestion: "Run `supabase link`.", + }), + ), + }), + ), + Layer.succeed( + MigrationRunner, + MigrationRunner.of({ + listRemote: () => Effect.succeed(opts.remoteHistory ?? []), + listRemoteStatements: () => Effect.succeed([]), + showServerVersion: () => Effect.succeed(undefined), + listInstalledExtensions: () => Effect.die("unused"), + applyPending: () => Effect.die("unused"), + markApplied: () => Effect.die("unused"), + }), + ), + Layer.succeed( + PgDeltaSchemaEngine, + PgDeltaSchemaEngine.of({ + exportSchema: () => Effect.die("unused"), + planFiles: (input) => + Effect.sync(() => { + planInputs.push(input); + planCalls += 1; + if (opts.write === true) { + if (opts.verifySql !== undefined && planCalls > 1) { + return { + ...planView(true), + files: [ + { + sequence: 1, + suffix: null, + sql: opts.verifySql, + transactional: true, + actionCount: 1, + }, + ], + }; + } + return planView(planCalls === 1); + } + return planView(opts.changes === true, opts.plan ?? {}); + }), + diffPools: () => Effect.die("unused"), + applyPlan: () => Effect.die("unused"), + provisionShadow: Effect.sync(() => { + shadowProvisions += 1; + return { + url: "postgresql://postgres:postgres@127.0.0.1:1/postgres", + }; + }), + provisionPlatform: Effect.die("unused"), + provisionMigrations: Effect.sync(() => { + shadowProvisions += 1; + return { + url: "postgresql://postgres:postgres@127.0.0.1:1/postgres", + }; + }), + }), + ), + ); + return { + layer, + out, + get cleared() { + return cleared; + }, + get wrote() { + return wrote; + }, + get shadowProvisions() { + return shadowProvisions; + }, + planInputs, + }; +} + +describe("generateSchema", () => { + it.live("plans without a local database target and never records history", () => { + const ctx = setup({ changes: false }); + return Effect.gen(function* () { + const result = yield* generateSchema({ dryRun: true, baseline: false }).pipe( + Effect.provide(ctx.layer), + ); + expect(result.mutatedDatabase).toBe(false); + expect(result.mutatedFiles).toBe(false); + expect(result.message).toContain("not the linked project"); + expect(ctx.cleared).toBe(false); + }); + }); + + it.live("forwards export loadOrder into planFiles", () => { + const schemasDir = mkdtempSync(join(tmpdir(), "schema-generate-manifest-")); + const loadOrder = ["public/tables/t.sql", "_cluster/publications.sql"]; + writeFileSync( + join(schemasDir, ".pgdelta-export.json"), + JSON.stringify({ + formatVersion: 1, + redactSecrets: true, + scope: "database", + loadOrder, + }), + ); + const ctx = setup({ changes: false, schemasDir }); + return Effect.gen(function* () { + yield* generateSchema({ dryRun: true, baseline: false }).pipe(Effect.provide(ctx.layer)); + expect(ctx.planInputs).toHaveLength(1); + expect(ctx.planInputs[0]?.manifest?.loadOrder).toEqual(loadOrder); + }).pipe( + Effect.ensuring(Effect.sync(() => rmSync(schemasDir, { recursive: true, force: true }))), + ); + }); + + it.live("clears a leftover draft when generate finds no changes", () => { + const ctx = setup({ changes: false, journal: draftJournal }); + return Effect.gen(function* () { + const result = yield* generateSchema({ dryRun: false, baseline: false }).pipe( + Effect.provide(ctx.layer), + ); + expect(result.status).toBe("clean"); + expect(result.message).toContain("not the linked project"); + expect(result.mutatedFiles).toBe(false); + expect(ctx.cleared).toBe(true); + }); + }); + + it.live("fails closed when migration files change during an ungenerated draft", () => { + const ctx = setup({ + journal: { + ...draftJournal, + startingMigrationHeadDigest: "not-the-current-head", + }, + }); + return Effect.gen(function* () { + const exit = yield* generateSchema({ dryRun: true, baseline: false }).pipe( + Effect.provide(ctx.layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + expect(ctx.cleared).toBe(false); + }); + }); + + it.live("points a dry-run with changes at the write command", () => { + const ctx = setup({ changes: true, plan: { hazards: { destructive: 1 } } }); + return Effect.gen(function* () { + const result = yield* generateSchema({ dryRun: true, baseline: false }); + expect(result.status).toBe("needs_approval"); + expect(result.message).toContain("Dry-run; nothing was written."); + expect(result.message).toContain("1 statement"); + expect(result.message).toContain("Hazards:"); + expect(result.message).not.toContain("create table t"); + expect("body" in result ? result.body : undefined).toBe("create table t (id int);"); + expect(result.data).toEqual( + expect.objectContaining({ + sql: "create table t (id int);", + files: [expect.objectContaining({ sql: "create table t (id int);" })], + hazards: expect.objectContaining({ destructive: 1 }), + }), + ); + expect(result.message).toContain( + "Compared declarations vs migration replay on a local Postgres shadow, not the linked project.", + ); + expect(result.nextActions).toEqual([ + "to write the migration: supabase schema generate --name ", + ]); + yield* renderSchemaResult("Generate schema migrations", result); + expect(ctx.out.stdoutText).toContain("create table t (id int);"); + expect(ctx.out.rawChunks).toHaveLength(1); + }).pipe(Effect.provide(ctx.layer)); + }); + + it.live("clears the draft journal after writing files", () => { + const ctx = setup({ write: true, journal: draftJournal }); + return Effect.gen(function* () { + const result = yield* generateSchema({ dryRun: false, baseline: false }).pipe( + Effect.provide(ctx.layer), + ); + expect(result.mutatedFiles).toBe(true); + expect(result.mutatedDatabase).toBe(false); + expect(ctx.cleared).toBe(true); + expect(result.message).toContain("Wrote supabase/migrations/20260101000001_schema.sql"); + expect(result.message).not.toContain("plan-1"); + expect(result.message).not.toContain("source-fingerprint"); + expect(result.nextActions).toEqual(["to deploy: supabase migrations push"]); + expect(result.nextActions.join("\n")).not.toContain("migration repair"); + expect(result.data).toEqual( + expect.objectContaining({ + sql: "create table t (id int);", + files: [expect.objectContaining({ sql: "create table t (id int);" })], + }), + ); + }); + }); + + it.live("suggests migration repair after writing a baseline", () => { + const ctx = setup({ write: true, localMigrations: "empty" }); + return Effect.gen(function* () { + const result = yield* generateSchema({ dryRun: false, baseline: true }).pipe( + Effect.provide(ctx.layer), + ); + expect(result.nextActions.join("\n")).toContain( + "supabase migration repair --status applied 20260101000001", + ); + }); + }); + + it.live("names privilege-only leftover when generated files do not converge", () => { + const ctx = setup({ + write: true, + verifySql: "GRANT EXECUTE ON FUNCTION public.accept_invitation() TO service_role;", + }); + return Effect.gen(function* () { + const exit = yield* generateSchema({ dryRun: false, baseline: false }).pipe( + Effect.provide(ctx.layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : Option.none(); + expect(failure._tag).toBe("Some"); + if (failure._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaEngineError); + expect(JSON.stringify(exit)).toContain("privileges only"); + expect(JSON.stringify(exit)).toContain("schema pull --force"); + expect(JSON.stringify(exit)).not.toContain("did not converge"); + } + expect(ctx.wrote).toBe(true); + }); + }); + + it.live("fails closed when --baseline runs against remote history", () => { + const ctx = setup({ + localMigrations: "empty", + linked: true, + remoteHistory: [{ version: "20260101000000", name: "alice" }], + }); + return Effect.gen(function* () { + const exit = yield* generateSchema({ dryRun: false, baseline: true }).pipe( + Effect.provide(ctx.layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : Option.none(); + expect(failure._tag).toBe("Some"); + if (failure._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaBaselineMigrationsExistError); + expect(JSON.stringify(exit)).toContain("migrations pull --from linked"); + expect(JSON.stringify(exit)).toContain("schema pull --from linked"); + expect(JSON.stringify(exit)).not.toContain("migration repair --status applied"); + } + expect(ctx.wrote).toBe(false); + expect(ctx.shadowProvisions).toBe(0); + }); + }); + + it.live("fails closed when --baseline runs against existing migration files", () => { + const ctx = setup(); + return Effect.gen(function* () { + const exit = yield* generateSchema({ dryRun: false, baseline: true }).pipe( + Effect.provide(ctx.layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : Option.none(); + expect(failure._tag).toBe("Some"); + if (failure._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaBaselineMigrationsExistError); + expect(failure.value._tag).toBe("SchemaBaselineMigrationsExistError"); + } + expect(ctx.wrote).toBe(false); + expect(ctx.shadowProvisions).toBe(0); + expect(ctx.cleared).toBe(false); + }); + }); + + it.live("names coverage objects on dry-run without failing closed", () => { + const ctx = setup({ + plan: { + coverageBlocked: true, + diagnostics: [ + { + code: "unmodeled_kind", + severity: "warning", + message: + '1 unmodeled "cast" object not managed by this engine (e.g. public.widget AS integer)', + context: { kind: "cast", count: 1, samples: ["public.widget AS integer"] }, + }, + ], + }, + }); + return Effect.gen(function* () { + const result = yield* generateSchema({ dryRun: true, baseline: false }).pipe( + Effect.provide(ctx.layer), + ); + expect(result.status).toBe("clean"); + expect(result.message).toContain("1 unmodeled cast (public.widget AS integer)"); + expect(ctx.cleared).toBe(false); + }); + }); + + it.live("fails closed when dry-run --baseline runs against existing migration files", () => { + const ctx = setup(); + return Effect.gen(function* () { + const exit = yield* generateSchema({ dryRun: true, baseline: true }).pipe( + Effect.provide(ctx.layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + const failure = Exit.isFailure(exit) ? Cause.findErrorOption(exit.cause) : Option.none(); + expect(failure._tag).toBe("Some"); + if (failure._tag === "Some") { + expect(failure.value).toBeInstanceOf(SchemaBaselineMigrationsExistError); + expect(failure.value._tag).toBe("SchemaBaselineMigrationsExistError"); + } + expect(ctx.wrote).toBe(false); + expect(ctx.shadowProvisions).toBe(0); + }); + }); +}); diff --git a/apps/cli/src/shared/schema/generate-schema.ts b/apps/cli/src/shared/schema/generate-schema.ts new file mode 100644 index 0000000000..20fad8dfb5 --- /dev/null +++ b/apps/cli/src/shared/schema/generate-schema.ts @@ -0,0 +1,252 @@ +import { Clock, Effect } from "effect"; +import { readExportManifest } from "@supabase/pg-delta/frontends"; +import { acquireDatabasePool } from "../database/database-pool.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { + classifyPrivilegePlan, + PRIVILEGE_REFRESH_SUGGESTION, +} from "../migrations/privilege-offer.ts"; +import { MigrationRepository } from "../migrations/migration-repository.service.ts"; +import { MigrationRunner } from "../migrations/migration-runner.service.ts"; +import { formatMigrationRepairCommand } from "../migrations/migration-repair-suggest.ts"; +import { + generateLocalShadowBanner, + readConfigPostgresMajor, +} from "../migrations/remote-postgres.ts"; +import { formatPlanSql } from "./schema-body.ts"; +import { digestVersions } from "./schema-digest.ts"; +import { + SchemaBaselineMigrationsExistError, + SchemaDraftConflictError, + SchemaEngineError, + SchemaLinkedConnectionError, + SchemaTargetRequiredError, + SchemaWorkspaceIoError, +} from "./schema-errors.ts"; +import { formatMigrationFilePath, formatNextAction, withPlanSummary } from "./schema-output.ts"; +import { assertPlanActionable } from "./schema-plan-gate.ts"; +import { SchemaStateStore } from "./schema-state.service.ts"; +import type { SchemaCommandResult } from "./schema-types.ts"; +import { SchemaWorkspace } from "./schema-workspace.service.ts"; +import { PgDeltaSchemaEngine } from "./pg-delta-engine.service.ts"; + +export type GenerateSchemaInput = { + readonly name?: string; + readonly dryRun: boolean; + readonly baseline: boolean; +}; + +export const generateSchema = Effect.fn("schema.generate")(function* (input: GenerateSchemaInput) { + const workspace = yield* SchemaWorkspace; + const state = yield* SchemaStateStore; + const engine = yield* PgDeltaSchemaEngine; + const migrations = yield* MigrationRepository; + + const declarations = yield* workspace.readDeclarationFiles; + const localMigrations = yield* migrations.listLocal; + const name = input.name ?? (input.baseline ? "initial_schema" : "schema"); + const nextName = input.name ?? (input.baseline ? "initial_schema" : ""); + const banner = generateLocalShadowBanner(yield* readConfigPostgresMajor()); + + if (input.baseline && localMigrations.length > 0) { + return yield* new SchemaBaselineMigrationsExistError({ + detail: `--baseline cannot run because ${workspace.migrationsDirDisplay} already has files.`, + suggestion: + "supabase schema generate --dry-run to preview, or supabase schema generate --name to add a change. --baseline is only for empty migration history.", + }); + } + + if (input.baseline) { + const targets = yield* DatabaseTargetResolver; + const runner = yield* MigrationRunner; + const linked = yield* targets.resolve({ kind: "linked" }).pipe( + Effect.catchIf( + (error): error is SchemaLinkedConnectionError | SchemaTargetRequiredError => + error._tag === "SchemaLinkedConnectionError" || + error._tag === "SchemaTargetRequiredError", + () => Effect.succeed(undefined), + ), + ); + if (linked !== undefined) { + const remoteHistory = yield* Effect.scoped( + Effect.gen(function* () { + const pool = yield* acquireDatabasePool(linked.connectionString); + return yield* runner.listRemote(pool); + }), + ); + if (remoteHistory.length > 0) { + return yield* new SchemaBaselineMigrationsExistError({ + detail: `--baseline cannot run because the linked database already has migration history.`, + suggestion: + "Remote has migration history. Run supabase migrations pull --from linked, then supabase schema pull --from linked.", + }); + } + } + } + + return yield* state.withLock( + Effect.scoped( + Effect.gen(function* () { + const journal = yield* state.readJournal; + if ( + journal._tag === "Some" && + journal.value.declarativelyAhead && + journal.value.generated !== true + ) { + const currentHead = digestVersions(localMigrations.map((file) => file.version)); + if (currentHead !== journal.value.startingMigrationHeadDigest) { + return yield* new SchemaDraftConflictError({ + detail: "Migration files changed while a declarative draft is active.", + suggestion: + "Reset the local database, discard the draft, or restore the migration files from before schema apply.", + }); + } + } + + const sourceShadow = input.baseline + ? yield* engine.provisionShadow + : yield* engine.provisionMigrations; + const desiredShadow = yield* engine.provisionShadow; + + const sourcePool = yield* acquireDatabasePool(sourceShadow.url); + const desiredPool = yield* acquireDatabasePool(desiredShadow.url); + + const manifest = yield* Effect.try({ + try: () => readExportManifest(workspace.schemasDir), + catch: (cause) => + new SchemaWorkspaceIoError({ + detail: cause instanceof Error ? cause.message : String(cause), + suggestion: "Fix supabase/schemas/.pgdelta-export.json or remove it and retry.", + }), + }); + const plan = yield* engine.planFiles({ + targetPool: sourcePool, + shadowPool: desiredPool, + files: declarations, + allowDrops: true, + ...(manifest !== undefined ? { manifest } : {}), + }); + + if (!input.dryRun) { + yield* assertPlanActionable(plan); + } + + if (input.dryRun || !plan.changes) { + if (!input.dryRun && !plan.changes) { + yield* state.clearJournal; + } + const nextActions = plan.changes + ? [ + formatNextAction( + "to write the migration", + `supabase schema generate --name ${nextName}`, + ), + ] + : []; + const sql = formatPlanSql(plan); + const statusLine = plan.changes + ? "Dry-run; nothing was written." + : "Declarations already match migration replay."; + return { + status: plan.changes ? "needs_approval" : "clean", + message: `${withPlanSummary(statusLine, plan)}\n${banner}`, + ...(plan.changes && sql.length > 0 ? { body: sql } : {}), + data: { + status: plan.changes ? "needs_approval" : "clean", + plan_id: plan.planId, + source_fingerprint: plan.sourceFingerprint, + desired_fingerprint: plan.desiredFingerprint, + hazards: plan.hazards, + files_written: [], + sql, + files: plan.files, + mutated_database: false, + mutated_files: false, + next_actions: nextActions, + }, + nextActions, + mutatedDatabase: false, + mutatedFiles: false, + } satisfies SchemaCommandResult; + } + + const written = yield* migrations.writeGenerated({ + name, + baseMillis: yield* Clock.currentTimeMillis, + files: plan.files.map((file) => ({ + suffix: file.suffix, + sql: file.sql, + transactional: file.transactional, + })), + }); + + const persistGenerated = Effect.gen(function* () { + const verifyShadow = yield* engine.provisionMigrations; + const verifySource = yield* acquireDatabasePool(verifyShadow.url); + const verifyDesired = yield* engine.provisionShadow; + const verifyDesiredPool = yield* acquireDatabasePool(verifyDesired.url); + const verify = yield* engine.planFiles({ + targetPool: verifySource, + shadowPool: verifyDesiredPool, + files: declarations, + allowDrops: true, + ...(manifest !== undefined ? { manifest } : {}), + }); + if (verify.changes) { + const aheadKind = yield* classifyPrivilegePlan(verify); + return yield* new SchemaEngineError({ + detail: + aheadKind === "not_acl" + ? "Generated migrations did not converge to the declared schema." + : "Generated migrations differ from declarations by privileges only.", + suggestion: + aheadKind === "not_acl" + ? "Inspect the generated files and rerun schema generate." + : PRIVILEGE_REFRESH_SUGGESTION, + }); + } + + yield* state.clearJournal; + + const nextActions = input.baseline + ? [ + formatNextAction( + "to record it as applied", + formatMigrationRepairCommand({ + status: "applied", + versions: written.map((file) => file.version), + }), + ), + ] + : [formatNextAction("to deploy", "supabase migrations push")]; + + return { + status: "generated", + message: `${withPlanSummary( + `Wrote ${written.map((file) => formatMigrationFilePath(file.fileName)).join(", ")}`, + plan, + )}\n${banner}`, + data: { + status: "generated", + plan_id: plan.planId, + source_fingerprint: plan.sourceFingerprint, + desired_fingerprint: plan.desiredFingerprint, + hazards: plan.hazards, + files_written: written.map((file) => file.fileName), + sql: formatPlanSql(plan), + files: plan.files, + mutated_database: false, + mutated_files: true, + next_actions: nextActions, + }, + nextActions, + mutatedDatabase: false, + mutatedFiles: true, + } satisfies SchemaCommandResult; + }); + + return yield* persistGenerated.pipe(Effect.tapError(() => migrations.remove(written))); + }), + ), + ); +}); diff --git a/apps/cli/src/shared/schema/isolated-shadow.service.ts b/apps/cli/src/shared/schema/isolated-shadow.service.ts new file mode 100644 index 0000000000..0195851530 --- /dev/null +++ b/apps/cli/src/shared/schema/isolated-shadow.service.ts @@ -0,0 +1,18 @@ +import type { Effect, Scope } from "effect"; +import { Context } from "effect"; +import type { SchemaEngineError } from "./schema-errors.ts"; +import type { SchemaShadow } from "./schema-shadow.ts"; + +interface IsolatedShadowProvisionerShape { + /** Platform baseline with webhooks disabled; image extensions stay installed. */ + readonly provision: Effect.Effect; + /** Platform baseline with no project migrations; webhooks follow config. */ + readonly provisionPlatform: Effect.Effect; + /** Platform-baselined Docker shadow with local migration files applied. */ + readonly provisionMigrations: Effect.Effect; +} + +export class IsolatedShadowProvisioner extends Context.Service< + IsolatedShadowProvisioner, + IsolatedShadowProvisionerShape +>()("supabase/schema/IsolatedShadowProvisioner") {} diff --git a/apps/cli/src/shared/schema/pg-delta-engine.layer.ts b/apps/cli/src/shared/schema/pg-delta-engine.layer.ts new file mode 100644 index 0000000000..fa1e430867 --- /dev/null +++ b/apps/cli/src/shared/schema/pg-delta-engine.layer.ts @@ -0,0 +1,199 @@ +import { Effect, Layer } from "effect"; +import type { Pool } from "pg"; +import { apply } from "@supabase/pg-delta/apply"; +import { encodeId, serializeSnapshot, type Diagnostic } from "@supabase/pg-delta/core"; +import { + buildSchemaExport, + dataLossActions, + hasBlockingDiagnostics, + planSchemaFiles, + renderPlanFiles, + ShadowLoadError, +} from "@supabase/pg-delta/frontends"; +import { resolveProfile, supabaseProfile } from "@supabase/pg-delta/integrations"; +import { + classifyPlanHazards, + ENGINE_VERSION, + plan as planCatalogs, + type Plan, +} from "@supabase/pg-delta/plan"; +import { IsolatedShadowProvisioner } from "./isolated-shadow.service.ts"; +import { SchemaEngineError } from "./schema-errors.ts"; +import { + PgDeltaSchemaEngine, + type SchemaDiffPoolsInput, + type SchemaExportResult, + type SchemaPlanFilesInput, +} from "./pg-delta-engine.service.ts"; +import { + filesForDeclarativeShadowLoad, + prepareDeclarativeShadow, +} from "./prepare-declarative-shadow.ts"; +import { schemaIsolatedPlanOptions } from "./schema-plan-options.ts"; +import { formatSchemaSql, SCHEMA_SQL_FORMAT_DEFAULTS } from "./sql-format-defaults.ts"; +import type { SchemaHazardSummary, SchemaPlanView, SchemaRenderedFile } from "./schema-types.ts"; + +const engineError = (detail: string, suggestion = "Inspect diagnostics and retry.") => + new SchemaEngineError({ detail, suggestion }); + +const engineCause = (cause: unknown, suggestion: string) => { + if (cause instanceof ShadowLoadError) { + const details = cause.details.map((diagnostic) => ` - ${diagnostic.message}`).join("\n"); + return engineError( + details.length > 0 ? `${cause.message}\n${details}` : cause.message, + suggestion, + ); + } + return engineError(cause instanceof Error ? cause.message : String(cause), suggestion); +}; + +function toPlanView( + thePlan: Plan, + allowDrops: boolean, + diagnostics: ReadonlyArray, +): SchemaPlanView { + const rendered = renderPlanFiles(thePlan, { allowDrops }); + const files: Array = rendered.files.map((file, index) => ({ + sequence: index + 1, + suffix: file.suffix, + sql: formatSchemaSql(file.contents, SCHEMA_SQL_FORMAT_DEFAULTS), + transactional: file.transactional, + actionCount: file.actionCount, + })); + const hazards = classifyPlanHazards(thePlan, diagnostics); + const destructive = dataLossActions(thePlan.actions).length; + const summary: SchemaHazardSummary = { + kinds: [...hazards.kinds], + destructive, + rewrite: hazards.kinds.includes("rewrite_risk") ? 1 : 0, + coverageGaps: hazards.coverage.length, + report: hazards, + }; + const renameBlocked = thePlan.renameCandidates.some( + (candidate) => candidate.status === "ambiguous", + ); + return { + planId: thePlan.planId, + sourceFingerprint: thePlan.source.fingerprint, + desiredFingerprint: thePlan.target.fingerprint, + engineVersion: thePlan.engineVersion, + profile: thePlan.profile?.id ?? "supabase", + changes: rendered.changes, + files, + hazards: summary, + destructive: destructive > 0, + renameCandidates: thePlan.renameCandidates.map((candidate) => ({ + from: encodeId(candidate.from), + to: encodeId(candidate.to), + })), + acceptedRenames: (thePlan.acceptedRenames ?? []).map((rename) => ({ + from: encodeId(rename.from), + to: encodeId(rename.to), + })), + coverageBlocked: + hasBlockingDiagnostics(diagnostics, { strictCoverage: true }) || hazards.coverage.length > 0, + renameBlocked, + diagnostics, + plan: thePlan, + }; +} + +export const pgDeltaSchemaEngineLayer = Layer.effect( + PgDeltaSchemaEngine, + Effect.gen(function* () { + const shadows = yield* IsolatedShadowProvisioner; + return PgDeltaSchemaEngine.of({ + exportSchema: (pool: Pool) => + Effect.tryPromise({ + try: async (): Promise => { + const exported = await buildSchemaExport(pool, { + profile: supabaseProfile, + scope: "database", + redactSecrets: true, + format: SCHEMA_SQL_FORMAT_DEFAULTS, + }); + const ctx = await resolveProfile(pool, supabaseProfile, { redactSecrets: true }); + const extracted = await ctx.extract(pool, { redactSecrets: true }); + return { + files: exported.files.map((file) => ({ name: file.name, sql: file.sql })), + manifest: { + ...exported.manifest, + files: exported.files.map((file) => file.name).sort(), + }, + snapshot: serializeSnapshot(extracted.factBase, { + pgVersion: extracted.pgVersion, + redactSecrets: true, + profile: ctx.id, + }), + engineVersion: ENGINE_VERSION, + }; + }, + catch: (cause) => + engineCause(cause, "Confirm the database is reachable and retry schema pull."), + }), + planFiles: (input: SchemaPlanFilesInput) => + Effect.gen(function* () { + yield* prepareDeclarativeShadow(input.shadowPool, input.files); + return yield* Effect.tryPromise({ + try: async () => { + const result = await planSchemaFiles( + input.targetPool, + input.shadowPool, + [...filesForDeclarativeShadowLoad(input.files)], + { + ...schemaIsolatedPlanOptions, + ...(input.manifest !== undefined ? { manifest: input.manifest } : {}), + }, + ); + return toPlanView(result.plan, input.allowDrops ?? true, [ + ...result.loadDiagnostics, + ...result.targetDiagnostics, + ...result.driftDiagnostics, + ]); + }, + catch: (cause) => engineCause(cause, "Fix declaration or coverage issues, then retry."), + }); + }), + diffPools: (input: SchemaDiffPoolsInput) => + Effect.tryPromise({ + try: async () => { + const profile = await resolveProfile(input.sourcePool, supabaseProfile, { + redactSecrets: true, + }); + const source = await profile.extract(input.sourcePool, { redactSecrets: true }); + const desired = await profile.extract(input.desiredPool, { redactSecrets: true }); + const generated = planCatalogs(source.factBase, desired.factBase, { + ...profile.planOptions, + redactSecrets: true, + }); + return toPlanView(generated, input.allowDrops ?? true, [ + ...source.diagnostics, + ...desired.diagnostics, + ]); + }, + catch: (cause) => engineCause(cause, "Confirm both databases are reachable and retry."), + }), + applyPlan: (input) => + Effect.tryPromise({ + try: async () => { + const report = await apply(input.plan.plan, input.pool, { + fingerprintGate: true, + ...input.applyOptions, + }); + return { + report, + partial: + report.status === "failed" || + report.actionStatuses.some( + (status) => status === "inDoubt" || status === "unapplied", + ), + }; + }, + catch: (cause) => engineCause(cause, "Retry `supabase schema apply`."), + }), + provisionShadow: shadows.provision, + provisionPlatform: shadows.provisionPlatform, + provisionMigrations: shadows.provisionMigrations, + }); + }), +); diff --git a/apps/cli/src/shared/schema/pg-delta-engine.service.ts b/apps/cli/src/shared/schema/pg-delta-engine.service.ts new file mode 100644 index 0000000000..dc34ba8d53 --- /dev/null +++ b/apps/cli/src/shared/schema/pg-delta-engine.service.ts @@ -0,0 +1,56 @@ +import type { Effect, Scope } from "effect"; +import { Context } from "effect"; +import type { Pool } from "pg"; +import type { ApplyOptions } from "@supabase/pg-delta/apply"; +import type { ExportManifest } from "@supabase/pg-delta/frontends"; +import type { SchemaApplyOutcome, SchemaPlanView, SchemaSqlFile } from "./schema-types.ts"; +import type { SchemaEngineError } from "./schema-errors.ts"; +import type { SchemaShadow } from "./schema-shadow.ts"; + +export type SchemaExportResult = { + readonly files: ReadonlyArray; + readonly manifest: ExportManifest & { readonly files: ReadonlyArray }; + readonly snapshot: string; + readonly engineVersion: string; +}; + +export type SchemaPlanFilesInput = { + readonly targetPool: Pool; + readonly shadowPool: Pool; + readonly files: ReadonlyArray; + readonly manifest?: ExportManifest; + readonly allowDrops?: boolean; +}; + +export type SchemaDiffPoolsInput = { + readonly sourcePool: Pool; + readonly desiredPool: Pool; + readonly allowDrops?: boolean; +}; + +type SchemaApplyPlanInput = { + readonly pool: Pool; + readonly plan: SchemaPlanView; + readonly applyOptions?: ApplyOptions; +}; + +interface PgDeltaSchemaEngineShape { + readonly exportSchema: (pool: Pool) => Effect.Effect; + readonly planFiles: ( + input: SchemaPlanFilesInput, + ) => Effect.Effect; + readonly diffPools: ( + input: SchemaDiffPoolsInput, + ) => Effect.Effect; + readonly applyPlan: ( + input: SchemaApplyPlanInput, + ) => Effect.Effect; + readonly provisionShadow: Effect.Effect; + readonly provisionPlatform: Effect.Effect; + readonly provisionMigrations: Effect.Effect; +} + +export class PgDeltaSchemaEngine extends Context.Service< + PgDeltaSchemaEngine, + PgDeltaSchemaEngineShape +>()("supabase/schema/PgDeltaSchemaEngine") {} diff --git a/apps/cli/src/shared/schema/prepare-declarative-shadow.ts b/apps/cli/src/shared/schema/prepare-declarative-shadow.ts new file mode 100644 index 0000000000..991bdc98a8 --- /dev/null +++ b/apps/cli/src/shared/schema/prepare-declarative-shadow.ts @@ -0,0 +1,155 @@ +import { Effect } from "effect"; +import { SchemaEngineError } from "./schema-errors.ts"; +import type { SchemaSqlFile } from "./schema-types.ts"; + +export type DeclarativeShadowClient = { + readonly query: (sql: string) => Promise<{ readonly rows: ReadonlyArray }>; +}; + +/** Image-default extensions the user may still declare; omit means keep the install. */ +const IMAGE_DEFAULT_EXTENSIONS = ["pgjwt", "pgcrypto", "uuid-ossp"] as const; + +const IMAGE_DEFAULT_EXTENSION_SET = new Set(IMAGE_DEFAULT_EXTENSIONS); + +const DROP_IMAGE_DEFAULT_EXTENSION: Record<(typeof IMAGE_DEFAULT_EXTENSIONS)[number], string> = { + pgjwt: "DROP EXTENSION IF EXISTS pgjwt", + pgcrypto: "DROP EXTENSION IF EXISTS pgcrypto", + "uuid-ossp": 'DROP EXTENSION IF EXISTS "uuid-ossp"', +}; + +const CREATE_EXTENSION_RE = + /\bCREATE\s+EXTENSION\s+(?:IF\s+NOT\s+EXISTS\s+)?(?:"([^"]+)"|([a-zA-Z_][\w$-]*))/gi; + +/** Blank comments and literals so CREATE EXTENSION in those positions is ignored. */ +const maskSqlNonCode = (sql: string): string => + sql.replaceAll( + /--[^\r\n]*|\/\*[\s\S]*?\*\/|'(?:''|[^'])*'|\$(?:[a-zA-Z_][\w$]*)?\$[\s\S]*?\$(?:[a-zA-Z_][\w$]*)?\$/g, + (matched) => matched.replaceAll(/[^\r\n]/g, " "), + ); + +export const declaredSqlExtensions = ( + files: ReadonlyArray<{ readonly name: string; readonly sql: string }>, +): ReadonlySet => { + const declared = new Set(); + for (const file of files) { + for (const match of maskSqlNonCode(file.sql).matchAll(CREATE_EXTENSION_RE)) { + const name = (match[1] ?? match[2] ?? "").toLowerCase(); + if (name !== "") declared.add(name); + } + } + return declared; +}; + +const stripAllowedExtensionCatchup = (sql: string): string => + maskSqlNonCode(sql) + .replace(/\bSET\s+[^;]*;/giu, "") + .replace(/\bCREATE\s+EXTENSION\s+[^;]*;/giu, "") + .replace(/\bCOMMENT\s+ON\s+EXTENSION\s+[^;]*;/giu, "") + .replace(/\s+/gu, " ") + .trim(); + +/** First-push catchup that only recreates image-default extensions. */ +export function isImageExtensionCatchupSql(sql: string): boolean { + const declared = declaredSqlExtensions([{ name: "catchup.sql", sql }]); + if (declared.size === 0) return false; + if ([...declared].some((name) => !IMAGE_DEFAULT_EXTENSION_SET.has(name))) return false; + return stripAllowedExtensionCatchup(sql) === ""; +} + +/** True when that catchup is already installed on the live catalog. */ +export function imageExtensionCatchupAlreadyPresent( + sql: string, + installed: ReadonlySet, +): boolean { + if (!isImageExtensionCatchupSql(sql)) return false; + const declared = declaredSqlExtensions([{ name: "catchup.sql", sql }]); + return [...declared].every((name) => installed.has(name)); +} + +const declaredImageExtensions = (files: ReadonlyArray): ReadonlySet => { + const declared = new Set(); + for (const name of declaredSqlExtensions(files)) { + if (IMAGE_DEFAULT_EXTENSION_SET.has(name)) declared.add(name); + } + return declared; +}; + +export const parsePostgresMajorVersion = (serverVersion: string): number => { + const major = Number.parseInt(serverVersion, 10); + return Number.isInteger(major) ? major : 0; +}; + +export const declarativeBaselinePrepStatements = ( + majorVersion: number, + declared: ReadonlySet, +): ReadonlyArray => { + const dropPgcrypto = declared.has("pgcrypto"); + // Image pgjwt depends on pgcrypto; drop it first so pgcrypto can drop. + const dropPgjwt = declared.has("pgjwt") || dropPgcrypto; + const dropUuidOssp = declared.has("uuid-ossp"); + const statements: string[] = []; + if (majorVersion === 14 && dropUuidOssp) { + statements.push("ALTER TABLE storage.objects ALTER COLUMN id DROP DEFAULT"); + } + if (dropPgjwt) statements.push(DROP_IMAGE_DEFAULT_EXTENSION.pgjwt); + if (dropPgcrypto) statements.push(DROP_IMAGE_DEFAULT_EXTENSION.pgcrypto); + if (dropUuidOssp) statements.push(DROP_IMAGE_DEFAULT_EXTENSION["uuid-ossp"]); + return statements; +}; + +/** Recreate image pgjwt after a pgcrypto-only drop so omit still means keep. */ +export const filesForDeclarativeShadowLoad = ( + files: ReadonlyArray, +): ReadonlyArray => { + const declared = declaredImageExtensions(files); + if (!declared.has("pgcrypto") || declared.has("pgjwt")) return files; + return [ + ...files, + { + name: "_cli/restore-pgjwt.sql", + sql: "CREATE EXTENSION IF NOT EXISTS pgjwt WITH SCHEMA extensions;\n", + }, + ]; +}; + +/** User cannot edit this SQL; a persistent miss is a CLI bug. */ +const DECLARATIVE_SHADOW_PREP_FAILURE_SUGGESTION = + "This statement is CLI-owned shadow prep, not a project migration or schema file. If it persists, report it with supabase issue bug."; + +const queryError = (sql: string, cause: unknown) => + new SchemaEngineError({ + detail: `Failed to prepare the isolated declaration shadow (${sql}): ${ + cause instanceof Error ? cause.message : String(cause) + }`, + suggestion: DECLARATIVE_SHADOW_PREP_FAILURE_SUGGESTION, + }); + +const readServerVersion = (rows: ReadonlyArray): string => { + const row = rows[0]; + if (row === undefined || typeof row !== "object" || row === null) return ""; + const value = Reflect.get(row, "server_version"); + return typeof value === "string" ? value : ""; +}; + +export const prepareDeclarativeShadow = ( + client: DeclarativeShadowClient, + files: ReadonlyArray, +) => + Effect.gen(function* () { + const declared = declaredImageExtensions(files); + if (declared.size === 0) return; + const versionRows = yield* Effect.tryPromise({ + try: () => client.query("SHOW server_version"), + catch: (cause) => queryError("SHOW server_version", cause), + }); + const statements = declarativeBaselinePrepStatements( + parsePostgresMajorVersion(readServerVersion(versionRows.rows)), + declared, + ); + for (const sql of statements) { + yield* Effect.tryPromise({ + try: () => client.query(sql), + catch: (cause) => queryError(sql, cause), + }); + } + }); diff --git a/apps/cli/src/shared/schema/prepare-declarative-shadow.unit.test.ts b/apps/cli/src/shared/schema/prepare-declarative-shadow.unit.test.ts new file mode 100644 index 0000000000..7b35970a86 --- /dev/null +++ b/apps/cli/src/shared/schema/prepare-declarative-shadow.unit.test.ts @@ -0,0 +1,174 @@ +import { describe, expect, it } from "@effect/vitest"; +import { Cause, Effect, Exit, Option } from "effect"; +import { + declaredSqlExtensions, + declarativeBaselinePrepStatements, + filesForDeclarativeShadowLoad, + imageExtensionCatchupAlreadyPresent, + isImageExtensionCatchupSql, + parsePostgresMajorVersion, + prepareDeclarativeShadow, +} from "./prepare-declarative-shadow.ts"; + +const allImageCreates = [ + { name: "_cluster/extensions/pgjwt.sql", sql: 'CREATE EXTENSION "pgjwt";' }, + { name: "_cluster/extensions/pgcrypto.sql", sql: 'CREATE EXTENSION "pgcrypto";' }, + { name: "_cluster/extensions/uuid-ossp.sql", sql: 'CREATE EXTENSION "uuid-ossp";' }, +]; + +describe("declarativeBaselinePrepStatements", () => { + it("emits nothing when declarations do not recreate image defaults", () => { + expect(declarativeBaselinePrepStatements(14, new Set())).toEqual([]); + expect(declarativeBaselinePrepStatements(17, new Set())).toEqual([]); + }); + + it("detaches PG14 storage.objects before dropping declared uuid-ossp", () => { + expect(declarativeBaselinePrepStatements(14, new Set(["uuid-ossp"]))).toEqual([ + "ALTER TABLE storage.objects ALTER COLUMN id DROP DEFAULT", + 'DROP EXTENSION IF EXISTS "uuid-ossp"', + ]); + }); + + it("drops image pgjwt before a declared pgcrypto recreate", () => { + expect(declarativeBaselinePrepStatements(14, new Set(["pgcrypto"]))).toEqual([ + "DROP EXTENSION IF EXISTS pgjwt", + "DROP EXTENSION IF EXISTS pgcrypto", + ]); + expect(declarativeBaselinePrepStatements(17, new Set(["pgcrypto"]))).toEqual([ + "DROP EXTENSION IF EXISTS pgjwt", + "DROP EXTENSION IF EXISTS pgcrypto", + ]); + }); + + it("drops image pgjwt when migration SQL recreates it", () => { + expect( + declarativeBaselinePrepStatements( + 17, + declaredSqlExtensions([{ name: "catchup.sql", sql: "create extension pgjwt;" }]), + ), + ).toEqual(["DROP EXTENSION IF EXISTS pgjwt"]); + }); + + it("drops declared image defaults on PG15+, pgjwt before pgcrypto", () => { + expect( + declarativeBaselinePrepStatements(17, new Set(["pgjwt", "pgcrypto", "uuid-ossp"])), + ).toEqual([ + "DROP EXTENSION IF EXISTS pgjwt", + "DROP EXTENSION IF EXISTS pgcrypto", + 'DROP EXTENSION IF EXISTS "uuid-ossp"', + ]); + }); +}); + +describe("filesForDeclarativeShadowLoad", () => { + it("restores omitted pgjwt after a pgcrypto recreate", () => { + const files = [{ name: "public/01.sql", sql: "CREATE EXTENSION pgcrypto;" }]; + expect(filesForDeclarativeShadowLoad(files)).toEqual([ + ...files, + { + name: "_cli/restore-pgjwt.sql", + sql: "CREATE EXTENSION IF NOT EXISTS pgjwt WITH SCHEMA extensions;\n", + }, + ]); + }); + + it("does not restore pgjwt when declarations recreate it", () => { + expect(filesForDeclarativeShadowLoad(allImageCreates)).toEqual(allImageCreates); + }); +}); + +describe("isImageExtensionCatchupSql", () => { + const aliceCatchup = `SET local check_function_bodies = off; + +CREATE EXTENSION "pgjwt" SCHEMA "extensions"; + +COMMENT ON EXTENSION "pgjwt" IS 'JSON Web Token API for Postgresql'; +`; + + it("accepts first-push image-extension catchup", () => { + expect(isImageExtensionCatchupSql(aliceCatchup)).toBe(true); + expect(imageExtensionCatchupAlreadyPresent(aliceCatchup, new Set(["pgjwt"]))).toBe(true); + expect(imageExtensionCatchupAlreadyPresent(aliceCatchup, new Set())).toBe(false); + }); + + it("rejects catchup that also changes schema objects", () => { + expect( + isImageExtensionCatchupSql(`${aliceCatchup}\nCREATE TABLE public.todos (id int);\n`), + ).toBe(false); + }); + + it("rejects a non-image extension", () => { + expect(isImageExtensionCatchupSql('CREATE EXTENSION "postgis";')).toBe(false); + }); +}); + +describe("parsePostgresMajorVersion", () => { + it("reads the leading major from SHOW server_version", () => { + expect(parsePostgresMajorVersion("17.6")).toBe(17); + expect(parsePostgresMajorVersion("14.15 (Debian)")).toBe(14); + expect(parsePostgresMajorVersion("")).toBe(0); + }); +}); + +describe("prepareDeclarativeShadow", () => { + it.live("skips the shadow when declarations omit image-default extensions", () => { + const queries: string[] = []; + const client = { + query: (sql: string) => { + queries.push(sql); + return Promise.resolve({ rows: [] }); + }, + }; + return Effect.gen(function* () { + yield* prepareDeclarativeShadow(client, [{ name: "a.sql", sql: "create table a (id int);" }]); + expect(queries).toEqual([]); + }); + }); + + it.live("names the failing prep statement", () => { + const client = { + query: (sql: string) => { + if (sql === "SHOW server_version") { + return Promise.resolve({ rows: [{ server_version: "15.8" }] }); + } + if (sql.includes("pgcrypto")) { + return Promise.reject(new Error("cannot drop extension pgcrypto (SQLSTATE 2BP01)")); + } + return Promise.resolve({ rows: [] }); + }, + }; + return Effect.gen(function* () { + const exit = yield* prepareDeclarativeShadow(client, [ + { name: "public/01.sql", sql: "CREATE EXTENSION pgcrypto;" }, + ]).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + const error = Exit.isFailure(exit) + ? Option.getOrUndefined(Cause.findErrorOption(exit.cause)) + : undefined; + expect(error !== undefined && "detail" in error ? String(error.detail) : "").toContain( + "DROP EXTENSION IF EXISTS pgcrypto", + ); + }); + }); + + it.live("runs the version-selected prep statements against the shadow", () => { + const queries: string[] = []; + const client = { + query: (sql: string) => { + queries.push(sql); + return Promise.resolve({ + rows: sql === "SHOW server_version" ? [{ server_version: "17.6" }] : [], + }); + }, + }; + return Effect.gen(function* () { + yield* prepareDeclarativeShadow(client, allImageCreates); + expect(queries).toEqual([ + "SHOW server_version", + "DROP EXTENSION IF EXISTS pgjwt", + "DROP EXTENSION IF EXISTS pgcrypto", + 'DROP EXTENSION IF EXISTS "uuid-ossp"', + ]); + }); + }); +}); diff --git a/apps/cli/src/shared/schema/pull-schema.integration.test.ts b/apps/cli/src/shared/schema/pull-schema.integration.test.ts new file mode 100644 index 0000000000..670553849d --- /dev/null +++ b/apps/cli/src/shared/schema/pull-schema.integration.test.ts @@ -0,0 +1,288 @@ +import { describe, expect, it } from "@effect/vitest"; +import { BunServices } from "@effect/platform-bun"; +import { existsSync, mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { Effect, Exit, Layer } from "effect"; +import type { Pool } from "pg"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import type { DatabaseTargetSelector } from "../database/database-target.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { MigrationRepository } from "../migrations/migration-repository.service.ts"; +import { MigrationRunner } from "../migrations/migration-runner.service.ts"; +import { pullSchema } from "./pull-schema.ts"; +import { PgDeltaSchemaEngine } from "./pg-delta-engine.service.ts"; +import { schemaStateLayer } from "./schema-state.layer.ts"; +import { schemaWorkspaceLayer } from "./schema-workspace.layer.ts"; + +function tempProject() { + const root = mkdtempSync(join(tmpdir(), "schema-pull-")); + const supabaseDir = join(root, "supabase"); + const projectHomeDir = join(root, ".supabase"); + mkdirSync(supabaseDir, { recursive: true }); + mkdirSync(projectHomeDir, { recursive: true }); + return { root, supabaseDir, projectHomeDir }; +} + +function mockEngine(files: Array<{ name: string; sql: string }>) { + return Layer.succeed( + PgDeltaSchemaEngine, + PgDeltaSchemaEngine.of({ + exportSchema: (_pool: Pool) => + Effect.succeed({ + files, + manifest: { + redactSecrets: true, + profile: "supabase", + scope: "database", + files: files.map((f) => f.name), + }, + snapshot: '{"catalog":true}', + engineVersion: "0.3.0", + }), + planFiles: () => Effect.die("unused"), + diffPools: () => Effect.die("unused"), + applyPlan: () => Effect.die("unused"), + provisionShadow: Effect.die("unused"), + provisionPlatform: Effect.die("unused"), + provisionMigrations: Effect.die("unused"), + }), + ); +} + +function mockTarget() { + const resolved: DatabaseTargetSelector[] = []; + return { + resolved, + layer: Layer.succeed( + DatabaseTargetResolver, + DatabaseTargetResolver.of({ + resolve: (selector) => + Effect.sync(() => { + resolved.push(selector); + return { + kind: "local" as const, + identity: "local:default", + connectionString: "postgresql://postgres:postgres@127.0.0.1:54322/postgres", + disposable: true, + durable: false, + connectionVerified: true, + }; + }), + }), + ), + }; +} + +function mockMigrations() { + return Layer.succeed( + MigrationRepository, + MigrationRepository.of({ + listLocal: Effect.succeed([]), + createEmpty: () => Effect.die("unused"), + writeFetched: () => Effect.die("unused"), + writeGenerated: () => Effect.die("unused"), + remove: () => Effect.die("unused"), + }), + ); +} + +function mockRunner(history: ReadonlyArray<{ version: string; name: string }> = []) { + return Layer.succeed( + MigrationRunner, + MigrationRunner.of({ + listRemote: () => Effect.succeed(history), + listRemoteStatements: () => Effect.succeed([]), + showServerVersion: () => Effect.succeed(undefined), + listInstalledExtensions: () => Effect.die("unused"), + applyPending: () => Effect.die("unused"), + markApplied: () => Effect.die("unused"), + }), + ); +} + +function setup( + files = [{ name: "public.sql", sql: "create table public.t (id int);\n" }], + opts: { readonly history?: ReadonlyArray<{ version: string; name: string }> } = {}, +) { + const project = tempProject(); + const out = mockOutput({ format: "json", interactive: false }); + const workspace = schemaWorkspaceLayer({ + projectRoot: project.root, + supabaseDir: project.supabaseDir, + projectHomeDir: project.projectHomeDir, + }).pipe(Layer.provide(BunServices.layer)); + const target = mockTarget(); + const layer = Layer.mergeAll( + out.layer, + BunServices.layer, + workspace, + schemaStateLayer.pipe(Layer.provide(workspace), Layer.provide(BunServices.layer)), + mockEngine(files), + target.layer, + mockMigrations(), + mockRunner(opts.history), + ); + return { project, layer, target }; +} + +describe("pullSchema", () => { + it.live("defaults to the local database when --from is omitted", () => { + const { layer, target } = setup(); + return Effect.gen(function* () { + const result = yield* pullSchema({ force: false, pruneUnmanaged: false }).pipe( + Effect.provide(layer), + ); + expect(target.resolved).toEqual([{ kind: "local" }]); + expect(result.nextActions).toEqual([ + "to create a baseline: supabase schema generate --baseline --name initial_schema", + ]); + }); + }); + + it.live("points at migrations pull when the source already has history", () => { + const { layer } = setup([{ name: "public.sql", sql: "create table public.t (id int);\n" }], { + history: [{ version: "20260101000000", name: "alice" }], + }); + return Effect.gen(function* () { + const result = yield* pullSchema({ force: false, pruneUnmanaged: false }).pipe( + Effect.provide(layer), + ); + expect(result.nextActions).toEqual([ + "to fetch missing files: supabase migrations pull --from local", + ]); + expect(result.nextActions.join("\n")).not.toContain("--baseline"); + }); + }); + + it.live("does not paste a connection string into the --output next command", () => { + const { project, layer } = setup(); + return Effect.gen(function* () { + const result = yield* pullSchema({ + from: "postgresql://postgres:secret@db.example/postgres", + output: join(project.root, "snapshot"), + force: false, + pruneUnmanaged: false, + }).pipe(Effect.provide(layer)); + expect(result.nextActions.join("\n")).not.toContain("secret"); + expect(result.nextActions).toEqual([ + "to replace the managed schema: supabase schema pull --from --force", + ]); + }); + }); + + it.live("writes declarations and the export manifest into an empty tree", () => { + const { project, layer } = setup(); + return Effect.gen(function* () { + const result = yield* pullSchema({ from: "local", force: false, pruneUnmanaged: false }).pipe( + Effect.provide(layer), + ); + expect(result.mutatedFiles).toBe(true); + expect(result.data["created"]).toEqual(["public.sql"]); + expect(existsSync(join(project.supabaseDir, "schemas", "public.sql"))).toBe(true); + expect(existsSync(join(project.supabaseDir, "schemas", ".schema-checkpoint.json"))).toBe( + false, + ); + expect(existsSync(join(project.supabaseDir, "schemas", ".pgdelta-export.json"))).toBe(true); + }); + }); + + it.live("fails closed when declarations already exist", () => { + const { project, layer } = setup(); + mkdirSync(join(project.supabaseDir, "schemas"), { recursive: true }); + writeFileSync(join(project.supabaseDir, "schemas", "existing.sql"), "select 1;\n"); + return Effect.gen(function* () { + const exit = yield* pullSchema({ from: "local", force: false, pruneUnmanaged: false }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + }); + }); + + it.live("does not touch _custom when replacing", () => { + const { project, layer } = setup(); + const custom = join(project.supabaseDir, "schemas", "_custom"); + mkdirSync(custom, { recursive: true }); + writeFileSync(join(custom, "hand.sql"), "create cast (int as text);\n"); + return Effect.gen(function* () { + yield* pullSchema({ from: "local", force: true, pruneUnmanaged: false }).pipe( + Effect.provide(layer), + ); + expect(readFileSync(join(custom, "hand.sql"), "utf8")).toBe("create cast (int as text);\n"); + }); + }); + + it.live("leaves unmanaged files on --force and hints _custom/", () => { + const { project, layer } = setup([{ name: "kept.sql", sql: "select 1;\n" }]); + const schemas = join(project.supabaseDir, "schemas"); + mkdirSync(schemas, { recursive: true }); + writeFileSync(join(schemas, "kept.sql"), "select 1;\n"); + writeFileSync(join(schemas, "stray.sql"), "select 2;\n"); + writeFileSync( + join(schemas, ".pgdelta-export.json"), + `${JSON.stringify({ formatVersion: 1, files: ["kept.sql"] }, null, 2)}\n`, + ); + return Effect.gen(function* () { + const result = yield* pullSchema({ from: "local", force: true, pruneUnmanaged: false }).pipe( + Effect.provide(layer), + ); + expect(existsSync(join(schemas, "stray.sql"))).toBe(true); + expect(result.data["unmanaged"]).toEqual(["stray.sql"]); + expect(result.nextActions.join("\n")).toContain("supabase/schemas/_custom/"); + }); + }); + + it.live("prunes unmanaged files when --prune-unmanaged is passed", () => { + const { project, layer } = setup([{ name: "kept.sql", sql: "select 1;\n" }]); + const schemas = join(project.supabaseDir, "schemas"); + mkdirSync(schemas, { recursive: true }); + writeFileSync(join(schemas, "kept.sql"), "select 1;\n"); + writeFileSync(join(schemas, "stray.sql"), "select 2;\n"); + writeFileSync( + join(schemas, ".pgdelta-export.json"), + `${JSON.stringify({ formatVersion: 1, files: ["kept.sql"] }, null, 2)}\n`, + ); + return Effect.gen(function* () { + yield* pullSchema({ from: "local", force: true, pruneUnmanaged: true }).pipe( + Effect.provide(layer), + ); + expect(existsSync(join(schemas, "kept.sql"))).toBe(true); + expect(existsSync(join(schemas, "stray.sql"))).toBe(false); + }); + }); + + it.live("refuses a primary-tree pull while a draft is ahead, without writing files", () => { + const { project, layer } = setup(); + const schemas = join(project.supabaseDir, "schemas"); + mkdirSync(schemas, { recursive: true }); + writeFileSync(join(schemas, "existing.sql"), "select 1;\n"); + writeFileSync( + join(project.projectHomeDir, "schema-draft.json"), + `${JSON.stringify( + { + version: 1, + draftId: "draft-1", + targetIdentity: "local:default", + startingMigrationHeadDigest: "abc", + sourceFingerprint: "def", + plans: [], + engineVersion: "0.3.0", + declarativelyAhead: true, + generated: false, + }, + null, + 2, + )}\n`, + ); + return Effect.gen(function* () { + const exit = yield* pullSchema({ from: "local", force: true, pruneUnmanaged: true }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + expect(readFileSync(join(schemas, "existing.sql"), "utf8")).toBe("select 1;\n"); + expect(existsSync(join(schemas, "public.sql"))).toBe(false); + }); + }); +}); diff --git a/apps/cli/src/shared/schema/pull-schema.ts b/apps/cli/src/shared/schema/pull-schema.ts new file mode 100644 index 0000000000..31d8f8baab --- /dev/null +++ b/apps/cli/src/shared/schema/pull-schema.ts @@ -0,0 +1,131 @@ +import { Effect } from "effect"; +import { readExportManifest } from "@supabase/pg-delta/frontends"; +import { acquireDatabasePool } from "../database/database-pool.ts"; +import { DatabaseTargetResolver } from "../database/database-target.service.ts"; +import { parseTargetSelector, redactConnectionString } from "../database/database-target.ts"; +import { SchemaDraftConflictError } from "./schema-errors.ts"; +import { SchemaStateStore } from "./schema-state.service.ts"; +import type { SchemaCommandResult } from "./schema-types.ts"; +import { SchemaWorkspace } from "./schema-workspace.service.ts"; +import { PgDeltaSchemaEngine } from "./pg-delta-engine.service.ts"; +import { formatFileSummary, formatNextAction } from "./schema-output.ts"; +import { MigrationRepository } from "../migrations/migration-repository.service.ts"; +import { MigrationRunner } from "../migrations/migration-runner.service.ts"; + +export type PullSchemaInput = { + readonly from?: string; + readonly output?: string; + readonly force: boolean; + readonly pruneUnmanaged: boolean; +}; + +export const pullSchema = Effect.fn("schema.pull")(function* (input: PullSchemaInput) { + const workspace = yield* SchemaWorkspace; + const state = yield* SchemaStateStore; + const engine = yield* PgDeltaSchemaEngine; + const targets = yield* DatabaseTargetResolver; + const migrations = yield* MigrationRepository; + const runner = yield* MigrationRunner; + + const selector = parseTargetSelector(input.from ?? "local"); + const target = yield* targets.resolve(selector); + const mode = input.output !== undefined ? "output" : input.force ? "force" : "init"; + + return yield* state.withLock( + Effect.scoped( + Effect.gen(function* () { + if (mode !== "output") { + const journal = yield* state.readJournal; + if ( + journal._tag === "Some" && + journal.value.declarativelyAhead && + journal.value.generated !== true + ) { + return yield* new SchemaDraftConflictError({ + detail: "A declarative draft is active. Pull would hide ungenerated changes.", + suggestion: + "Run `supabase schema generate` or discard the draft before pulling the primary tree.", + }); + } + } + + const pool = yield* acquireDatabasePool(target.connectionString); + const exported = yield* engine.exportSchema(pool); + const installed = yield* workspace.installExport({ + files: exported.files, + manifest: Object.fromEntries(Object.entries(exported.manifest)), + mode, + ...(input.output !== undefined ? { outputDir: input.output } : {}), + pruneUnmanaged: input.pruneUnmanaged, + }); + + const localMigrations = yield* migrations.listLocal; + const remoteHistory = yield* runner.listRemote(pool); + const from = selector.kind === "url" ? "" : selector.kind; + + const summary = installed.classification; + const nextActions = [ + ...(mode === "output" + ? [ + formatNextAction( + "to replace the managed schema", + `supabase schema pull --from ${from} --force`, + ), + ] + : localMigrations.length > 0 + ? [formatNextAction("to check they match", "supabase schema generate --dry-run")] + : remoteHistory.length > 0 + ? [ + formatNextAction( + "to fetch missing files", + `supabase migrations pull --from ${from}`, + ), + ] + : [ + formatNextAction( + "to create a baseline", + "supabase schema generate --baseline --name initial_schema", + ), + ]), + ...(summary.unmanaged.length > 0 + ? [ + formatNextAction( + "to keep hand-authored SQL", + "move those files to supabase/schemas/_custom/", + ), + ] + : []), + ]; + + return { + status: "clean", + message: `Declarative schema written to ${installed.directoryDisplay}.`, + data: { + status: "clean", + source: { + kind: target.kind, + identity: target.identity, + connection: redactConnectionString(target.connectionString), + }, + output: installed.directoryDisplay, + replaced: installed.replaced, + merge: false, + summary: formatFileSummary(summary), + created: summary.created, + updated: summary.updated, + unchanged: summary.unchanged, + removed: summary.removed, + unmanaged: summary.unmanaged, + next_actions: nextActions, + mutated_database: false, + mutated_files: true, + export_manifest: readExportManifest(installed.directory), + }, + nextActions, + mutatedDatabase: false, + mutatedFiles: true, + } satisfies SchemaCommandResult; + }), + ), + ); +}); diff --git a/apps/cli/src/shared/schema/schema-body.ts b/apps/cli/src/shared/schema/schema-body.ts new file mode 100644 index 0000000000..9f3842d522 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-body.ts @@ -0,0 +1,62 @@ +import { formatTableRow } from "../output/table.ts"; + +export type SchemaScriptFile = { + readonly name: string; + readonly version?: string; + readonly status?: string; + readonly sql?: string; +}; + +export type MigrationInventoryStatus = "applied" | "pending" | "remote-only"; + +export type MigrationInventoryRow = { + readonly version: string; + readonly name: string; + readonly status?: MigrationInventoryStatus; +}; + +export function formatPlanSql(plan: { + readonly files: ReadonlyArray<{ readonly sql: string }>; +}): string { + return plan.files.map((file) => file.sql).join("\n\n"); +} + +export function planStatementCount(plan: { + readonly files: ReadonlyArray<{ readonly sql: string }>; + readonly plan?: { readonly actions: ReadonlyArray }; +}): number { + if (plan.plan !== undefined && plan.plan.actions.length > 0) { + return plan.plan.actions.length; + } + return formatPlanSql(plan) + .split(";") + .map((statement) => statement.trim()) + .filter((statement) => statement.length > 0).length; +} + +export function formatMigrationInventory(rows: ReadonlyArray): string { + if (rows.length === 0) return ""; + const withStatus = rows.some((row) => row.status !== undefined); + const cells = rows.map((row) => + withStatus ? [row.version, row.name, row.status ?? ""] : [row.version, row.name], + ); + const colCount = withStatus ? 3 : 2; + const widths = Array.from({ length: colCount }, (_, index) => + Math.max(...cells.map((cell) => cell[index]?.length ?? 0)), + ); + return cells.map((cell) => formatTableRow(cell, widths).trimEnd()).join("\n"); +} + +export function humanTarget( + target: "local" | "linked" | "url" | { readonly kind: "local" | "linked" | "url" }, +): string { + const kind = typeof target === "string" ? target : target.kind; + switch (kind) { + case "local": + return "the local database"; + case "linked": + return "the linked project"; + case "url": + return "the given database"; + } +} diff --git a/apps/cli/src/shared/schema/schema-body.unit.test.ts b/apps/cli/src/shared/schema/schema-body.unit.test.ts new file mode 100644 index 0000000000..b1024548b4 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-body.unit.test.ts @@ -0,0 +1,77 @@ +import { describe, expect, it } from "vitest"; +import { + formatMigrationInventory, + formatPlanSql, + humanTarget, + planStatementCount, +} from "./schema-body.ts"; + +describe("formatPlanSql", () => { + it("joins file sql with a blank line", () => { + expect( + formatPlanSql({ + files: [{ sql: "CREATE TABLE t (id int);" }, { sql: "ALTER TABLE t ADD COLUMN n int;" }], + }), + ).toBe("CREATE TABLE t (id int);\n\nALTER TABLE t ADD COLUMN n int;"); + }); + + it("returns empty when there are no files", () => { + expect(formatPlanSql({ files: [] })).toBe(""); + }); +}); + +describe("planStatementCount", () => { + it("prefers plan.actions when present", () => { + expect( + planStatementCount({ + files: [{ sql: "CREATE TABLE t (id int);" }], + plan: { actions: [{}, {}] }, + }), + ).toBe(2); + }); + + it("counts statements from joined SQL when actions are empty", () => { + expect( + planStatementCount({ + files: [{ sql: "CREATE TABLE t (id int); ALTER TABLE t ADD COLUMN n int;" }], + plan: { actions: [] }, + }), + ).toBe(2); + }); +}); + +describe("formatMigrationInventory", () => { + it("formats version, name, and status as aligned columns", () => { + expect( + formatMigrationInventory([ + { version: "20240101000000", name: "init", status: "applied" }, + { version: "20240102000000", name: "add_users", status: "pending" }, + { version: "20240103000000", name: "from_ci", status: "remote-only" }, + ]), + ).toBe( + [ + "20240101000000 init applied", + "20240102000000 add_users pending", + "20240103000000 from_ci remote-only", + ].join("\n"), + ); + }); + + it("omits the status column when no row has one", () => { + expect(formatMigrationInventory([{ version: "20240101000000", name: "init" }])).toBe( + "20240101000000 init", + ); + }); + + it("returns empty for no rows", () => { + expect(formatMigrationInventory([])).toBe(""); + }); +}); + +describe("humanTarget", () => { + it("names local, linked, and url targets", () => { + expect(humanTarget("local")).toBe("the local database"); + expect(humanTarget({ kind: "linked" })).toBe("the linked project"); + expect(humanTarget({ kind: "url" })).toBe("the given database"); + }); +}); diff --git a/apps/cli/src/shared/schema/schema-digest.ts b/apps/cli/src/shared/schema/schema-digest.ts new file mode 100644 index 0000000000..0a51088462 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-digest.ts @@ -0,0 +1,22 @@ +import { createHash } from "node:crypto"; +import type { SchemaSqlFile } from "./schema-types.ts"; + +export function digestUtf8(value: string): string { + return createHash("sha256").update(value).digest("hex"); +} + +export function digestFileSet(files: ReadonlyArray): string { + const hash = createHash("sha256"); + const sorted = [...files].sort((left, right) => left.name.localeCompare(right.name)); + for (const file of sorted) { + hash.update(file.name); + hash.update("\0"); + hash.update(file.sql); + hash.update("\0"); + } + return hash.digest("hex"); +} + +export function digestVersions(versions: ReadonlyArray): string { + return digestUtf8(versions.join("\n")); +} diff --git a/apps/cli/src/shared/schema/schema-digest.unit.test.ts b/apps/cli/src/shared/schema/schema-digest.unit.test.ts new file mode 100644 index 0000000000..2e050e2902 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-digest.unit.test.ts @@ -0,0 +1,28 @@ +import { describe, expect, it } from "vitest"; +import { digestFileSet, digestUtf8, digestVersions } from "./schema-digest.ts"; + +describe("schema-digest", () => { + it("is stable across file order", () => { + expect( + digestFileSet([ + { name: "b.sql", sql: "select 2" }, + { name: "a.sql", sql: "select 1" }, + ]), + ).toBe( + digestFileSet([ + { name: "a.sql", sql: "select 1" }, + { name: "b.sql", sql: "select 2" }, + ]), + ); + }); + + it("changes when content changes", () => { + expect(digestFileSet([{ name: "a.sql", sql: "select 1" }])).not.toBe( + digestFileSet([{ name: "a.sql", sql: "select 2" }]), + ); + }); + + it("hashes versions and utf8", () => { + expect(digestVersions(["1", "2"])).toBe(digestUtf8("1\n2")); + }); +}); diff --git a/apps/cli/src/shared/schema/schema-ecosystem.ts b/apps/cli/src/shared/schema/schema-ecosystem.ts new file mode 100644 index 0000000000..3a11ddb145 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-ecosystem.ts @@ -0,0 +1,3 @@ +export const SCHEMA_PULL_NO_MERGE_HELP = `supabase/schemas already has SQL. Pull does not merge files. + --output Write a side-by-side copy + --force Replace managed files (_custom/ is left alone)`; diff --git a/apps/cli/src/shared/schema/schema-errors.ts b/apps/cli/src/shared/schema/schema-errors.ts new file mode 100644 index 0000000000..34876eabe8 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-errors.ts @@ -0,0 +1,229 @@ +import { Data } from "effect"; +import { + actionability, + type CliErrorActionabilityDeclaration, + ErrorActionabilityId, +} from "../telemetry/error-actionability.ts"; +import type { SchemaScriptFile } from "./schema-body.ts"; + +function SchemaCliError(tag: Tag) { + return class extends Data.TaggedError(tag)<{ + readonly detail: string; + readonly suggestion: string; + }> { + override get message() { + return this.detail; + } + }; +} + +export class SchemaDeclarationsExistError extends SchemaCliError("SchemaDeclarationsExistError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.invalidInput; + } +} + +export class SchemaUnmanagedFilesError extends Data.TaggedError("SchemaUnmanagedFilesError")<{ + readonly detail: string; + readonly suggestion: string; + readonly paths: ReadonlyArray; +}> { + override get message() { + return this.detail; + } + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return { ...actionability.invalidInput, fingerprint_suffix: "conflict" }; + } +} + +export class SchemaWorkspaceIoError extends SchemaCliError("SchemaWorkspaceIoError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.invalidConfig; + } +} + +export class SchemaLockError extends SchemaCliError("SchemaLockError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.invalidConfig; + } +} + +export class SchemaStateError extends SchemaCliError("SchemaStateError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.invalidConfig; + } +} + +export class SchemaLocalStackNotRunningError extends SchemaCliError( + "SchemaLocalStackNotRunningError", +) { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.startStack; + } +} + +export class SchemaLinkedConnectionError extends SchemaCliError("SchemaLinkedConnectionError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.projectNotLinked; + } +} + +export class SchemaDurableTargetError extends SchemaCliError("SchemaDurableTargetError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.permission; + } +} + +export class SchemaDestructiveAuthError extends SchemaCliError("SchemaDestructiveAuthError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.permission; + } +} + +export class SchemaProjectRefMismatchError extends SchemaCliError("SchemaProjectRefMismatchError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.missingProjectRef; + } +} + +export class SchemaAllowRemoteRequiredError extends SchemaCliError( + "SchemaAllowRemoteRequiredError", +) { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.permission; + } +} + +export class SchemaPlanningBlockedError extends SchemaCliError("SchemaPlanningBlockedError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.dbFinding; + } +} + +export class SchemaDeclarationsAheadError extends Data.TaggedError("SchemaDeclarationsAheadError")<{ + readonly detail: string; + readonly suggestion: string; + readonly sql?: string; + readonly files?: ReadonlyArray; +}> { + override get message() { + return this.detail; + } + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.migrationDrift; + } +} + +export class SchemaRemoteDriftError extends Data.TaggedError("SchemaRemoteDriftError")<{ + readonly detail: string; + readonly suggestion: string; + readonly sql?: string; + readonly files?: ReadonlyArray; +}> { + override get message() { + return this.detail; + } + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.migrationDrift; + } +} + +export class SchemaPrivilegeOfferError extends Data.TaggedError("SchemaPrivilegeOfferError")<{ + readonly detail: string; + readonly suggestion: string; + readonly sql: string; + readonly files?: ReadonlyArray; +}> { + override get message() { + return this.detail; + } + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.migrationDrift; + } +} + +export class SchemaCatalogAdoptError extends Data.TaggedError("SchemaCatalogAdoptError")<{ + readonly detail: string; + readonly suggestion: string; + readonly sql: string; + readonly files?: ReadonlyArray; +}> { + override get message() { + return this.detail; + } + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.migrationDrift; + } +} + +export class SchemaEmptyHistoryReplayError extends SchemaCliError("SchemaEmptyHistoryReplayError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.migrationDrift; + } +} + +export class SchemaEmptyMigrationStatementsError extends SchemaCliError( + "SchemaEmptyMigrationStatementsError", +) { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.invalidConfig; + } +} + +export class SchemaDraftConflictError extends SchemaCliError("SchemaDraftConflictError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.migrationDrift; + } +} + +export class SchemaEngineError extends SchemaCliError("SchemaEngineError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.dbFinding; + } +} + +export class SchemaPartialApplyError extends SchemaCliError("SchemaPartialApplyError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.dbFinding; + } +} + +export class SchemaMigrationNameError extends SchemaCliError("SchemaMigrationNameError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.invalidInput; + } +} + +export class SchemaBaselineMigrationsExistError extends SchemaCliError( + "SchemaBaselineMigrationsExistError", +) { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.invalidInput; + } +} + +export class SchemaHistoryConflictError extends SchemaCliError("SchemaHistoryConflictError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.migrationDrift; + } +} + +export class SchemaTargetRequiredError extends SchemaCliError("SchemaTargetRequiredError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.invalidInput; + } +} + +export class SchemaCancelledError extends SchemaCliError("SchemaCancelledError") { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.cancelled; + } +} + +export class SchemaMigrationsPrivilegeError extends SchemaCliError( + "SchemaMigrationsPrivilegeError", +) { + get [ErrorActionabilityId](): CliErrorActionabilityDeclaration { + return actionability.permission; + } +} diff --git a/apps/cli/src/shared/schema/schema-output.ts b/apps/cli/src/shared/schema/schema-output.ts new file mode 100644 index 0000000000..abe49257a9 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-output.ts @@ -0,0 +1,157 @@ +import { STRICT_COVERAGE_CODES } from "@supabase/pg-delta/frontends"; +import { explicitBooleanLongFlag } from "../cli/cobra-flag-groups.ts"; +import { planStatementCount } from "./schema-body.ts"; +import { MIGRATIONS_DIRECTORY_NAME } from "./schema-paths.ts"; +import type { SchemaFileSummary, SchemaPlanView } from "./schema-types.ts"; + +export type CoverageFormatOptions = { + readonly verbose?: boolean; +}; + +const DEFAULT_COVERAGE_LINES = 3; + +const SHADOW_LOAD_ASSIST_CODES = new Set(["session_pollution", "reorder_on_failure"]); + +function localizeShadowLoadAssist(code: string, message: string): string { + const text = message.replaceAll(".pgdelta-export.json", "supabase/schemas/.pgdelta-export.json"); + if (code !== "session_pollution") return text; + return text + .replace("session poisoning", "supautils session poisoning") + .replace( + /\nRemove session-setting statements from declarative SQL, or do not share that session with later DDL\.$/, + "", + ); +} + +export function formatShadowLoadAssist(plan: SchemaPlanView, opts?: CoverageFormatOptions): string { + if (!coverageVerbose(opts)) return ""; + return plan.diagnostics + .filter((diagnostic) => SHADOW_LOAD_ASSIST_CODES.has(diagnostic.code)) + .map((diagnostic) => localizeShadowLoadAssist(diagnostic.code, diagnostic.message)) + .join("\n"); +} + +export function coverageVerbose(opts?: CoverageFormatOptions): boolean { + return opts?.verbose ?? explicitBooleanLongFlag(process.argv, "debug") === true; +} + +function coverageDiagnostics(plan: SchemaPlanView): SchemaPlanView["diagnostics"] { + const seen = new Set(); + const unique: SchemaPlanView["diagnostics"][number][] = []; + for (const diagnostic of plan.diagnostics) { + if (diagnostic.severity !== "error" && !STRICT_COVERAGE_CODES.has(diagnostic.code)) continue; + const key = `${diagnostic.code}\0${diagnostic.message}`; + if (seen.has(key)) continue; + seen.add(key); + unique.push(diagnostic); + } + return unique; +} + +function stringSamples(value: unknown): string[] { + if (!Array.isArray(value)) return []; + return value.filter((item): item is string => typeof item === "string"); +} + +function shortCoverageLine(diagnostic: SchemaPlanView["diagnostics"][number]): string { + const kind = diagnostic.context?.["kind"]; + const samples = stringSamples(diagnostic.context?.["samples"]); + const items = samples.length > 0 ? samples : stringSamples(diagnostic.context?.["missing"]); + if (typeof kind === "string" && items.length > 0) { + const rawCount = diagnostic.context?.["count"]; + const count = typeof rawCount === "number" ? rawCount : items.length; + const shown = items.slice(0, 3); + const extra = count > shown.length ? ", …" : ""; + return `${count} unmodeled ${kind} (${shown.join(", ")}${extra})`; + } + const sentence = diagnostic.message.split(" — ")[0]; + return sentence !== undefined && sentence.length > 0 ? sentence : diagnostic.message; +} + +export function formatCoverageDiagnostics( + plan: SchemaPlanView, + opts?: CoverageFormatOptions, +): string { + const diagnostics = coverageDiagnostics(plan); + if (diagnostics.length === 0) return ""; + if (coverageVerbose(opts)) { + return diagnostics.map((diagnostic) => diagnostic.message).join("\n"); + } + const shown = diagnostics.slice(0, DEFAULT_COVERAGE_LINES); + const lines = shown.map(shortCoverageLine); + const hidden = diagnostics.length - shown.length; + if (hidden > 0) { + lines.push(`${hidden} more. Re-run with --debug for full diagnostics.`); + } + return lines.join("\n"); +} + +function withStatusExtras(status: string, extras: ReadonlyArray): string { + const lines = extras.filter((line): line is string => line !== undefined && line.length > 0); + return lines.length > 0 ? `${status}\n${lines.join("\n")}` : status; +} + +export function withCoverageMessage( + status: string, + plan: SchemaPlanView, + opts?: CoverageFormatOptions, +): string { + return withStatusExtras(status, [ + formatCoverageDiagnostics(plan, opts), + formatShadowLoadAssist(plan, opts), + ]); +} + +export function formatMigrationFilePath(fileName: string): string { + return `supabase/${MIGRATIONS_DIRECTORY_NAME}/${fileName}`; +} + +export function formatNextAction(why: string, command: string): string { + return `${why}: ${command}`; +} + +export function formatPlanSummary(input: { + readonly plan: SchemaPlanView; + readonly verbose?: boolean; +}): string { + const verbose = coverageVerbose({ verbose: input.verbose }); + const statements = planStatementCount(input.plan); + const { rewrite, destructive, coverageGaps } = input.plan.hazards; + const coverage = formatCoverageDiagnostics(input.plan, { verbose: input.verbose }); + const lines: string[] = []; + if (statements > 0) { + lines.push(`${statements} ${statements === 1 ? "statement" : "statements"}`); + } + if (rewrite > 0 || destructive > 0 || coverageGaps > 0) { + lines.push( + `Hazards: ${rewrite} rewrite, ${destructive} destructive, ${coverageGaps} coverage gaps`, + ); + } + if (coverage.length > 0) { + lines.push(coverage); + } + const loadAssist = formatShadowLoadAssist(input.plan, { verbose: input.verbose }); + if (loadAssist.length > 0) { + lines.push(loadAssist); + } + if (verbose) { + lines.push(`Plan: ${input.plan.planId}`); + lines.push( + `${input.plan.sourceFingerprint.slice(0, 8)} -> ${input.plan.desiredFingerprint.slice(0, 8)}`, + ); + } + return lines.join("\n"); +} + +export function withPlanSummary( + status: string, + plan: SchemaPlanView, + opts?: CoverageFormatOptions, +): string { + const summary = formatPlanSummary({ plan, verbose: opts?.verbose }); + return summary.length > 0 ? `${status}\n${summary}` : status; +} + +export function formatFileSummary(summary: SchemaFileSummary): string { + return `${summary.created.length} created, ${summary.updated.length} updated, ${summary.unchanged.length} unchanged, ${summary.removed.length} removed`; +} diff --git a/apps/cli/src/shared/schema/schema-output.unit.test.ts b/apps/cli/src/shared/schema/schema-output.unit.test.ts new file mode 100644 index 0000000000..e58f351e9e --- /dev/null +++ b/apps/cli/src/shared/schema/schema-output.unit.test.ts @@ -0,0 +1,228 @@ +import { describe, expect, it } from "@effect/vitest"; +import { classifyPlanHazards, type Plan } from "@supabase/pg-delta/plan"; +import { + formatCoverageDiagnostics, + formatPlanSummary, + formatShadowLoadAssist, + withCoverageMessage, +} from "./schema-output.ts"; +import type { SchemaPlanView } from "./schema-types.ts"; + +const dummyAction: Plan["actions"][number] = { + sql: "create table t (id int)", + verb: "create", + produces: [], + consumes: [], + destroys: [], + releases: [], + transactionality: "transactional", + lockClass: "none", + newSegmentBefore: false, + dataLoss: "none", + rewriteRisk: false, +}; + +function view( + diagnostics: SchemaPlanView["diagnostics"], + opts: { + readonly actions?: Plan["actions"]; + readonly rewrite?: number; + readonly destructive?: number; + } = {}, +): SchemaPlanView { + const actions = opts.actions ?? []; + const plan = { + formatVersion: 1, + engineVersion: "0.3.0", + planId: "2727ec28b32ec7af6ab", + source: { fingerprint: "38e478bcsource" }, + target: { fingerprint: "f9f5e272desired" }, + preamble: [], + deltas: [], + filteredDeltas: [], + renameCandidates: [], + actions, + safetyReport: { + destructiveActions: 0, + rewriteRiskActions: 0, + nonTransactionalActions: 0, + lockClasses: {}, + }, + } satisfies Plan; + return { + planId: plan.planId, + sourceFingerprint: plan.source.fingerprint, + desiredFingerprint: plan.target.fingerprint, + engineVersion: plan.engineVersion, + profile: "supabase", + changes: false, + files: [], + hazards: { + kinds: [], + destructive: opts.destructive ?? 0, + rewrite: opts.rewrite ?? 0, + coverageGaps: diagnostics.length, + report: classifyPlanHazards(plan), + }, + destructive: false, + renameCandidates: [], + acceptedRenames: [], + coverageBlocked: diagnostics.length > 0, + renameBlocked: false, + diagnostics, + plan, + }; +} + +describe("formatCoverageDiagnostics", () => { + it("shortens to kind and samples and caps the default list", () => { + const diagnostics = ["cast", "operator", "collation", "language"].map((kind) => ({ + code: "unmodeled_kind", + severity: "warning" as const, + message: `1 unmodeled "${kind}" object — v1 detects but does not model this kind`, + context: { kind, count: 1, samples: [kind] }, + })); + expect(formatCoverageDiagnostics(view(diagnostics), { verbose: false })).toBe( + [ + "1 unmodeled cast (cast)", + "1 unmodeled operator (operator)", + "1 unmodeled collation (collation)", + "1 more. Re-run with --debug for full diagnostics.", + ].join("\n"), + ); + }); +}); + +describe("formatShadowLoadAssist", () => { + it("keeps pg-delta copy and names the session as supautils poisoning", () => { + const plan = view([ + { + code: "session_pollution", + severity: "warning", + message: [ + "New connection unblocked a stuck load (session poisoning).", + ' stuck storage/tables/objects.sql:1 CREATE POLICY "foo" ON storage.objects (must be owner of table objects)', + ' earlier _cluster/publications.sql:1 ALTER PUBLICATION "supabase_realtime" ADD TABLE "public"."n8n_chat_histories"', + "Remove session-setting statements from declarative SQL, or do not share that session with later DDL.", + ].join("\n"), + }, + ]); + expect(formatShadowLoadAssist(plan, { verbose: false })).toBe(""); + expect( + withCoverageMessage("Declarations already match migration replay.", plan, { verbose: false }), + ).not.toContain("session poisoning"); + expect(formatShadowLoadAssist(plan, { verbose: true })).toBe( + [ + "New connection unblocked a stuck load (supautils session poisoning).", + ' stuck storage/tables/objects.sql:1 CREATE POLICY "foo" ON storage.objects (must be owner of table objects)', + ' earlier _cluster/publications.sql:1 ALTER PUBLICATION "supabase_realtime" ADD TABLE "public"."n8n_chat_histories"', + ].join("\n"), + ); + expect( + withCoverageMessage("Declarations already match migration replay.", plan, { verbose: true }), + ).toContain("supautils session poisoning"); + }); + + it("keeps same-file reorder advice and points loadOrder at the export sidecar", () => { + const plan = view([ + { + code: "reorder_on_failure", + severity: "warning", + message: [ + "Default load order stuck; reordered (statement-kind).", + ' move public/tables/reorder_probe.sql:1 ALTER PUBLICATION "supabase_realtime" ADD TABLE "public"."reorder_probe";', + ' after public/tables/reorder_probe.sql:2 CREATE TABLE "public"."reorder_probe" (id integer);', + "loadOrder cannot fix same-file order — edit or split the file.", + ].join("\n"), + }, + { + code: "reorder_on_failure", + severity: "warning", + message: [ + "Default load order stuck; reordered (file-kind).", + " stuck _cluster/publications.sql:1 ALTER PUBLICATION p ADD TABLE public.t;", + " after public/tables/t.sql:1 CREATE TABLE public.t (id integer);", + "Set loadOrder on .pgdelta-export.json to put public/tables/t.sql before _cluster/publications.sql.", + ].join("\n"), + }, + ]); + expect(formatShadowLoadAssist(plan, { verbose: false })).toBe(""); + expect(formatShadowLoadAssist(plan, { verbose: true })).toBe( + [ + "Default load order stuck; reordered (statement-kind).", + ' move public/tables/reorder_probe.sql:1 ALTER PUBLICATION "supabase_realtime" ADD TABLE "public"."reorder_probe";', + ' after public/tables/reorder_probe.sql:2 CREATE TABLE "public"."reorder_probe" (id integer);', + "loadOrder cannot fix same-file order — edit or split the file.", + "Default load order stuck; reordered (file-kind).", + " stuck _cluster/publications.sql:1 ALTER PUBLICATION p ADD TABLE public.t;", + " after public/tables/t.sql:1 CREATE TABLE public.t (id integer);", + "Set loadOrder on supabase/schemas/.pgdelta-export.json to put public/tables/t.sql before _cluster/publications.sql.", + ].join("\n"), + ); + }); +}); + +describe("formatPlanSummary", () => { + it("omits hashes, hazards, and a statements line on a clean empty plan", () => { + expect(formatPlanSummary({ plan: view([]), verbose: false })).toBe(""); + }); + + it("counts statements from plan files when actions are empty", () => { + const base = view([]); + const text = formatPlanSummary({ + plan: { + ...base, + files: [ + { + sequence: 1, + suffix: null, + sql: "create table t (id int); drop table u;", + transactional: true, + actionCount: 2, + }, + ], + hazards: { ...base.hazards, kinds: ["destructive"], destructive: 1 }, + }, + verbose: false, + }); + expect(text).toContain("2 statements"); + expect(text).toContain("Hazards: 0 rewrite, 1 destructive, 0 coverage gaps"); + }); + + it("prints a statements count without a hazards line when all hazard counts are zero", () => { + const text = formatPlanSummary({ + plan: view([], { actions: [dummyAction, dummyAction, dummyAction, dummyAction] }), + verbose: false, + }); + expect(text).toBe("4 statements"); + expect(text).not.toContain("Hazards:"); + expect(text).not.toContain("2727ec28"); + }); + + it("appends coverage diagnostics when coverageGaps is non-zero", () => { + const text = formatPlanSummary({ + plan: view([ + { + code: "unmodeled_kind", + severity: "warning", + message: '1 unmodeled "cast" object — v1 detects but does not model this kind', + context: { kind: "cast", count: 1, samples: ["public.widget AS integer"] }, + }, + ]), + verbose: false, + }); + expect(text).toContain("Hazards: 0 rewrite, 0 destructive, 1 coverage gaps"); + expect(text).toContain("1 unmodeled cast (public.widget AS integer)"); + expect(text).not.toContain("2727ec28"); + }); + + it("prints plan id and fingerprints only when verbose", () => { + const text = formatPlanSummary({ + plan: view([], { actions: [dummyAction] }), + verbose: true, + }); + expect(text).toContain("1 statement"); + expect(text).toContain("Plan: 2727ec28b32ec7af6ab"); + expect(text).toContain("38e478bc -> f9f5e272"); + }); +}); diff --git a/apps/cli/src/shared/schema/schema-paths.ts b/apps/cli/src/shared/schema/schema-paths.ts new file mode 100644 index 0000000000..e0c54c478a --- /dev/null +++ b/apps/cli/src/shared/schema/schema-paths.ts @@ -0,0 +1,6 @@ +export const SCHEMA_DIRECTORY_NAME = "schemas"; +export const SCHEMA_CUSTOM_DIRECTORY_NAME = "_custom"; +export const SCHEMA_DRAFT_JOURNAL_FILE_NAME = "schema-draft.json"; +export const SCHEMA_LOCK_FILE_NAME = "schema.lock"; +export const MIGRATIONS_DIRECTORY_NAME = "migrations"; +export const MIGRATION_NO_TRANSACTION_DIRECTIVE = "-- pg-delta: transaction=false"; diff --git a/apps/cli/src/shared/schema/schema-plan-gate.ts b/apps/cli/src/shared/schema/schema-plan-gate.ts new file mode 100644 index 0000000000..7438277536 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-plan-gate.ts @@ -0,0 +1,35 @@ +import { Effect } from "effect"; +import { SchemaPlanningBlockedError } from "./schema-errors.ts"; +import type { SchemaPlanView } from "./schema-types.ts"; +import { coverageVerbose, formatCoverageDiagnostics } from "./schema-output.ts"; + +export const assertPlanActionable = (plan: SchemaPlanView) => { + if (plan.renameBlocked) { + return Effect.fail( + new SchemaPlanningBlockedError({ + detail: "Planning found an ambiguous rename and cannot guess.", + suggestion: + "Rename explicitly in declarations, accept a rename decision, or reset the local database.", + }), + ); + } + if (plan.coverageBlocked) { + const verbose = coverageVerbose(); + const named = formatCoverageDiagnostics(plan, { verbose }); + return Effect.fail( + new SchemaPlanningBlockedError({ + detail: + named.length > 0 + ? `Planning found a coverage gap or unmodeled object.\n${named}` + : "Planning found a coverage gap or unmodeled object.", + suggestion: + named.length > 0 + ? verbose + ? "See the engine diagnostics above." + : "Re-run with --debug for full engine diagnostics." + : "Move unsupported objects to _custom/ or a manual migration, then retry.", + }), + ); + } + return Effect.void; +}; diff --git a/apps/cli/src/shared/schema/schema-plan-options.ts b/apps/cli/src/shared/schema/schema-plan-options.ts new file mode 100644 index 0000000000..269323b041 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-plan-options.ts @@ -0,0 +1,14 @@ +import { supabaseProfile } from "@supabase/pg-delta/integrations"; + +/** Isolated-cluster `planSchemaFiles` options. `reorder` escalates after default order + reconnect. */ +export const schemaIsolatedPlanOptions = { + profile: supabaseProfile, + scope: "database" as const, + redactSecrets: true, + isolatedShadow: true, + seedAssumedSchemas: false, + allowSameDatabaseIdentity: true, + strictDataStatements: true, + reorder: true, + connectionReuse: "reconnect-on-stuck" as const, +}; diff --git a/apps/cli/src/shared/schema/schema-plan-options.unit.test.ts b/apps/cli/src/shared/schema/schema-plan-options.unit.test.ts new file mode 100644 index 0000000000..abaa1134a8 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-plan-options.unit.test.ts @@ -0,0 +1,13 @@ +import { describe, expect, it } from "@effect/vitest"; +import { schemaIsolatedPlanOptions } from "./schema-plan-options.ts"; + +describe("schemaIsolatedPlanOptions", () => { + it("uses an isolated cluster and does not seed assumed schemas", () => { + expect(schemaIsolatedPlanOptions.isolatedShadow).toBe(true); + expect(schemaIsolatedPlanOptions.seedAssumedSchemas).toBe(false); + expect(schemaIsolatedPlanOptions.allowSameDatabaseIdentity).toBe(true); + expect(schemaIsolatedPlanOptions.scope).toBe("database"); + expect(schemaIsolatedPlanOptions.reorder).toBe(true); + expect(schemaIsolatedPlanOptions.connectionReuse).toBe("reconnect-on-stuck"); + }); +}); diff --git a/apps/cli/src/shared/schema/schema-render.integration.test.ts b/apps/cli/src/shared/schema/schema-render.integration.test.ts new file mode 100644 index 0000000000..ae6f8bd031 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-render.integration.test.ts @@ -0,0 +1,151 @@ +import { describe, expect, it } from "@effect/vitest"; +import { Effect } from "effect"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import { renderSchemaResult } from "./schema-render.ts"; +import type { SchemaCommandResult } from "./schema-types.ts"; + +function result(overrides: Partial = {}): SchemaCommandResult { + return { + status: "clean", + message: "4 migrations applied on the local database. History matches files.", + data: {}, + nextActions: [], + mutatedDatabase: false, + mutatedFiles: false, + ...overrides, + }; +} + +describe("renderSchemaResult", () => { + it.live("prints a single next action on one line", () => { + const out = mockOutput(); + return Effect.gen(function* () { + yield* renderSchemaResult( + "Create migration", + result({ + nextActions: ["to apply it locally: supabase migrations apply"], + }), + ).pipe(Effect.provide(out.layer)); + expect(out.messages).toEqual([ + { type: "intro", message: "Create migration" }, + { + type: "info", + message: "Next: to apply it locally: supabase migrations apply", + }, + { + type: "outro", + message: "4 migrations applied on the local database. History matches files.", + }, + ]); + }); + }); + + it.live("skipIntro does not emit a second intro", () => { + const out = mockOutput(); + return Effect.gen(function* () { + yield* renderSchemaResult("Push migrations", result(), { skipIntro: true }).pipe( + Effect.provide(out.layer), + ); + expect(out.messages).toEqual([ + { + type: "outro", + message: "4 migrations applied on the local database. History matches files.", + }, + ]); + }); + }); + + it.live("prints a single-line success once as the outro", () => { + const out = mockOutput(); + return Effect.gen(function* () { + yield* renderSchemaResult("List migrations", result()).pipe(Effect.provide(out.layer)); + expect(out.messages).toEqual([ + { type: "intro", message: "List migrations" }, + { + type: "outro", + message: "4 migrations applied on the local database. History matches files.", + }, + ]); + }); + }); + + it.live("outros the status line and lists extra lines above it", () => { + const out = mockOutput(); + return Effect.gen(function* () { + yield* renderSchemaResult( + "Generate schema migrations", + result({ + message: + "Declarations already match migration replay.\n1 unmodeled cast (log_min_messages)", + nextActions: [ + "to check they match: supabase schema generate --dry-run", + "to generate a migration: supabase schema generate --name ", + ], + }), + ).pipe(Effect.provide(out.layer)); + expect(out.messages).toEqual([ + { type: "intro", message: "Generate schema migrations" }, + { type: "info", message: "1 unmodeled cast (log_min_messages)" }, + { type: "info", message: "Next:" }, + { + type: "info", + message: " 1. to check they match: supabase schema generate --dry-run", + }, + { + type: "info", + message: " 2. to generate a migration: supabase schema generate --name ", + }, + { type: "outro", message: "Declarations already match migration replay." }, + ]); + }); + }); + + it.live("writes body as one raw chunk and keeps extra message lines as info", () => { + const out = mockOutput(); + const sql = "CREATE TABLE t (id int);\nALTER TABLE t ADD COLUMN n int;"; + return Effect.gen(function* () { + yield* renderSchemaResult( + "Generate schema migrations", + result({ + message: "Dry-run; nothing was written.\n2 statements", + body: sql, + nextActions: ["to write the migration: supabase schema generate --name "], + }), + ).pipe(Effect.provide(out.layer)); + expect(out.messages).toEqual([ + { type: "intro", message: "Generate schema migrations" }, + { type: "info", message: "2 statements" }, + { + type: "info", + message: "Next: to write the migration: supabase schema generate --name ", + }, + { type: "outro", message: "Dry-run; nothing was written." }, + ]); + expect(out.rawChunks).toEqual([{ text: `${sql}\n`, stream: "stdout" }]); + }); + }); + + it.live("emits JSON from message and data without dumping body", () => { + const out = mockOutput({ format: "json" }); + const sql = "CREATE TABLE t (id int);"; + return Effect.gen(function* () { + yield* renderSchemaResult( + "Generate schema migrations", + result({ + message: "2 statements", + body: sql, + data: { sql, files: [{ name: "schema.sql", sql }] }, + }), + ).pipe(Effect.provide(out.layer)); + expect(out.messages).toEqual([ + { type: "intro", message: "Generate schema migrations" }, + { + type: "success", + message: "2 statements", + data: { sql, files: [{ name: "schema.sql", sql }] }, + }, + ]); + expect(out.rawChunks).toEqual([]); + }); + }); +}); diff --git a/apps/cli/src/shared/schema/schema-render.ts b/apps/cli/src/shared/schema/schema-render.ts new file mode 100644 index 0000000000..e947c1e9a7 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-render.ts @@ -0,0 +1,57 @@ +import { Effect } from "effect"; +import { Output } from "../output/output.service.ts"; +import type { SchemaCommandResult } from "./schema-types.ts"; + +export const renderSchemaResult = Effect.fnUntraced(function* ( + title: string, + result: SchemaCommandResult, + opts?: { readonly skipIntro?: boolean }, +) { + const output = yield* Output; + if (opts?.skipIntro !== true) { + yield* output.intro(title); + } + if (output.format !== "text") { + yield* output.success(result.message, result.data); + return; + } + const lines = result.message.split("\n").filter((line) => line.length > 0); + if (result.status === "failed") { + for (const line of lines) yield* output.info(line); + yield* writeBody(output, result.body); + yield* writeNextActions(output, result.nextActions); + yield* output.outro("Failed."); + return; + } + for (const line of lines.slice(1)) yield* output.info(line); + yield* writeBody(output, result.body); + yield* writeNextActions(output, result.nextActions); + yield* output.outro(lines[0] ?? "Done."); +}); + +const writeBody = Effect.fnUntraced(function* ( + output: { + readonly raw: (text: string) => Effect.Effect; + }, + body: string | undefined, +) { + if (body === undefined || body.length === 0) return; + yield* output.raw(body.endsWith("\n") ? body : `${body}\n`); +}); + +const writeNextActions = Effect.fnUntraced(function* ( + output: { + readonly info: (message: string) => Effect.Effect; + }, + actions: ReadonlyArray, +) { + if (actions.length === 0) return; + if (actions.length === 1) { + yield* output.info(`Next: ${actions[0]}`); + return; + } + yield* output.info("Next:"); + for (const [index, action] of actions.entries()) { + yield* output.info(` ${index + 1}. ${action}`); + } +}); diff --git a/apps/cli/src/shared/schema/schema-shadow.ts b/apps/cli/src/shared/schema/schema-shadow.ts new file mode 100644 index 0000000000..7fb21950af --- /dev/null +++ b/apps/cli/src/shared/schema/schema-shadow.ts @@ -0,0 +1,3 @@ +export type SchemaShadow = { + readonly url: string; +}; diff --git a/apps/cli/src/shared/schema/schema-state.layer.ts b/apps/cli/src/shared/schema/schema-state.layer.ts new file mode 100644 index 0000000000..82691279bc --- /dev/null +++ b/apps/cli/src/shared/schema/schema-state.layer.ts @@ -0,0 +1,151 @@ +import { Clock, Effect, FileSystem, Layer, Option, Path, Schema } from "effect"; +import { SchemaLockError, SchemaStateError } from "./schema-errors.ts"; +import { SchemaStateStore } from "./schema-state.service.ts"; +import type { SchemaDraftJournal } from "./schema-types.ts"; +import { SchemaWorkspace } from "./schema-workspace.service.ts"; + +const JournalSchema = Schema.Struct({ + version: Schema.Literal(1), + draftId: Schema.String, + targetIdentity: Schema.String, + startingMigrationHeadDigest: Schema.String, + sourceFingerprint: Schema.String, + plans: Schema.Array( + Schema.Struct({ + planId: Schema.String, + targetFingerprint: Schema.String, + acceptedRenames: Schema.Array(Schema.Struct({ from: Schema.String, to: Schema.String })), + segmentDigests: Schema.Array(Schema.String), + hazards: Schema.Struct({ + kinds: Schema.Array(Schema.String), + destructive: Schema.Number, + rewrite: Schema.Number, + coverageGaps: Schema.Number, + }), + actionStatuses: Schema.Array(Schema.Literals(["applied", "unapplied", "inDoubt"])), + outcome: Schema.Literals(["applied", "failed", "partial"]), + }), + ), + engineVersion: Schema.String, + declarativelyAhead: Schema.Boolean, + generated: Schema.optionalKey(Schema.Boolean), + invalidationReason: Schema.optionalKey(Schema.String), +}); + +const STALE_LOCK_MS = 10 * 60 * 1000; + +const stateError = (detail: string) => + new SchemaStateError({ + detail, + suggestion: "Fix or delete `.supabase/schema-draft.json` and rerun the command.", + }); + +export const schemaStateLayer = Layer.effect( + SchemaStateStore, + Effect.gen(function* () { + const workspace = yield* SchemaWorkspace; + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + + const readDecoded = ( + filePath: string, + decode: (value: unknown) => A, + ): Effect.Effect, SchemaStateError> => + Effect.gen(function* () { + const exists = yield* fs.exists(filePath).pipe(Effect.orElseSucceed(() => false)); + if (!exists) return Option.none(); + const raw = yield* fs + .readFileString(filePath) + .pipe( + Effect.mapError((error) => stateError(`Failed to read ${filePath}: ${error.message}`)), + ); + const parsed = yield* Effect.try({ + try: (): unknown => JSON.parse(raw), + catch: () => stateError(`Malformed ${path.basename(filePath)}.`), + }); + return Option.some( + yield* Effect.try({ + try: () => decode(parsed), + catch: (error) => + stateError( + `Malformed ${path.basename(filePath)}: ${error instanceof Error ? error.message : String(error)}`, + ), + }), + ); + }); + + const writeJson = (filePath: string, value: unknown) => + Effect.gen(function* () { + yield* fs + .makeDirectory(path.dirname(filePath), { recursive: true }) + .pipe( + Effect.mapError((error) => + stateError(`Failed to create ${path.dirname(filePath)}: ${error.message}`), + ), + ); + yield* fs + .writeFileString(filePath, `${JSON.stringify(value, null, 2)}\n`) + .pipe( + Effect.mapError((error) => stateError(`Failed to write ${filePath}: ${error.message}`)), + ); + }); + + return SchemaStateStore.of({ + readJournal: readDecoded(workspace.journalPath, Schema.decodeUnknownSync(JournalSchema)), + writeJournal: (journal: SchemaDraftJournal) => writeJson(workspace.journalPath, journal), + clearJournal: Effect.gen(function* () { + yield* fs + .remove(workspace.journalPath) + .pipe( + Effect.catchTag("PlatformError", (error) => + error.reason._tag === "NotFound" + ? Effect.void + : Effect.fail(stateError(error.message)), + ), + ); + }), + withLock: (effect) => + Effect.gen(function* () { + yield* fs.makeDirectory(path.dirname(workspace.lockPath), { recursive: true }).pipe( + Effect.mapError( + (error) => + new SchemaLockError({ + detail: `Failed to create lock directory: ${error.message}`, + suggestion: "Check permissions on .supabase/.", + }), + ), + ); + const now = yield* Clock.currentTimeMillis; + const exists = yield* fs + .exists(workspace.lockPath) + .pipe(Effect.orElseSucceed(() => false)); + if (exists) { + const raw = yield* fs + .readFileString(workspace.lockPath) + .pipe(Effect.orElseSucceed(() => "")); + const stamped = Number.parseInt(raw, 10); + const stale = !Number.isFinite(stamped) || now - stamped > STALE_LOCK_MS; + if (!stale) { + return yield* new SchemaLockError({ + detail: "Another schema or migrations command is already running.", + suggestion: + "Wait for it to finish, or remove .supabase/schema.lock if it is stale.", + }); + } + } + yield* fs.writeFileString(workspace.lockPath, `${now}\n`).pipe( + Effect.mapError( + (error) => + new SchemaLockError({ + detail: `Failed to acquire schema lock: ${error.message}`, + suggestion: "Check permissions on .supabase/schema.lock.", + }), + ), + ); + return yield* effect.pipe( + Effect.ensuring(fs.remove(workspace.lockPath).pipe(Effect.ignore)), + ); + }), + }); + }), +); diff --git a/apps/cli/src/shared/schema/schema-state.service.ts b/apps/cli/src/shared/schema/schema-state.service.ts new file mode 100644 index 0000000000..c77f164ee4 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-state.service.ts @@ -0,0 +1,17 @@ +import type { Effect, Option } from "effect"; +import { Context } from "effect"; +import type { SchemaDraftJournal } from "./schema-types.ts"; +import type { SchemaLockError, SchemaStateError } from "./schema-errors.ts"; + +interface SchemaStateStoreShape { + readonly readJournal: Effect.Effect, SchemaStateError>; + readonly writeJournal: (journal: SchemaDraftJournal) => Effect.Effect; + readonly clearJournal: Effect.Effect; + readonly withLock: ( + effect: Effect.Effect, + ) => Effect.Effect; +} + +export class SchemaStateStore extends Context.Service()( + "supabase/schema/SchemaStateStore", +) {} diff --git a/apps/cli/src/shared/schema/schema-types.ts b/apps/cli/src/shared/schema/schema-types.ts new file mode 100644 index 0000000000..f27bceb558 --- /dev/null +++ b/apps/cli/src/shared/schema/schema-types.ts @@ -0,0 +1,98 @@ +import type { Diagnostic } from "@supabase/pg-delta/core"; +import type { SqlFileClassification } from "@supabase/pg-delta/frontends"; +import type { HazardReport } from "@supabase/pg-delta/plan"; +import type { Plan } from "@supabase/pg-delta/plan"; +import type { ApplyReport } from "@supabase/pg-delta/apply"; + +type SchemaCommandStatus = + | "clean" + | "draft" + | "needs_approval" + | "generated" + | "drift" + | "conflict" + | "partial" + | "failed"; + +export type SchemaSqlFile = { + readonly name: string; + readonly sql: string; +}; + +export type SchemaHazardSummary = { + readonly kinds: ReadonlyArray; + readonly destructive: number; + readonly rewrite: number; + readonly coverageGaps: number; + readonly report: HazardReport; +}; + +export type SchemaRenderedFile = { + readonly sequence: number; + readonly suffix: string | null; + readonly sql: string; + readonly transactional: boolean; + readonly actionCount: number; +}; + +export type SchemaPlanView = { + readonly planId: string; + readonly sourceFingerprint: string; + readonly desiredFingerprint: string; + readonly engineVersion: string; + readonly profile: string; + readonly changes: boolean; + readonly files: ReadonlyArray; + readonly hazards: SchemaHazardSummary; + readonly destructive: boolean; + readonly renameCandidates: ReadonlyArray<{ readonly from: string; readonly to: string }>; + readonly acceptedRenames: ReadonlyArray<{ readonly from: string; readonly to: string }>; + readonly coverageBlocked: boolean; + readonly renameBlocked: boolean; + readonly diagnostics: ReadonlyArray; + readonly plan: Plan; +}; + +export type SchemaApplyOutcome = { + readonly report: ApplyReport; + readonly partial: boolean; +}; + +export type SchemaFileSummary = Pick< + SqlFileClassification, + "created" | "updated" | "unchanged" | "removed" | "unmanaged" +>; + +type SchemaJournaledPlan = { + readonly planId: string; + readonly targetFingerprint: string; + readonly acceptedRenames: ReadonlyArray<{ readonly from: string; readonly to: string }>; + readonly segmentDigests: ReadonlyArray; + readonly hazards: Pick; + readonly actionStatuses: ReadonlyArray<"applied" | "unapplied" | "inDoubt">; + readonly outcome: "applied" | "failed" | "partial"; +}; + +export type SchemaDraftJournal = { + readonly version: 1; + readonly draftId: string; + readonly targetIdentity: string; + readonly startingMigrationHeadDigest: string; + readonly sourceFingerprint: string; + readonly plans: ReadonlyArray; + readonly engineVersion: string; + readonly declarativelyAhead: boolean; + readonly generated?: boolean; + readonly invalidationReason?: string; +}; + +export type SchemaCommandResult = { + readonly status: SchemaCommandStatus; + readonly message: string; + /** Unframed SQL or inventory. Rendered with `output.raw`; never put this in `message`. */ + readonly body?: string; + readonly data: Record; + readonly nextActions: ReadonlyArray; + readonly mutatedDatabase: boolean; + readonly mutatedFiles: boolean; +}; diff --git a/apps/cli/src/shared/schema/schema-workspace.layer.ts b/apps/cli/src/shared/schema/schema-workspace.layer.ts new file mode 100644 index 0000000000..4b6c53568d --- /dev/null +++ b/apps/cli/src/shared/schema/schema-workspace.layer.ts @@ -0,0 +1,301 @@ +import { Effect, FileSystem, Layer, Path } from "effect"; +import { + classifySqlFiles, + EXPORT_MANIFEST_FILE, + readExportManifest, + type SqlFileClassification, +} from "@supabase/pg-delta/frontends"; +import { + SCHEMA_CUSTOM_DIRECTORY_NAME, + SCHEMA_DIRECTORY_NAME, + SCHEMA_DRAFT_JOURNAL_FILE_NAME, + SCHEMA_LOCK_FILE_NAME, + MIGRATIONS_DIRECTORY_NAME, +} from "./schema-paths.ts"; +import { SCHEMA_PULL_NO_MERGE_HELP } from "./schema-ecosystem.ts"; +import { + SchemaDeclarationsExistError, + SchemaUnmanagedFilesError, + SchemaWorkspaceIoError, +} from "./schema-errors.ts"; +import type { SchemaSqlFile } from "./schema-types.ts"; +import { + SchemaWorkspace, + type SchemaInstallInput, + type SchemaInstallResult, +} from "./schema-workspace.service.ts"; + +const ioError = (detail: string, suggestion = "Check filesystem permissions and retry.") => + new SchemaWorkspaceIoError({ detail, suggestion }); + +function isCustomPath(relative: string): boolean { + return relative.split("/")[0] === SCHEMA_CUSTOM_DIRECTORY_NAME; +} + +function posixRel(path: Path.Path, value: string): string { + return path.normalize(value.split("\\").join("/")).split("\\").join("/"); +} + +function parseSafeRelative( + path: Path.Path, + name: string, +): Effect.Effect { + const rel = posixRel(path, name); + if (rel.startsWith("..") || path.isAbsolute(rel)) { + return Effect.fail(ioError(`Unsafe declarative export path: ${name}`)); + } + if (isCustomPath(rel)) { + return Effect.fail( + ioError( + `Refusing to write into reserved path: ${name}`, + "Keep hand-authored SQL in _custom/.", + ), + ); + } + return Effect.succeed(rel); +} + +function walkSqlFiles( + fs: FileSystem.FileSystem, + path: Path.Path, + directory: string, + prefix = "", +): Effect.Effect, SchemaWorkspaceIoError> { + return Effect.gen(function* () { + const names = yield* fs + .readDirectory(directory) + .pipe(Effect.mapError((error) => ioError(`Failed to read ${directory}: ${error.message}`))); + const files: Array = []; + for (const name of names) { + const relative = prefix === "" ? name : `${prefix}/${name}`; + if (prefix === "" && name === SCHEMA_CUSTOM_DIRECTORY_NAME) continue; + const absolute = path.join(directory, name); + const isSymlink = yield* fs.readLink(absolute).pipe( + Effect.as(true), + Effect.orElseSucceed(() => false), + ); + if (isSymlink) continue; + const info = yield* fs + .stat(absolute) + .pipe(Effect.mapError((error) => ioError(`Failed to stat ${absolute}: ${error.message}`))); + if (info.type === "Directory") { + files.push(...(yield* walkSqlFiles(fs, path, absolute, relative))); + } else if (info.type === "File" && name.endsWith(".sql")) { + files.push({ + name: relative.split("\\").join("/"), + sql: yield* fs + .readFileString(absolute) + .pipe( + Effect.mapError((error) => ioError(`Failed to read ${absolute}: ${error.message}`)), + ), + }); + } + } + // pg-delta uses this array order when loadOrder is absent. + return files.sort((left, right) => left.name.localeCompare(right.name)); + }); +} + +export type SchemaWorkspacePaths = { + readonly projectRoot: string; + readonly supabaseDir: string; + readonly projectHomeDir: string; +}; + +export const schemaWorkspaceLayer = (paths: SchemaWorkspacePaths) => + Layer.effect( + SchemaWorkspace, + Effect.gen(function* () { + const path = yield* Path.Path; + const fs = yield* FileSystem.FileSystem; + + const schemasDir = path.join(paths.supabaseDir, SCHEMA_DIRECTORY_NAME); + const migrationsDir = path.join(paths.supabaseDir, MIGRATIONS_DIRECTORY_NAME); + const customDir = path.join(schemasDir, SCHEMA_CUSTOM_DIRECTORY_NAME); + const journalPath = path.join(paths.projectHomeDir, SCHEMA_DRAFT_JOURNAL_FILE_NAME); + const lockPath = path.join(paths.projectHomeDir, SCHEMA_LOCK_FILE_NAME); + + const readExistingSql = (directory = schemasDir) => + Effect.gen(function* () { + const exists = yield* fs.exists(directory).pipe(Effect.orElseSucceed(() => false)); + if (!exists) return new Map(); + const files = yield* walkSqlFiles(fs, path, directory); + return new Map(files.map((file) => [file.name, file.sql])); + }); + + const classifyProposed = (proposed: ReadonlyArray, directory = schemasDir) => + Effect.gen(function* () { + const existing = yield* readExistingSql(directory); + const exists = yield* fs.exists(directory).pipe(Effect.orElseSucceed(() => false)); + const previous = exists ? readExportManifest(directory) : undefined; + return classifySqlFiles({ + proposed, + existing, + ...(previous?.files !== undefined ? { previouslyOwned: new Set(previous.files) } : {}), + }); + }); + + const writeTree = Effect.fnUntraced(function* ( + directory: string, + files: ReadonlyArray, + classification: SqlFileClassification, + pruneUnmanaged: boolean, + manifest: Record, + ) { + yield* fs + .makeDirectory(directory, { recursive: true }) + .pipe( + Effect.mapError((error) => ioError(`Failed to create ${directory}: ${error.message}`)), + ); + + const changed = new Set([...classification.created, ...classification.updated]); + for (const file of files) { + const rel = yield* parseSafeRelative(path, file.name); + if (!changed.has(rel) && existingHas(classification, rel)) continue; + const target = path.join(directory, rel); + yield* fs + .makeDirectory(path.dirname(target), { recursive: true }) + .pipe( + Effect.mapError((error) => + ioError(`Failed to create ${path.dirname(target)}: ${error.message}`), + ), + ); + yield* fs + .writeFileString(target, file.sql) + .pipe( + Effect.mapError((error) => ioError(`Failed to write ${target}: ${error.message}`)), + ); + } + + for (const name of classification.removed) { + yield* fs + .remove(path.join(directory, name)) + .pipe( + Effect.catchTag("PlatformError", (error) => + error.reason._tag === "NotFound" + ? Effect.void + : Effect.fail(ioError(`Failed to remove ${name}: ${error.message}`)), + ), + ); + } + + if (pruneUnmanaged) { + for (const name of classification.unmanaged) { + yield* fs + .remove(path.join(directory, name)) + .pipe( + Effect.mapError((error) => ioError(`Failed to prune ${name}: ${error.message}`)), + ); + } + } + + const owned: Array = []; + for (const file of files) { + owned.push(yield* parseSafeRelative(path, file.name)); + } + owned.sort(); + const serialized = `${JSON.stringify({ formatVersion: 1, ...manifest, files: owned }, null, 2)}\n`; + yield* fs + .writeFileString(path.join(directory, EXPORT_MANIFEST_FILE), serialized) + .pipe( + Effect.mapError((error) => + ioError(`Failed to write export manifest: ${error.message}`), + ), + ); + }); + + function existingHas(classification: SqlFileClassification, rel: string): boolean { + return ( + classification.unchanged.includes(rel) || + classification.updated.includes(rel) || + classification.created.includes(rel) + ); + } + + const installExport = (input: SchemaInstallInput) => + Effect.gen(function* () { + const directory = input.mode === "output" ? (input.outputDir ?? schemasDir) : schemasDir; + const directoryDisplay = + input.mode === "output" + ? path.relative(paths.projectRoot, directory) + : path.join("supabase", SCHEMA_DIRECTORY_NAME); + + const proposed: Array = []; + for (const file of input.files) { + proposed.push({ + name: yield* parseSafeRelative(path, file.name), + sql: file.sql, + }); + } + + const destExists = yield* fs.exists(directory).pipe(Effect.orElseSucceed(() => false)); + const existing = destExists + ? yield* readExistingSql(directory) + : new Map(); + const hasSql = existing.size > 0; + + if (hasSql && input.mode === "init") { + return yield* new SchemaDeclarationsExistError({ + detail: "Declarative schema already exists.", + suggestion: SCHEMA_PULL_NO_MERGE_HELP, + }); + } + + if (hasSql && input.mode === "output") { + return yield* new SchemaDeclarationsExistError({ + detail: `Output directory already contains SQL: ${directoryDisplay}`, + suggestion: + "Choose an empty --output directory or pass --force to replace the primary tree.", + }); + } + + const classification = yield* classifyProposed(proposed, directory); + if ( + classification.unmanaged.length > 0 && + !input.pruneUnmanaged && + input.mode !== "force" + ) { + return yield* new SchemaUnmanagedFilesError({ + detail: `Unmanaged declarative files would be left in place: ${classification.unmanaged.join(", ")}`, + suggestion: + "Move hand-authored SQL to supabase/schemas/_custom/, or pass --prune-unmanaged to delete.", + paths: classification.unmanaged, + }); + } + + yield* writeTree( + directory, + proposed, + classification, + input.pruneUnmanaged, + input.manifest, + ); + + return { + directory, + directoryDisplay, + classification, + replaced: input.mode === "force", + manifestPath: path.join(directory, EXPORT_MANIFEST_FILE), + } satisfies SchemaInstallResult; + }); + + return SchemaWorkspace.of({ + schemasDir, + schemasDirDisplay: path.join("supabase", SCHEMA_DIRECTORY_NAME), + migrationsDir, + migrationsDirDisplay: path.join("supabase", MIGRATIONS_DIRECTORY_NAME), + customDir, + journalPath, + lockPath, + readDeclarationFiles: Effect.gen(function* () { + const exists = yield* fs.exists(schemasDir).pipe(Effect.orElseSucceed(() => false)); + if (!exists) return []; + return yield* walkSqlFiles(fs, path, schemasDir); + }), + readExistingSql, + classifyProposed, + installExport, + }); + }), + ); diff --git a/apps/cli/src/shared/schema/schema-workspace.service.ts b/apps/cli/src/shared/schema/schema-workspace.service.ts new file mode 100644 index 0000000000..26448c0c0e --- /dev/null +++ b/apps/cli/src/shared/schema/schema-workspace.service.ts @@ -0,0 +1,58 @@ +import type { Effect } from "effect"; +import { Context } from "effect"; +import type { SqlFileClassification } from "@supabase/pg-delta/frontends"; +import type { SchemaSqlFile } from "./schema-types.ts"; +import type { + SchemaDeclarationsExistError, + SchemaUnmanagedFilesError, + SchemaWorkspaceIoError, +} from "./schema-errors.ts"; + +type SchemaInstallMode = "init" | "force" | "output"; + +export type SchemaInstallInput = { + readonly files: ReadonlyArray; + readonly manifest: Record; + readonly mode: SchemaInstallMode; + readonly outputDir?: string; + readonly pruneUnmanaged: boolean; +}; + +export type SchemaInstallResult = { + readonly directory: string; + readonly directoryDisplay: string; + readonly classification: SqlFileClassification; + readonly replaced: boolean; + readonly manifestPath: string; +}; + +interface SchemaWorkspaceShape { + readonly schemasDir: string; + readonly schemasDirDisplay: string; + readonly migrationsDir: string; + readonly migrationsDirDisplay: string; + readonly customDir: string; + readonly journalPath: string; + readonly lockPath: string; + readonly readDeclarationFiles: Effect.Effect< + ReadonlyArray, + SchemaWorkspaceIoError + >; + readonly readExistingSql: ( + directory?: string, + ) => Effect.Effect, SchemaWorkspaceIoError>; + readonly classifyProposed: ( + proposed: ReadonlyArray, + directory?: string, + ) => Effect.Effect; + readonly installExport: ( + input: SchemaInstallInput, + ) => Effect.Effect< + SchemaInstallResult, + SchemaDeclarationsExistError | SchemaUnmanagedFilesError | SchemaWorkspaceIoError + >; +} + +export class SchemaWorkspace extends Context.Service()( + "supabase/schema/SchemaWorkspace", +) {} diff --git a/apps/cli/src/shared/schema/shadow-replay-output.ts b/apps/cli/src/shared/schema/shadow-replay-output.ts new file mode 100644 index 0000000000..be25416cce --- /dev/null +++ b/apps/cli/src/shared/schema/shadow-replay-output.ts @@ -0,0 +1,30 @@ +import { Effect } from "effect"; +import { Output } from "../output/output.service.ts"; + +const APPLYING_MIGRATION_PREFIX = "Applying migration "; +const REPLAY_BANNER = + "Replaying migrations on a shadow to compare catalogs (not a live apply)...\n"; + +export function wrapShadowReplayOutput( + real: typeof Output.Service, + opts: { readonly debug: boolean }, +): typeof Output.Service { + let announced = false; + return Output.of({ + ...real, + raw: (text, stream = "stdout") => + Effect.suspend(() => { + if (!text.startsWith(APPLYING_MIGRATION_PREFIX)) { + return real.raw(text, stream); + } + if (opts.debug) { + return real.raw(`Shadow: ${text}`, stream); + } + if (real.format !== "text" || announced) { + return Effect.void; + } + announced = true; + return real.raw(REPLAY_BANNER, stream); + }), + }); +} diff --git a/apps/cli/src/shared/schema/shadow-replay-output.unit.test.ts b/apps/cli/src/shared/schema/shadow-replay-output.unit.test.ts new file mode 100644 index 0000000000..4623d96d95 --- /dev/null +++ b/apps/cli/src/shared/schema/shadow-replay-output.unit.test.ts @@ -0,0 +1,46 @@ +import { describe, expect, it } from "@effect/vitest"; +import { Effect } from "effect"; +import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import { Output } from "../output/output.service.ts"; +import { wrapShadowReplayOutput } from "./shadow-replay-output.ts"; + +describe("wrapShadowReplayOutput", () => { + it.effect("emits one replay line and swallows per-file apply lines", () => { + const out = mockOutput(); + return Effect.gen(function* () { + const wrapped = wrapShadowReplayOutput(yield* Output, { debug: false }); + yield* wrapped.raw("Applying migration a.sql...\n", "stderr"); + yield* wrapped.raw("Applying migration b.sql...\n", "stderr"); + yield* wrapped.raw("Seeding globals from roles.sql...\n", "stderr"); + expect(out.rawChunks.map((chunk) => chunk.text)).toEqual([ + "Replaying migrations on a shadow to compare catalogs (not a live apply)...\n", + "Seeding globals from roles.sql...\n", + ]); + }).pipe(Effect.provide(out.layer)); + }); + + it.effect("prefixes each apply line when debug is on", () => { + const out = mockOutput(); + return Effect.gen(function* () { + const wrapped = wrapShadowReplayOutput(yield* Output, { debug: true }); + yield* wrapped.raw("Applying migration a.sql...\n", "stderr"); + yield* wrapped.raw("Applying migration b.sql...\n", "stderr"); + expect(out.rawChunks.map((chunk) => chunk.text)).toEqual([ + "Shadow: Applying migration a.sql...\n", + "Shadow: Applying migration b.sql...\n", + ]); + }).pipe(Effect.provide(out.layer)); + }); + + it.effect("swallows apply lines in json without a replay banner", () => { + const out = mockOutput({ format: "json" }); + return Effect.gen(function* () { + const wrapped = wrapShadowReplayOutput(yield* Output, { debug: false }); + yield* wrapped.raw("Applying migration a.sql...\n", "stderr"); + yield* wrapped.raw("Seeding globals from roles.sql...\n", "stderr"); + expect(out.rawChunks.map((chunk) => chunk.text)).toEqual([ + "Seeding globals from roles.sql...\n", + ]); + }).pipe(Effect.provide(out.layer)); + }); +}); diff --git a/apps/cli/src/shared/schema/sql-format-defaults.ts b/apps/cli/src/shared/schema/sql-format-defaults.ts new file mode 100644 index 0000000000..b5e27b1c1c --- /dev/null +++ b/apps/cli/src/shared/schema/sql-format-defaults.ts @@ -0,0 +1,21 @@ +import { formatSqlStatements, type SqlFormatOptions } from "@supabase/pg-delta/sql-format"; + +/** Human-readable SQL for pg-delta emitters when no override is supplied. */ +export const SCHEMA_SQL_FORMAT_DEFAULTS = { + keywordCase: "upper", + indent: 2, + maxWidth: 180, + commaStyle: "trailing", + alignColumns: true, + alignKeyValues: true, +} satisfies SqlFormatOptions; + +function terminateStatement(sql: string): string { + const trimmed = sql.trimEnd(); + return trimmed.endsWith(";") ? trimmed : `${trimmed};`; +} + +export function formatSchemaSql(sql: string, options: SqlFormatOptions | undefined): string { + if (options === undefined) return sql; + return `${formatSqlStatements([sql], options).map(terminateStatement).join("\n\n")}\n`; +} diff --git a/apps/cli/src/shared/schema/sql-format-defaults.unit.test.ts b/apps/cli/src/shared/schema/sql-format-defaults.unit.test.ts new file mode 100644 index 0000000000..6faa243ba7 --- /dev/null +++ b/apps/cli/src/shared/schema/sql-format-defaults.unit.test.ts @@ -0,0 +1,17 @@ +import { describe, expect, it } from "vitest"; +import { formatSchemaSql, SCHEMA_SQL_FORMAT_DEFAULTS } from "./sql-format-defaults.ts"; + +describe("formatSchemaSql", () => { + it("pretty-prints with the CLI default options", () => { + expect( + formatSchemaSql( + "create table public.widgets (id integer, display_name text);", + SCHEMA_SQL_FORMAT_DEFAULTS, + ), + ).toBe(`CREATE TABLE public.widgets ( + id integer, + display_name text +); +`); + }); +}); diff --git a/docs/cli/dev-alpha-command-structure.md b/docs/cli/dev-alpha-command-structure.md index ed7b075931..d6af0cba00 100644 --- a/docs/cli/dev-alpha-command-structure.md +++ b/docs/cli/dev-alpha-command-structure.md @@ -2,7 +2,9 @@ ## Purpose -This document defines the alpha command structure for the new Supabase CLI. +This document defines the command structure for the new Supabase CLI. + +The `schema` and `migrations` verbs ship on the stable (legacy) shell only. Singular `migration` on stable remains the Go-parity group. For alpha, we will design the command surface from `supabase dev` outward. The goal is not to mirror the old CLI or the Management API. The goal is to give both humans and LLMs one command set that feels obvious, consistent, and reusable. @@ -27,13 +29,13 @@ For alpha, we will use `schema` as the user-facing command group for database sh For alpha, the declarative schema workflow comes first. `schema` is the default path we will teach, document, and optimize for. -`schema generate` means "turn my declared schema intent into migration files without applying them yet." +`schema generate` means "turn my declared schema intent into migration files without applying them yet." `--dry-run` previews that same pipeline. -`schema apply` means "apply my declared schema intent to the local database." Under the hood, that may derive or update migration files before applying them, but the public workflow stays schema-first. +`schema apply` means "apply my declared schema intent to a verified-disposable local database." It does not write migration files. -`schema push` means "sync my declared schema intent to the platform." In practice, that can include deriving or updating migrations and then pushing that result to the platform as one schema-first workflow. It is a platform-sync command, not a local database mutation command. +There is no `schema push`. Durable remotes change only through `migrations push`. -`schema pull` means "pull schema state from the platform into the local schema representation." It is the reverse platform-sync command. +`schema pull` means "introspect a database into declarative SQL." The database is authoritative; pull does not merge. ### `migrations` is the advanced escape hatch @@ -49,7 +51,8 @@ For alpha, `push` and `pull` mean sync with the platform only. That rule applies across: -- `schema push` / `schema pull` +- `schema pull` +- `migrations push` / `migrations pull` - `functions push` / `functions pull` - `config push` / `config pull` - `env push` / `env pull` @@ -76,7 +79,7 @@ For alpha, we will use `push` and `pull` for platform Edge Function sync. This keeps the command language consistent across platform-sync asset types: -- `schema push` +- `migrations push` - `functions push` - `config push` - `env push` @@ -128,14 +131,13 @@ The public command surface for alpha is: - `supabase push` - `supabase pull` - Schema - - `supabase schema diff` - - `supabase schema generate` - - `supabase schema apply` - - `supabase schema push` - `supabase schema pull` + - `supabase schema generate` (`--dry-run` previews the same pipeline) + - `supabase schema apply` - Migrations - `supabase migrations new` - `supabase migrations list` + - `supabase migrations diff` - `supabase migrations apply` - `supabase migrations push` - `supabase migrations pull` @@ -203,7 +205,7 @@ The local workflow should feel like a single command, but it should still be bui At a high level, it will coordinate: -- `schema push` +- `migrations push` - `functions push` - remote config sync @@ -288,7 +290,7 @@ Some users will need more control than the high-level schema workflow provides. ### Why platform-only `push` and `pull` improve learnability -Using `push` and `pull` only for platform sync creates one directional vocabulary for the entire CLI. Once a user understands `schema push`, it is natural to understand `functions push`, `config push`, `env push`, and then top-level `push` without wondering whether the command will mutate a local database. +Using `push` and `pull` only for platform sync creates one directional vocabulary for the entire CLI. Once a user understands `migrations push`, it is natural to understand `functions push`, `config push`, `env push`, and then top-level `push` without wondering whether the command will mutate a local database. There is no `schema push` in V1: durable schema changes go through migration files. ### Why `apply` is clearer than overloading `push` @@ -315,7 +317,7 @@ The public command surface is: For database changes specifically, the alpha model is: -- `schema` for declarative authoring, diffing, generation, local apply, and schema-first platform sync -- `migrations` for direct file-level control, explicit local application, and explicit migration-level platform sync +- `schema` for declarative authoring, pull, generation (`--dry-run` preview), and local apply +- `migrations` for file-level control, `diff`, local apply, and the only remote schema path (`push` / `pull`) `dev` will orchestrate this command tree rather than replace it. diff --git a/docs/cli/schema-first-v1-plan.md b/docs/cli/schema-first-v1-plan.md new file mode 100644 index 0000000000..9c0a87ebdb --- /dev/null +++ b/docs/cli/schema-first-v1-plan.md @@ -0,0 +1,86 @@ +# Plan: Schema-First Database Development (V1) + +This is the implementation plan for the Schema-First RFC (revised 2026-08-18). +It is the working spec for the `feat/implement-rfc-schema-first-development` branch. + +## Product decisions (locked) + +1. Declarative SQL in `supabase/schemas/*.sql` is the primary source of database-shape intent. +2. Migrations in `supabase/migrations/*.sql` are the durable deployment recipe. +3. `schema apply` may journal-apply a pg-delta plan only to a verified-disposable local target. +4. `migrations push` is the only CLI path that mutates a durable remote schema. No `schema push`. +5. `schema generate` is declarations → migrations. `schema pull` is database → declarations. +6. `schema generate --dry-run` previews generate. `migrations diff` replaces `db diff`. +7. `schema pull` is database-authoritative regeneration (`--force` / `--output`). No merge. +8. `--yes` answers ordinary prompts. Durable identity is `--yes` or matching `--project-ref` for linked targets, and `--allow-remote` for raw URLs. There is no `--allow-data-loss`. Local disposable `schema apply` auto-approves modeled hazards. +9. New commands live at top level in the **legacy** (stable) CLI only. `next/` is going away and must not grow these verbs. Go-parity `db` and singular `migration` stay on stable. Plural `migrations` is the schema-first group (it is no longer an alias of `migration`). +10. `schema generate` / `apply` / `migrations diff|push|pull` plan against **isolated Docker shadows** restored from the existing tar cache (`$SUPABASE_HOME/cache/shadow-baseline/shadow-baseline-.tar`) — the same pool `#6223` shares with the main DB. Not native Postgres binaries, and not co-located `CREATE DATABASE` shadows. `planSchemaFiles` always uses `isolatedShadow: true` (separate cluster, provisioned as a Docker container). + +## Open questions (resolved for V1) + +| # | Decision | +| - | -------- | +| 1 | No tracked schema checkpoint sidecar. Export ownership stays in `.pgdelta-export.json`. Draft journal: `.supabase/schema-draft.json` (gitignored). Existing `.schema-checkpoint.json` files are ignored. | +| 2 | After a successful non-dry `schema generate` (including no-op when `M` already equals `D`), the draft journal is cleared. Local history is never written at generate time. `migrations apply` runs pending SQL, or inserts missing `supabase_migrations` rows for the longest pending prefix whose replay already matches the live catalog. Catalog match is schema-shape only — a pending DML-only file can be recorded without executing. Local `db reset` clears the journal immediately after the database is recreated. Reset is optional rebuild, not required to apply additive files. | +| 3 | `schema generate --baseline --name ` is the existing-database onboarding step. Registering that baseline as already applied on a remote is a separate, explicit history operation: `supabase migration repair --status applied `. It is not pull, and the CLI does not auto-repair. Push catalog-gap errors emit `migrations diff --against linked` then `migration repair --status applied`. Remote-only versions emit `migrations pull`. | +| 4 | Manual migration changes during an active ungenerated draft fail closed with generate / reset / discard. No automatic rebase. | +| 5 | Push does not classify pending files as destructive. Generate still fail-closes on ambiguous rename / coverage gaps. | +| 6 | Only a running local stack owned by this project is verified-disposable. Every remote/URL target is durable. No environment classification yet. | +| 7 | Clone proof is not required. Generate verifies by planning `M → D` and checking convergence on a clean replay. Push live-verifies declarations-ahead (`M → D`) and remote drift unless `--skip-verify`. | +| 8 | `migrations diff` supports `--file` / `-f` (preview-to-file, no apply). | +| 9 | Isolated shadows are Docker containers restored from `$SUPABASE_HOME/cache/shadow-baseline/`. The local target is this project's `supabase_db_` container. Co-located shadows and native Postgres binaries are not used on this path. | + +## Shell and ownership + +``` +apps/cli/src/shared/schema/ engine adapter, workspace, journal, use cases +apps/cli/src/shared/migrations/ repository/runner services, use cases +apps/cli/src/shared/database/ target resolution, pool, mutation auth +apps/cli/src/legacy/schema/ Docker shadows, Docker local target, linked connector, live repository/runner +apps/cli/src/legacy/commands/schema/ +apps/cli/src/legacy/commands/migrations/ +``` + +- `next/` must not import `legacy/`. `legacy/` must not import `next/`. +- Handlers call one use case and render. Handlers must not import other handlers. +- Use cases live in `shared/`. Live Docker/legacy helpers are provided from `legacy/schema/` layers (`shared/` cannot import `legacy/`). +- pg-delta stays the compiler. The CLI owns paths, targets, prompts, locks, and output. + +## Command surface (stable / legacy) + +| Command | Source → action | Side effects | +| ------- | --------------- | ------------ | +| `schema pull` | L/R → D | Declarative files, manifest (primary tree only) | +| `schema generate --dry-run` | M → D | None | +| `schema generate` | M → D | Migration files; clears draft journal | +| `schema apply` | L → D | Local DB + draft journal | +| `migrations new` | — | Empty migration file | +| `migrations list` | files ↔ history | None | +| `migrations diff` | M → live | Preview (optional `--file`) | +| `migrations apply` | pending files → L | Local DB + `supabase_migrations` | +| `migrations push` | pending files → R | Remote DB + history; fail closed on declarations-ahead or catalog gap unless `--skip-verify`. Remote-only versions → `migrations pull`. Histories aligned + catalog gap → privilege-offer, or `migrations diff --against linked` then `migration repair --status applied`. First push (empty history): privilege-offer if ACL-only; pending files that already match the live catalog → `migration repair --status applied`; otherwise show SQL + confirm, then apply. | +| `migrations pull` | remote `schema_migrations.statements` → files | Writes `supabase/migrations/_.sql`. Does not execute SQL. Does not catalog-diff. | + +Go-parity `db` / singular `migration` commands are unchanged on stable. This prototype does not add `next/` aliases or deprecations. + +## Safety + +- Target identity comes from stack ownership or linked project-ref, never hostname heuristics. +- `--yes` never bypasses the target gate or live verify (unless `--skip-verify`). +- Durable identity: interactive confirm-by-typing-ref; non-interactive `--yes` or matching `--project-ref`. +- `DATABASE_URL` / `SUPABASE_DB_URL` (and `--db-url`) are unverifiable URL targets: no `projectRef`, mutations require `--allow-remote`. An env URL is enough even when the project is not linked. Unset those env vars to use the linked project connection. +- Linked sockets (no env URL) come from the TypeScript linked resolver (`legacyResolveLinkedConn`) on the stable CLI. +- Raw URL targets: `--allow-remote` instead of ref assertion. +- Local `schema apply`: auto-approve modeled hazards. Ambiguous rename / coverage gap / unknown metadata still fail closed. +- `--skip-verify` skips push’s isolated-shadow declarations-ahead and remote-drift checks. Identity flags unchanged. +- Project lock: `.supabase/schema.lock`. + +## Out of scope + +- Top-level `push` / `pull` composition (CLI-1271 / CLI-1272) +- Composite `schema push` +- Semantic three-way merge +- File watcher / TUI +- Replacing Go-parity `db` / singular `migration` on stable +- Same verbs on `next/`, native Postgres binaries, or a private native-shadow cache +- Marketing "provable no-data-loss" diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index d2627512d5..3c258f1866 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -155,8 +155,8 @@ importers: specifier: workspace:* version: link:../../packages/config '@supabase/pg-delta': - specifier: 1.0.0-alpha.42 - version: 1.0.0-alpha.42(@supabase/pg-topo@1.0.0-alpha.5) + specifier: 1.0.0-alpha.46 + version: 1.0.0-alpha.46(@supabase/pg-topo@1.0.0-alpha.5) '@supabase/pg-topo': specifier: 1.0.0-alpha.5 version: 1.0.0-alpha.5 @@ -1363,6 +1363,13 @@ packages: '@emnapi/core': ^1.7.1 || ^2.0.0-alpha.3 '@emnapi/runtime': ^1.7.1 || ^2.0.0-alpha.3 + '@napi-rs/wasm-runtime@1.2.3': + resolution: {integrity: sha512-UMduMbqO5s5zF2NkNacMT/yK5Y5QiKvWr2+50bzIIxFDwVJ2h49b+oyjaCGPhJxd2/gC2x39EHv/gHVuu36x2Q==} + engines: {node: ^20.19.0 || ^22.13.0 || >=23.5.0} + peerDependencies: + '@emnapi/core': ^1.7.1 || ^2.0.0-alpha.4 + '@emnapi/runtime': ^1.7.1 || ^2.0.0-alpha.4 + '@next/env@16.3.0': resolution: {integrity: sha512-o9r1S0BNiNreHP9Vs+Qnqd9kviDkJh8xIACY7UFZSmiGbbQRzPBBosvHzAU4TULHOIuOj/18RSsyz2qrREmIFw==} @@ -2839,8 +2846,8 @@ packages: resolution: {integrity: sha512-gfv481mTOVWtZIJgXupxZpni2V2UWPf6jeF/jOK7HdMHdH+mt6sU0sHHwf0POsPip8ltlulu9OUHgwVzl5ddRw==} engines: {node: '>=22.0.0'} - '@supabase/pg-delta@1.0.0-alpha.42': - resolution: {integrity: sha512-E1t30VEBu4ZZF6fK90iVfBT3AJTXM70XOfeMnJQ0vh9kRSuVOnVoYXjWh9/Nf/faJ2+kOCRNGKiUDQaTAbzTzQ==} + '@supabase/pg-delta@1.0.0-alpha.46': + resolution: {integrity: sha512-PaziTZjZk+zMw+wL2iBR0kJB1rOMGPCanYYYjb2pxwINCAr+XMNjnbWxTDMVZjNhPdOYLe34ocpO2lHLv+LK1A==} engines: {node: '>=20.0.0'} hasBin: true peerDependencies: @@ -8016,17 +8023,17 @@ snapshots: '@emnapi/runtime': 1.4.5 '@tybys/wasm-util': 0.9.0 - '@napi-rs/wasm-runtime@1.2.2(@emnapi/core@1.11.1)(@emnapi/runtime@1.11.1)': + '@napi-rs/wasm-runtime@1.2.2(@emnapi/core@1.11.2)(@emnapi/runtime@1.11.2)': dependencies: - '@emnapi/core': 1.11.1 - '@emnapi/runtime': 1.11.1 + '@emnapi/core': 1.11.2 + '@emnapi/runtime': 1.11.2 '@tybys/wasm-util': 0.10.3 optional: true - '@napi-rs/wasm-runtime@1.2.2(@emnapi/core@1.11.2)(@emnapi/runtime@1.11.2)': + '@napi-rs/wasm-runtime@1.2.3(@emnapi/core@1.11.1)(@emnapi/runtime@1.11.1)': dependencies: - '@emnapi/core': 1.11.2 - '@emnapi/runtime': 1.11.2 + '@emnapi/core': 1.11.1 + '@emnapi/runtime': 1.11.1 '@tybys/wasm-util': 0.10.3 optional: true @@ -8997,7 +9004,7 @@ snapshots: dependencies: '@emnapi/core': 1.11.1 '@emnapi/runtime': 1.11.1 - '@napi-rs/wasm-runtime': 1.2.2(@emnapi/core@1.11.1)(@emnapi/runtime@1.11.1) + '@napi-rs/wasm-runtime': 1.2.3(@emnapi/core@1.11.1)(@emnapi/runtime@1.11.1) optional: true '@rolldown/binding-win32-arm64-msvc@1.1.5': @@ -9143,7 +9150,7 @@ snapshots: dependencies: tslib: 2.8.1 - '@supabase/pg-delta@1.0.0-alpha.42(@supabase/pg-topo@1.0.0-alpha.5)': + '@supabase/pg-delta@1.0.0-alpha.46(@supabase/pg-topo@1.0.0-alpha.5)': dependencies: debug: 4.4.3(supports-color@7.2.0) pg: 8.23.0 diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index ba2a607316..1f0895bcb4 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -49,7 +49,7 @@ minimumReleaseAgeExclude: - "@effect/platform-node-shared@4.0.0-beta.107" - "@effect/sql-pg@4.0.0-beta.107" - "@effect/vitest@4.0.0-beta.107" - - "@supabase/pg-delta@1.0.0-alpha.42" + - "@supabase/pg-delta@1.0.0-alpha.46" - "@supabase/pg-topo@1.0.0-alpha.5" - "effect@4.0.0-beta.107"