Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions apps/cli-go/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,15 @@ The Supabase API client is generated from OpenAPI spec. See [our guide](api/READ

## Testing local pg-delta builds

To exercise unpublished `@supabase/pg-delta` changes inside CLI edge-runtime scripts (`db pull`, `db diff`, `db push`, etc.), publish a local build via Verdaccio in [pg-toolbelt](https://github.com/supabase/pg-toolbelt) and point the CLI at that registry.
> **Scope:** this workflow only applies to the Go binary's own edge-runtime pg-delta
> path, which the TypeScript CLI still reaches through the delegated
> `db remote commit` / `db pull --experimental` commands. The main TypeScript CLI
> bundles `@supabase/pg-delta` in-process and reads neither `PGDELTA_NPM_REGISTRY`
Comment thread
avallete marked this conversation as resolved.
> nor `supabase/.temp/pgdelta-version` — to test a local pg-delta build there,
> update the `@supabase/pg-delta` dependency pin in `apps/cli/package.json` /
> `pnpm-workspace.yaml` instead.

To exercise unpublished `@supabase/pg-delta` changes inside the Go binary's edge-runtime scripts, publish a local build via Verdaccio in [pg-toolbelt](https://github.com/supabase/pg-toolbelt) and point the Go binary at that registry.

### 1. Start Verdaccio (pg-toolbelt)

Expand Down Expand Up @@ -81,10 +89,13 @@ export PGDELTA_NPM_REGISTRY=http://host.docker.internal:4873
# or: export PGDELTA_NPM_REGISTRY=http://172.17.0.1:4873
```

Then run any pg-delta-backed command, for example:
Then run one of the delegated commands that still reach the Go binary's edge-runtime
pg-delta path (ordinary `db diff` / `db pull` run the TypeScript in-process engine and
ignore this registry), for example:

```sh
supabase db pull --db-url "$DATABASE_URL" --diff-engine pg-delta
supabase db pull --experimental --db-url "$DATABASE_URL"
# or: supabase db remote commit
```

When set, the CLI injects a scoped `.npmrc` and forwards `NPM_CONFIG_REGISTRY` into the edge-runtime container (`PgDeltaNpmRegistryOption` in `internal/utils/pgdelta_local.go`).
Expand Down
6 changes: 5 additions & 1 deletion apps/cli-go/cmd/db.go
Original file line number Diff line number Diff line change
Expand Up @@ -175,8 +175,12 @@ var (
}
)

// pg-delta is the default engine; an explicit `[experimental.pgdelta] enabled = false`
// is the rollback, overridable per run by --use-pg-delta. The historical
// SUPABASE_EXPERIMENTAL_PG_DELTA opt-in env var is no longer consulted so the
// config rollback stays authoritative.
func shouldUsePgDelta() bool {
return utils.IsPgDeltaEnabled() || usePgDelta || viper.GetBool("EXPERIMENTAL_PG_DELTA")
return utils.IsPgDeltaEnabled() || usePgDelta
}

// resolveDiffEngine reports whether `db diff` should run in pg-delta mode. The config /
Expand Down
11 changes: 9 additions & 2 deletions apps/cli-go/internal/db/declarative/declarative_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,17 @@ import (
)

func TestWriteDeclarativeSchemas(t *testing.T) {
// This verifies the main happy path for declarative export materialization:
// files are written to expected locations and config is updated accordingly.
// This verifies the main happy path for declarative export materialization
// with pg-delta explicitly disabled: files are written to expected locations
// and [db.migrations] schema_paths is updated accordingly. (With pg-delta
// enabled — the default — the config update is skipped; see the tests below.)
fsys := afero.NewMemMapFs()
require.NoError(t, afero.WriteFile(fsys, utils.ConfigPath, []byte("[db]\n"), 0644))
original := utils.Config.Experimental.PgDelta
utils.Config.Experimental.PgDelta = &config.PgDeltaConfig{Enabled: false}
t.Cleanup(func() {
utils.Config.Experimental.PgDelta = original
})

output := diff.DeclarativeOutput{
Files: []diff.DeclarativeFile{
Expand Down
3 changes: 1 addition & 2 deletions apps/cli-go/internal/db/pgcache/cache.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ import (
"github.com/jackc/pgconn"
"github.com/jackc/pgx/v4"
"github.com/spf13/afero"
"github.com/spf13/viper"
"github.com/supabase/cli/internal/gen/types"
"github.com/supabase/cli/internal/utils"
"github.com/supabase/cli/pkg/config"
Expand Down Expand Up @@ -91,7 +90,7 @@ func TryCacheMigrationsCatalog(ctx context.Context, config pgconn.Config, prefix
}

func ShouldCacheMigrationsCatalog() bool {
return utils.IsPgDeltaEnabled() || viper.GetBool("EXPERIMENTAL_PG_DELTA")
return utils.IsPgDeltaEnabled()
}

func CatalogPrefixFromConfig(config pgconn.Config) string {
Expand Down
10 changes: 0 additions & 10 deletions apps/cli-go/internal/db/start/start.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ import (
"github.com/jackc/pgconn"
"github.com/jackc/pgx/v4"
"github.com/spf13/afero"
"github.com/supabase/cli/internal/db/pgcache"
"github.com/supabase/cli/internal/migration/apply"
"github.com/supabase/cli/internal/status"
"github.com/supabase/cli/internal/utils"
Expand Down Expand Up @@ -368,15 +367,6 @@ func SetupLocalDatabase(ctx context.Context, version string, fsys afero.Fs, w io
if err := apply.MigrateAndSeed(ctx, version, conn, fsys); err != nil {
return err
}
if err := pgcache.TryCacheMigrationsCatalog(ctx, pgconn.Config{
Host: utils.Config.Hostname,
Port: utils.Config.Db.Port,
User: "postgres",
Password: utils.Config.Db.Password,
Database: "postgres",
}, "local", version, fsys, options...); err != nil {
fmt.Fprintln(os.Stderr, "Warning: failed to cache migrations catalog:", err)
}
return nil
}

Expand Down
9 changes: 1 addition & 8 deletions apps/cli-go/internal/migration/down/down.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ import (
"github.com/jackc/pgconn"
"github.com/jackc/pgx/v4"
"github.com/spf13/afero"
"github.com/supabase/cli/internal/db/pgcache"
"github.com/supabase/cli/internal/migration/apply"
"github.com/supabase/cli/internal/utils"
"github.com/supabase/cli/pkg/migration"
Expand Down Expand Up @@ -52,13 +51,7 @@ func ResetAll(ctx context.Context, version string, conn *pgx.Conn, fsys afero.Fs
if err := vault.UpsertVaultSecrets(ctx, utils.Config.Db.Vault, conn); err != nil {
return err
}
if err := apply.MigrateAndSeed(ctx, version, conn, fsys); err != nil {
return err
}
if err := pgcache.TryCacheMigrationsCatalog(ctx, conn.Config().Config, "", version, fsys); err != nil {
fmt.Fprintln(os.Stderr, "Warning: failed to cache migrations catalog:", err)
}
return nil
return apply.MigrateAndSeed(ctx, version, conn, fsys)
}

func confirmResetAll(pending []string) string {
Expand Down
5 changes: 0 additions & 5 deletions apps/cli-go/internal/utils/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,6 @@ func ToRealtimeEnv(addr config.AddressFamily) string {
type InitParams struct {
ProjectId string
UseOrioleDB bool
UsePgDelta bool
Overwrite bool
}

Expand All @@ -226,10 +225,6 @@ func InitConfig(params InitParams, fsys afero.Fs) error {
if params.UseOrioleDB {
c.Experimental.OrioleDBVersion = "15.1.0.150"
}
// The supabase init command opts new projects into pg-delta. Existing configs are
// unaffected because mergeDefaultValues ejects with this flag false (default stays
// migra), and other InitConfig callers leave it disabled.
c.Experimental.PgDeltaInitEnabled = params.UsePgDelta
// Create config file
if err := MkdirIfNotExistFS(fsys, SupabaseDirPath); err != nil {
return err
Expand Down
19 changes: 2 additions & 17 deletions apps/cli-go/internal/utils/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,22 +72,7 @@ func TestInitConfig(t *testing.T) {
assert.True(t, exists)
})

t.Run("generated config enables pgdelta when requested", func(t *testing.T) {
fsys := afero.NewMemMapFs()
params := InitParams{
ProjectId: "test-project",
UsePgDelta: true,
}

err := InitConfig(params, fsys)

require.NoError(t, err)
content, err := afero.ReadFile(fsys, ConfigPath)
require.NoError(t, err)
assert.Contains(t, string(content), "[experimental.pgdelta]\nenabled = true")
})

t.Run("generated config leaves pgdelta disabled by default", func(t *testing.T) {
t.Run("generated config enables pgdelta by default", func(t *testing.T) {
fsys := afero.NewMemMapFs()
params := InitParams{
ProjectId: "test-project",
Expand All @@ -98,7 +83,7 @@ func TestInitConfig(t *testing.T) {
require.NoError(t, err)
content, err := afero.ReadFile(fsys, ConfigPath)
require.NoError(t, err)
assert.Contains(t, string(content), "[experimental.pgdelta]\nenabled = false")
assert.Contains(t, string(content), "[experimental.pgdelta]\nenabled = true")
})

t.Run("creates config with orioledb", func(t *testing.T) {
Expand Down
7 changes: 6 additions & 1 deletion apps/cli-go/internal/utils/misc.go
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,12 @@ func GetDeclarativeDir() string {
}

func IsPgDeltaEnabled() bool {
return Config.Experimental.PgDelta != nil && Config.Experimental.PgDelta.Enabled
// pg-delta is the default diff engine: an absent [experimental.pgdelta]
// section (nil before config load) resolves to enabled. The config template
// ejects `enabled = true` as the viper default, so a section that omits the
// key also resolves to enabled; only an explicit `enabled = false` opts back
// into migra.
return Config.Experimental.PgDelta == nil || Config.Experimental.PgDelta.Enabled
}

func GetCurrentTimestamp() string {
Expand Down
5 changes: 0 additions & 5 deletions apps/cli-go/pkg/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -347,11 +347,6 @@ type (
Webhooks *webhooks `toml:"webhooks" json:"webhooks"`
PgDelta *PgDeltaConfig `toml:"pgdelta" json:"pgdelta"`
Inspect inspect `toml:"inspect" json:"inspect"`
// PgDeltaInitEnabled drives the [experimental.pgdelta] enabled value rendered
// by Eject. It is true only for the supabase init scaffold so freshly generated
// projects opt into pg-delta, and false when Eject feeds mergeDefaultValues so
// existing configs without the section keep resolving to migra (non-breaking).
PgDeltaInitEnabled bool `toml:"-" json:"-"`
}
)

Expand Down
27 changes: 21 additions & 6 deletions apps/cli-go/pkg/config/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -245,8 +245,6 @@ format_options = "not-json"

t.Run("init scaffold opts into pgdelta", func(t *testing.T) {
config := NewConfig()
// supabase init renders the scaffold with the pg-delta opt-in flag set
config.Experimental.PgDeltaInitEnabled = true
var buf bytes.Buffer
require.NoError(t, config.Eject(&buf))
fsys := fs.MapFS{"supabase/config.toml": &fs.MapFile{Data: buf.Bytes()}}
Expand All @@ -256,7 +254,7 @@ format_options = "not-json"
assert.True(t, config.Experimental.PgDelta.Enabled)
})

t.Run("absent pgdelta section falls back to migra", func(t *testing.T) {
t.Run("absent pgdelta section defaults to pg-delta", func(t *testing.T) {
config := NewConfig()
fsys := fs.MapFS{
"supabase/config.toml": &fs.MapFile{Data: []byte(`
Expand All @@ -265,11 +263,28 @@ orioledb_version = ""
`)},
}

// The default ejected by mergeDefaultValues keeps pg-delta disabled, so a config
// without the section resolves to migra (PgDelta is non-nil only for version pinning).
// The default ejected by mergeDefaultValues enables pg-delta, so a config
// without the section resolves to pg-delta.
require.NoError(t, config.Load("", fsys))
require.NotNil(t, config.Experimental.PgDelta)
assert.False(t, config.Experimental.PgDelta.Enabled)
assert.True(t, config.Experimental.PgDelta.Enabled)
})

t.Run("pgdelta section without enabled key defaults to pg-delta", func(t *testing.T) {
config := NewConfig()
fsys := fs.MapFS{
"supabase/config.toml": &fs.MapFile{Data: []byte(`
[experimental.pgdelta]
declarative_schema_path = "./db/decl"
`)},
}

// viper merges the user file over the ejected defaults key-by-key, so a
// section that omits enabled keeps the default true rather than the Go
// zero value false.
require.NoError(t, config.Load("", fsys))
require.NotNil(t, config.Experimental.PgDelta)
assert.True(t, config.Experimental.PgDelta.Enabled)
})

t.Run("explicit enabled false restores migra", func(t *testing.T) {
Expand Down
2 changes: 1 addition & 1 deletion apps/cli-go/pkg/config/db.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ type (
Seed seed `toml:"seed" json:"seed"`
Settings settings `toml:"settings" json:"settings"`
NetworkRestrictions networkRestrictions `toml:"network_restrictions" json:"network_restrictions"`
SslEnforcement *sslEnforcement `toml:"ssl_enforcement" json:"ssl_enforcement"`
SslEnforcement *sslEnforcement `toml:"ssl_enforcement" json:"ssl_enforcement"`
Vault map[string]Secret `toml:"vault" json:"vault"`
}

Expand Down
4 changes: 2 additions & 2 deletions apps/cli-go/pkg/config/templates/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -404,10 +404,10 @@ s3_access_key = "env(S3_ACCESS_KEY)"
# Configures AWS_SECRET_ACCESS_KEY for S3 bucket
s3_secret_key = "env(S3_SECRET_KEY)"

# pg-delta is the schema diff engine for db diff / db pull / db remote commit.
# pg-delta is the default schema diff engine for db diff / db pull / db remote commit.
# Set enabled = false to fall back to the legacy migra engine.
[experimental.pgdelta]
enabled = {{ .Experimental.PgDeltaInitEnabled }}
enabled = true
# Directory under `supabase/` where declarative files are written.
# declarative_schema_path = "./schemas"
# JSON string passed through to pg-delta SQL formatting.
Expand Down
4 changes: 2 additions & 2 deletions apps/cli-go/pkg/function/deploy_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ func TestDeployAll(t *testing.T) {
JSON(map[string]string{"message": "deployment already exists"})
var bulkBody []byte
gock.New(mockApiHost).
Put("/v1/projects/"+mockProject+"/functions").
Put("/v1/projects/" + mockProject + "/functions").
AddMatcher(captureBody(&bulkBody)).
Reply(http.StatusOK).
JSON(api.BulkUpdateFunctionResponse{})
Expand Down Expand Up @@ -340,7 +340,7 @@ func TestDeployAll(t *testing.T) {
Reply(http.StatusConflict).
JSON(map[string]string{"message": "deployment already exists"})
gock.New(mockApiHost).
Put("/v1/projects/"+mockProject+"/functions").
Put("/v1/projects/" + mockProject + "/functions").
Reply(http.StatusBadRequest).
JSON(map[string]string{"message": "bulk update rejected"})
// Run test
Expand Down
4 changes: 2 additions & 2 deletions apps/cli/docs/supabase/db/diff.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ Diffs schema changes made to the local or remote database.

Requires the local development stack to be running when diffing against the local database. To diff against a remote or self-hosted database, specify the `--linked` or `--db-url` flag respectively.

Runs [djrobstep/migra](https://github.com/djrobstep/migra) in a container to compare schema differences between the target database and a shadow database. The shadow database is created by applying migrations in local `supabase/migrations` directory in a separate container. Output is written to stdout by default. For convenience, you can also save the schema diff as a new migration file by passing in `-f` flag.
Compares schema differences between the target database and a shadow database, using the bundled pg-delta engine by default. The legacy [djrobstep/migra](https://github.com/djrobstep/migra) engine, which runs in a container, remains available as a fallback (see below). The shadow database is created by applying migrations in local `supabase/migrations` directory in a separate container. Output is written to stdout by default. For convenience, you can also save the schema diff as a new migration file by passing in `-f` flag.

Explicit `--from`/`--to` mode always uses pg-delta. In this mode, `-f` is ignored and stdout (or `--output`) is a flattened representation for review, not a portable apply script. Do not apply it directly with plain `psql -f`: transactional units can contain `SET LOCAL` preambles that only take effect inside a transaction, while plans that mix transactional and non-transactional units cannot safely be wrapped in one transaction. To create an applicable migration, use normal target mode with `supabase db diff -f <name>`, then apply it through `supabase db reset` locally or `supabase db push` against the linked project. These paths preserve the plan's per-unit transaction semantics.

By default, all schemas in the target database are diffed. Use the `--schema public,extensions` flag to restrict diffing to a subset of schemas.

Projects created by a recent `supabase init` default to the pg-delta diff engine (`[experimental.pgdelta] enabled = true` in `config.toml`). Existing projects are unaffected and keep using migra unless they opt in. To fall back to the legacy migra engine, set `enabled = false` under `[experimental.pgdelta]`, or pass `--use-migra` for a single run.
pg-delta is the default diff engine for all projects. To fall back to the legacy migra engine, set `enabled = false` under `[experimental.pgdelta]` in `config.toml`, or pass `--use-migra` for a single run.
Comment thread
avallete marked this conversation as resolved.

With the bundled pg-delta engine, diff SQL defaults to uppercase keywords, indent 2, a maximum width of 180, trailing commas, and column/key alignment, matching its declarative export. When `-f` writes migrations, execution-aware transaction semantics are preserved as ordered per-unit files; non-transactional units carry a directive that the CLI apply path honors. Flattened review output retains the rendered SQL and preambles, but not the unit boundaries supplied to a migration runner. Configure overrides with `[experimental.pgdelta] format_options`, or set `format_options = "null"` to emit raw, unformatted statements.

Expand Down
Loading
Loading