Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,31 @@ jobs:
go clean -modcache
make build
file bin/manager
permission-repair:
name: Permission repair (PostgreSQL ${{ matrix.postgres }})
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
matrix:
postgres: ["16", "18"]
services:
postgres:
image: postgres:${{ matrix.postgres }}
env:
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 5s
--health-timeout 5s
--health-retries 10
steps:
- uses: actions/checkout@v7
- uses: actions/setup-go@v7
with:
go-version-file: go.mod
- name: Verify permission recovery and rollback
env:
PERMISSION_REPAIR_TEST_DSN: postgresql://postgres@localhost:5432/postgres?sslmode=disable
run: go test -race ./pkg/postgres -run TestPermissionRepairIntegration -count=1
127 changes: 91 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Manage external PostgreSQL databases in Kubernetes with ease—supporting AWS RD
- [Multiple Operator Support](#multiple-operator-support)
- [Secret Templating](#secret-templating)
- [Compatibility](#compatibility)
- [Additional Features](#additional-features)
- [Contributing](#contributing)
- [License](#license)

Expand Down Expand Up @@ -66,11 +67,11 @@ Set `POSTGRES_CLOUD_PROVIDER` to `AWS` via environment variable, Kubernetes Secr

Set environment variables in [`config/manager/operator.yaml`](config/manager/operator.yaml):

| Name | Description | Default |
| --- | --- | --- |
| `WATCH_NAMESPACE` | Namespace to watch. Empty string = all namespaces. | (all namespaces) |
| `POSTGRES_INSTANCE` | Operator identity for multi-instance deployments. | (empty) |
| `KEEP_SECRET_NAME` | Use user-provided secret names instead of auto-generated ones. | disabled |
| Name | Description | Default |
| ------------------- | -------------------------------------------------------------- | ---------------- |
| `WATCH_NAMESPACE` | Namespace to watch. Empty string = all namespaces. | (all namespaces) |
| `POSTGRES_INSTANCE` | Operator identity for multi-instance deployments. | (empty) |
| `KEEP_SECRET_NAME` | Use user-provided secret names instead of auto-generated ones. | disabled |

> **Note:**
> If enabling `KEEP_SECRET_NAME`, ensure there are no secret name conflicts in your namespace to avoid reconcile loops.
Expand All @@ -82,11 +83,13 @@ Set environment variables in [`config/manager/operator.yaml`](config/manager/ope
The Helm chart for this operator is located in the `charts/ext-postgres-operator` subdirectory. Follow these steps to install:

1. Add the Helm repository:

```bash
helm repo add ext-postgres-operator https://movetokube.github.io/postgres-operator/
```

2. Install the operator:

```bash
helm install -n operators ext-postgres-operator ext-postgres-operator/ext-postgres-operator
```
Expand Down Expand Up @@ -121,11 +124,13 @@ To install the operator using Kustomize, follow these steps:
1. Configure Postgres credentials for the operator in `config/default/secret.yaml`.

2. Deploy the operator:

```bash
kubectl kustomize config/default/ | kubectl apply -f -
```

Alternatively, use [Kustomize](https://github.com/kubernetes-sigs/kustomize) directly:

```bash
kustomize build config/default/ | kubectl apply -f -
```
Expand All @@ -149,11 +154,11 @@ spec:
dropOnDelete: false # Set to true if you want the operator to drop the database and role when this CR is deleted (optional)
masterRole: test-db-group (optional)
schemas: # List of schemas the operator should create in database (optional)
- stores
- customers
- stores
- customers
extensions: # List of extensions that should be created in the database (optional)
- fuzzystrmatch
- pgcrypto
- fuzzystrmatch
- pgcrypto
```

This creates a database called `test-db` and a role `test-db-group` that is set as the owner of the database.
Expand All @@ -173,14 +178,14 @@ metadata:
postgres.db.movetokube.com/instance: POSTGRES_INSTANCE
spec:
role: username
database: my-db # This references the Postgres CR
database: my-db # This references the Postgres CR
secretName: my-secret
privileges: OWNER # Can be OWNER/READ/WRITE
annotations: # Annotations to be propagated to the secrets metadata section (optional)
privileges: OWNER # Can be OWNER/READ/WRITE
annotations: # Annotations to be propagated to the secrets metadata section (optional)
foo: "bar"
labels:
foo: "bar" # Labels to be propagated to the secrets metadata section (optional)
secretTemplate: # Output secrets can be customized using standard Go templates
foo: "bar" # Labels to be propagated to the secrets metadata section (optional)
secretTemplate: # Output secrets can be customized using standard Go templates
PQ_URL: "host={{.Host}} user={{.Role}} password={{.Password}} dbname={{.Database}}"
```

Expand All @@ -191,22 +196,22 @@ This creates a user role `username-<hash>` and grants role `test-db-group`, `tes
Two `Postgres` referencing the same database can exist in more than one namespace. The last CR referencing a database will drop the group role and transfer database ownership to the role used by the operator.
Every PostgresUser has a generated Kubernetes secret attached to it, which contains the following data (i.e.):

| Key | Comment |
|----------------------|---------------------|
| `DATABASE_NAME` | Name of the database, same as in `Postgres` CR, copied for convenience |
| `HOST` | PostgreSQL server host (including port number) |
| `URI_ARGS` | URI Args, same as in `Postgres` CR, copied for convenience |
| `PASSWORD` | Autogenerated password for user |
| `ROLE` | Autogenerated role with login enabled (user) |
| `LOGIN` | Same as `ROLE`. In case `POSTGRES_CLOUD_PROVIDER` is set to "Azure", `LOGIN` it will be set to `{role}@{serverName}`, serverName is extracted from `POSTGRES_USER` from operator's config. |
| `POSTGRES_URL` | Connection string for Posgres, could be used for Go applications |
| `POSTGRES_JDBC_URL` | JDBC compatible Postgres URI, formatter as `jdbc:postgresql://{POSTGRES_HOST}/{DATABASE_NAME}` |
| `HOSTNAME` | The PostgreSQL server hostname (without port) |
| `PORT` | The PostgreSQL server port |

| Functions | Meaning |
|----------------|-------------------------------------------------------------------|
| `mergeUriArgs` | Merge any provided uri args with any set in the `Postgres` CR |
| Key | Comment |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `DATABASE_NAME` | Name of the database, same as in `Postgres` CR, copied for convenience |
| `HOST` | PostgreSQL server host (including port number) |
| `URI_ARGS` | URI Args, same as in `Postgres` CR, copied for convenience |
| `PASSWORD` | Autogenerated password for user |
| `ROLE` | Autogenerated role with login enabled (user) |
| `LOGIN` | Same as `ROLE`. In case `POSTGRES_CLOUD_PROVIDER` is set to "Azure", `LOGIN` it will be set to `{role}@{serverName}`, serverName is extracted from `POSTGRES_USER` from operator's config. |
| `POSTGRES_URL` | Connection string for Posgres, could be used for Go applications |
| `POSTGRES_JDBC_URL` | JDBC compatible Postgres URI, formatter as `jdbc:postgresql://{POSTGRES_HOST}/{DATABASE_NAME}` |
| `HOSTNAME` | The PostgreSQL server hostname (without port) |
| `PORT` | The PostgreSQL server port |

| Functions | Meaning |
| -------------- | ------------------------------------------------------------- |
| `mergeUriArgs` | Merge any provided uri args with any set in the `Postgres` CR |

### Multiple operator support

Expand All @@ -227,7 +232,7 @@ meeting the specific needs of different applications.
Available context:

| Variable | Meaning |
|-------------|------------------------------|
| ----------- | ---------------------------- |
| `.Host` | Database host |
| `.Role` | Generated user/role name |
| `.Database` | Referenced database name |
Expand All @@ -243,12 +248,62 @@ can be found [here](https://github.com/kubernetes/client-go/blob/master/README.m
Postgres operator compatibility with Operator SDK version is in the table below

| | Operator SDK version | apiextensions.k8s.io |
|---------------------------|----------------------|----------------------|
| `postgres-operator 0.4.x` | v0.17 | v1beta1 |
| `postgres-operator 1.x.x` | v0.18 | v1 |
| `postgres-operator 2.x.x` | v1.39 | v1 |
| `HEAD` | v1.39 | v1 |
| ------------------------- | -------------------- | -------------------- |
| `postgres-operator 0.4.x` | v0.17 | v1beta1 |
| `postgres-operator 1.x.x` | v0.18 | v1 |
| `postgres-operator 2.x.x` | v1.39 | v1 |
| `HEAD` | v1.39 | v1 |

## Additional Features

### AWS specific features (`cloud_provider: "AWS"`)

- Enable IAM authentication for this user (PostgreSQL on AWS RDS only)

```yaml
kind: PostgresUser
....
spec:
aws:
enableIamAuth: false # (by Default false)
```

- AWS `pg_repack` extension installation / properly alter privileges for the owner user if `cloud_provider: "AWS"`

```yaml
kind: Postgres
---
spec:
extensions:
- pg_repack
```

### Scheduled permission repair

Permission repair is optional per `Postgres`. It adds missing grants; it never
revokes custom grants, changes ownership, recreates roles or rotates credentials.

What is covered by permissionRepair:

- All application schemas.
- Tables, partitions, views, materialized views, and foreign tables.
- Sequences.
- Functions and procedures.
- Owner-managed types, domains, and large objects.
- Default privileges for future objects created by the owner role.

See [docs/permissionRepair.md](docs/permissionRepair.md) for the full reference.

```yaml
kind: Postgres
....
spec:
# Keep the existing database, masterRole and schema configuration.
permissionRepair:
schedule: "0 2 * * *" # Five cron fields; every day at 02:00 UTC
windowDuration: "30m" # Latest allowed start/end; defaults to 30m
timeout: "5m" # Maximum transaction duration; defaults to 5m
```

## Contributing

Expand Down
41 changes: 38 additions & 3 deletions api/v1alpha1/postgres_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@ import (

// PostgresSpec defines the desired state of Postgres
type PostgresSpec struct {
Database string `json:"database"`
// PermissionRepair enables scheduled, additive permission repair. Omit to disable.
// +optional
PermissionRepair *PermissionRepairSpec `json:"permissionRepair,omitempty"`
Database string `json:"database"`
// +optional
MasterRole string `json:"masterRole,omitempty"`
// +optional
Expand All @@ -22,10 +25,42 @@ type PostgresSpec struct {
Extensions []string `json:"extensions,omitempty"`
}

// PermissionRepairSpec schedules maintenance using a five-field UTC cron expression.
type PermissionRepairSpec struct {
// Schedule uses minute, hour, day of month, month, day of week, always in UTC.
// +kubebuilder:validation:MinLength=9
Schedule string `json:"schedule"`
// WindowDuration limits how late an occurrence can start, including after restart.
// +kubebuilder:default="30m"
// +optional
WindowDuration string `json:"windowDuration,omitempty"`
// Timeout bounds each transaction, also capped by the end of the window.
// +kubebuilder:default="5m"
// +optional
Timeout string `json:"timeout,omitempty"`
}

// PermissionRepairStatus persists scheduling across restarts and leader changes.
type PermissionRepairStatus struct {
// Configuration identifies the schedule configuration used for NextRunTime.
Configuration string `json:"configuration,omitempty"`
// +optional
NextRunTime *metav1.Time `json:"nextRunTime,omitempty"`
// +optional
LastAttemptTime *metav1.Time `json:"lastAttemptTime,omitempty"`
// +optional
LastSuccessTime *metav1.Time `json:"lastSuccessTime,omitempty"`
// Error is empty after a successful repair. It never contains connection credentials.
// +optional
Error string `json:"error,omitempty"`
}

// PostgresStatus defines the observed state of Postgres
type PostgresStatus struct {
Succeeded bool `json:"succeeded"`
Roles PostgresRoles `json:"roles"`
// +optional
PermissionRepair *PermissionRepairStatus `json:"permissionRepair,omitempty"`
Succeeded bool `json:"succeeded"`
Roles PostgresRoles `json:"roles"`
// +optional
// +listType=set
Schemas []string `json:"schemas,omitempty"`
Expand Down
52 changes: 52 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions charts/ext-postgres-operator/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ description: |

type: application

version: 3.0.0
appVersion: "2.4.0"
version: 3.1.0
appVersion: "2.6.0"
Loading