Skip to content

Add measured PostgreSQL/PostGIS tuning profiles and diagnostics #311

Description

@Medformatik

Problem

The Postgres/PostGIS service uses mostly upstream defaults despite workloads that include geospatial search, Overture ingestion/conflation, POI operations, job state, and account data. Copying generic tuning values would be unsafe because memory, concurrency, dataset size, and query mix vary by deployment.

Current state

  • services/postgis/service.json declares a 2 GiB memory class but no measured PostgreSQL tuning profile.
  • No repository configuration was found for shared_buffers, work_mem, planner cost constants, JIT policy, checkpoint/WAL behavior, autovacuum tuning, or pg_stat_statements.
  • Overture and data-manager jobs already have capacity/preflight concepts that can provide representative heavy workloads.

Proposed solution

Create reproducible benchmarks and a small set of conservative, memory-budgeted tuning profiles. Add diagnostics that let operators observe slow/regressed queries and validate changes with EXPLAIN (ANALYZE, BUFFERS) rather than accepting folklore defaults.

Acceptance criteria

  • A benchmark corpus covers representative search, spatial joins/conflation, ingestion/upsert, cleanup, and normal API queries.
  • Baselines capture p50/p95 runtime, buffer hits/reads, temp spill, WAL, CPU, memory, locks, and concurrent behavior.
  • pg_stat_statements (or an equally safe mechanism) is available with privacy-aware query normalization and documented reset/retention.
  • Small/default/large profiles derive memory settings from explicit container/host budgets and concurrency assumptions.
  • Aggregate worst-case memory math covers connections, parallel workers, maintenance, autovacuum, and per-operation memory.
  • Planner, JIT, random I/O, parallelism, checkpoint/WAL, connection, and autovacuum changes are individually justified by evidence.
  • PostGIS indexes/statistics and analyze/vacuum behavior are included, not only core Postgres knobs.
  • Deployment overrides are rendered deterministically and validated against supported PostgreSQL versions.
  • Automated regression checks prevent a profile from materially harming the representative read/write mix.
  • Each setting has rollback guidance; the default profile remains conservative.
  • Admin/monitoring docs explain finding slow queries, collecting plans safely, and avoiding secrets/coordinates in diagnostics.
  • Results and hardware/container assumptions are checked into a dated report.

Alternatives considered

Hard-coding values from another project ignores workload and memory differences. Leaving everything at defaults avoids tuning risk but obscures preventable spills, poor plans, and maintenance lag.

Area

PostgreSQL/PostGIS, deployment, observability, performance engineering.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestneeds-triageNeeds initial review and categorization

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions