diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 94f165ad0..4c20dca4f 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -18,7 +18,11 @@ This page tracks significant updates to the QuestDB documentation. - [Memory limits](/docs/configuration/cairo-engine/#memory-limits) - New section covering the per-query, materialized view refresh, WAL apply, and live view refresh memory limits, what counts toward them, and what happens on a breach, plus the previously undocumented [`cairo.mat.view.max.refresh.retries`](/docs/configuration/materialized-views/#cairomatviewmaxrefreshretries), [`cairo.mat.view.refresh.busy.retry.limit`](/docs/configuration/materialized-views/#cairomatviewrefreshbusyretrylimit), [`cairo.mat.view.refresh.busy.retry.timeout`](/docs/configuration/materialized-views/#cairomatviewrefreshbusyretrytimeout), [`cairo.write.back.off.timeout.on.mem.pressure`](/docs/configuration/cairo-engine/#cairowritebackofftimeoutonmempressure), [`ram.usage.limit.bytes`](/docs/configuration/cairo-engine/#ramusagelimitbytes), and [`ram.usage.limit.percent`](/docs/configuration/cairo-engine/#ramusagelimitpercent) keys - [RBAC memory limits](/docs/security/rbac/#memory-limits) - Per-user, per-group, and per-service-account query memory limits in QuestDB Enterprise: `SET MEMORY LIMIT` on `ALTER USER`, `ALTER GROUP`, and `ALTER SERVICE ACCOUNT`, how limits resolve, the `SET MEMORY LIMIT` permission, and the upgrade migration -- [ALTER GROUP](/docs/query/sql/acl/alter-group/) - New reference page covering `SET MEMORY LIMIT` and external alias mapping +- [Resource groups](/docs/concepts/resource-groups/) - Isolate competing query workloads inside one QuestDB Enterprise instance. Covers who a policy applies to, which statements are managed, and what admission, weighted CPU and memory limits each actually guarantee, including why CPU control is cooperative and what happens while a replica's catalog is behind +- [Configure and use resource groups](/docs/operations/resource-groups/) - Requirements, choosing limits, and six worked scenarios: an instance that stops answering while CPU looks idle, keeping dashboards responsive, throttling a background job, bounding a workload's memory, governing a service account, and sharing one instance between teams, plus inspection, monitoring, client errors and troubleshooting +- [Resource groups configuration](/docs/configuration/resource-groups/) - The `resource.groups.enabled` master switch and `resource.groups.process.memory.limit.bytes` process ceiling, neither reloadable +- [CREATE RESOURCE GROUP](/docs/query/sql/acl/create-resource-group/), [ALTER RESOURCE GROUP](/docs/query/sql/acl/alter-resource-group/) and [DROP RESOURCE GROUP](/docs/query/sql/acl/drop-resource-group/) - New reference pages for the policy statements, with the parameter table, the effect of an online change on running queries, and what happens to queries still using a dropped group +- `ALTER USER`, `ALTER GROUP` and `ALTER SERVICE ACCOUNT` are now one page per statement form, matching `ALTER TABLE`. Twenty-three pages covering `ENABLE`, `DISABLE`, `WITH PASSWORD`, `WITH NO PASSWORD`, `CREATE TOKEN`, `DROP TOKEN`, [SET MEMORY LIMIT](/docs/query/sql/acl/alter-group-set-memory-limit/), [SET RESOURCE GROUP](/docs/query/sql/acl/alter-user-set-resource-group/), [UNSET RESOURCE GROUP](/docs/query/sql/acl/alter-user-unset-resource-group/), [WITH EXTERNAL ALIAS](/docs/query/sql/acl/alter-group-with-external-alias/) and `DROP EXTERNAL ALIAS` - [Migrate QuestDB onto the Kubernetes Operator](/docs/enterprise-kubernetes-operator/getting-started/migrate/) - Move an existing Enterprise deployment onto the Operator with a replica-first cutover: restore the source backup, consume replication WAL, then promote after a controlled source drain - [Copy a schema to another instance](/docs/cookbook/operations/copy-schema-between-instances/) - Recreate one instance's tables, views, and materialized views on another from `SHOW CREATE DATABASE`, either by replaying the statements over the REST API or by dumping them to a `.sql` file, with `INCLUDE (SCHEMA)` and `INCLUDE (ACL)` for separating structure from permissions on Enterprise, plus the Web Console schema explorer as a manual alternative and the ordering caveat that comes with it - [SUBSAMPLE](/docs/query/sql/subsample/) - New SQL keyword that reduces a query result to a subset of its existing rows, without interpolating values. Covers the six methods with a diagram each: `lttb` for line charts, with a gap-preserving mode, `minmax` and `m4` for per-time-bucket envelopes, `uniform` and `cadence` for position-based selection, and `sdt` (Swinging Door Trending) for error-bounded compression with a `2 * compdev` reconstruction bound and an animated walkthrough. Also covers where the clause goes in a query, output order, `NULL` handling, and which query shapes each method accepts @@ -28,11 +32,20 @@ This page tracks significant updates to the QuestDB documentation. - Added [`cairo.sql.subsample.max.rows`](/docs/configuration/cairo-engine/#cairosqlsubsamplemaxrows), the input row limit for the `lttb`, `m4`, `minmax`, `uniform`, and `cadence` methods of `SUBSAMPLE`. It does not apply to `sdt` +### Reference + +- Added [`resource_groups()`](/docs/query/functions/meta/#resource_groups), which returns each group's policy alongside its live admission, CPU and memory counters, and [`current_resource_group()`](/docs/query/functions/meta/#current_resource_group), which reports the group a session's queries run in. `current_resource_group()` returns `DEFAULT` for an unmapped principal and `null` only when a query was never admitted to a group +- Added the [`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission, which gates creating, altering and dropping resource groups, mapping principals to them, and reading `resource_groups()`. `SQL ENGINE ADMIN` keeps its existing scope of listing and cancelling queries +- Added the [resource group metrics](/docs/operations/logging-metrics/#resource-group-metrics): nine Prometheus series per group labelled with `resource_group`, plus five instance-wide series for activation, catalog health, managed dispatch and scheduler degradation +- Added the Fiber mode properties that resource groups depend on: [`http.worker.fiber.enabled`](/docs/configuration/http-server/#httpworkerfiberenabled), [`pg.worker.fiber.enabled`](/docs/configuration/postgres-wire-protocol/#pgworkerfiberenabled), [`shared.network.worker.fiber.enabled`](/docs/configuration/shared-workers/#sharednetworkworkerfiberenabled), [`shared.query.worker.fiber.enabled`](/docs/configuration/shared-workers/#sharedqueryworkerfiberenabled) and [`mat.view.refresh.worker.fiber.enabled`](/docs/configuration/materialized-views/#matviewrefreshworkerfiberenabled), all defaulting to `true` and none reloadable + ### Updated -- [query_activity()](/docs/query/functions/meta/#query_activity) - Documented the `is_wal`, `memory_used`, and `memory_limit` columns +- [query.timeout](/docs/configuration/cairo-engine/#querytimeout) - The clock includes time spent queued for resource group admission, so a query can time out before it starts. Over PGWire each `Execute` message restarts it +- [Replication](/docs/high-availability/overview/#resource-groups-in-a-replicated-cluster) and [SWITCH ROLE](/docs/query/sql/switch-role/) - Resource group policies replicate through the WAL pipeline while enforcement stays local, so a lagging replica applies the policy it has, and a replica that has not received the catalog cannot be promoted +- [query_activity()](/docs/query/functions/meta/#query_activity) - Documented the `is_wal`, `memory_used`, and `memory_limit` columns, and added a `resource_group` column showing which group each running query was admitted to. `memory_limit` is now capped by the group budget as well, so a principal with no limit of its own still reports one when its group sets a ceiling - [wal_tables()](/docs/query/functions/meta/#wal_tables) - Documented the `errorTag`, `errorMessage`, and `memoryPressure` columns; `errorTag` reads `OUT OF MEMORY` after a WAL apply memory limit breach -- [SHOW](/docs/query/sql/show/) - `SHOW USERS`, `SHOW GROUPS`, and `SHOW SERVICE ACCOUNTS`, including their filtered forms, gain a trailing `memory_limit` column. Clients that read these results by position need [updating](/docs/security/rbac/#memory-limit-upgrade) +- [SHOW](/docs/query/sql/show/) - `SHOW USERS`, `SHOW GROUPS`, and `SHOW SERVICE ACCOUNTS`, including their filtered forms, gain a trailing `memory_limit` column. Clients that read these results by position need [updating](/docs/security/rbac/#memory-limit-upgrade). The unfiltered forms also gain `resource_group`, and `SHOW GROUPS` additionally `resource_group_priority` - [CREATE GROUP](/docs/query/sql/acl/create-group/) - Documented the `WITH EXTERNAL ALIAS` form that creates a group and its OIDC or LDAP mapping in one statement - [Kafka connector](/docs/connect/message-brokers/kafka/) - Updated for QWP with a quick start, guidance on preventing duplicates and recovering from outages, and migration steps for existing HTTP pipelines - [Kubernetes Operator](/docs/enterprise-kubernetes-operator/) - Refreshed for Operator 0.2.1 across installation, configuration, high availability, backup and restore, known limitations, troubleshooting, and the generated [API reference](/docs/enterprise-kubernetes-operator/reference/api/) diff --git a/documentation/concepts/resource-groups.md b/documentation/concepts/resource-groups.md new file mode 100644 index 000000000..0cf301d0c --- /dev/null +++ b/documentation/concepts/resource-groups.md @@ -0,0 +1,316 @@ +--- +title: Resource groups +sidebar_label: Resource groups +description: + Resource groups isolate query workloads inside one QuestDB instance. Learn how + a query is assigned to a group, and what admission, CPU weight and memory + limits actually guarantee. +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +A resource group is a named policy that limits what a set of principals may +consume while their queries run. One instance typically serves several workloads +at once: dashboards that must answer in milliseconds, an ad-hoc analyst, and a +nightly report that scans a year of data. When these workloads compete without +resource controls, the report can increase dashboard latency. +[Common scenarios](/docs/operations/resource-groups/#common-scenarios) walks +through that case and five others as policies you can copy. + +Creating one takes two statements: +[`CREATE RESOURCE GROUP`](/docs/query/sql/acl/create-resource-group/) for the +policy, and a mapping that puts principals under it. + +```questdb-sql title="Half the default CPU share, four concurrent queries, and a 2 GiB memory budget" +CREATE RESOURCE GROUP reporting WITH ( + cpu_weight = 50, + max_active_queries = 4, + memory_limit = '2G' +); + +ALTER USER analyst SET RESOURCE GROUP reporting; +``` + +A policy sets three controls, and they differ in how strong a guarantee they +give: + +- [Admission](#admission-is-a-hard-gate) is how many queries a group may run at + once, how many may wait, and how long they may wait. It is an exact limit. +- [Weighted CPU](#cpu-weight-is-a-share-not-a-reservation) is the share of query + CPU a group receives while groups compete. It is a share, not a reservation. +- [Memory](#memory-limits-use-batched-accounting) is the process and group + budgets for tracked native query memory. It is approximate at the margin. + +The design is [cooperative](#how-cooperative-cpu-scheduling-works). QuestDB +executes query work on shared worker pools, and resource groups do not create +one operating-system thread pool per group. A query and all of its parallel +tasks use the same resource group, while every worker stays available to every +group. + +## Who a resource group applies to + +A policy attaches to **principals**, and a principal is one of three things: a +user, an ACL group, or a service account. Assignment follows the authenticated +principal, not the statement. + +```questdb-sql title="The three kinds of principal you can map" +ALTER USER analyst SET RESOURCE GROUP reporting; +ALTER GROUP analysts SET RESOURCE GROUP reporting MAPPING PRIORITY 10; +ALTER SERVICE ACCOUNT ingest_bot SET RESOURCE GROUP reporting; +``` + +Mapping an ACL group covers a team without naming each member. Only users +inherit a mapping this way, since only a user can belong to an ACL group, and a +mapping on the user itself beats the one it would inherit. Each clause is +documented with the statement it belongs to: +[`ALTER USER`](/docs/query/sql/acl/alter-user-set-resource-group/), +[`ALTER GROUP`](/docs/query/sql/acl/alter-group-set-resource-group/) and +[`ALTER SERVICE ACCOUNT`](/docs/query/sql/acl/alter-service-account-set-resource-group/). + +A user can belong to several ACL groups, so each ACL group mapping carries a +priority and the highest one wins: + +```questdb-sql title="A user in both groups resolves to dashboards, the higher priority" +ALTER GROUP analysts SET RESOURCE GROUP adhoc MAPPING PRIORITY 10; +ALTER GROUP oncall SET RESOURCE GROUP dashboards MAPPING PRIORITY 20; +``` + +:::note + +If two ACL groups carry the same priority, the tie breaks on whichever resource +group was created first. That is rarely what anyone intends, so give ACL group +mappings distinct priorities when the order matters. + +::: + +Anything not mapped runs in `DEFAULT`, which always exists and carries no limits +of its own: a CPU weight of 100, unlimited admission, and the instance-wide +memory limits. You can change its policy, but you cannot drop or rename it. + +A session that assumes a service account keeps the group of the principal that +logged in, while the service account's own mapping applies to sessions that +authenticate as that account. + +The group is resolved once, when the query starts, and stays fixed for the +statement's lifetime. Changing a mapping affects statements that start after the +change, never one already running. + +## Which statements are managed + +Resource groups govern the statements that read data: + +- `SELECT` +- the source query of `CREATE TABLE ... AS SELECT` and `INSERT ... SELECT` +- query exports + +Everything else runs outside the feature and consumes no admission slot, CPU +grant or group memory budget: `EXPLAIN`, value `INSERT`, `UPDATE`, ordinary DDL, +`COPY`, transaction and session control, ILP and QWP ingestion, WAL apply, +materialized and live view refresh, and QuestDB's own internal SQL. + +For `CREATE TABLE ... AS SELECT` and `INSERT ... SELECT` the group is charged +for reading the source and producing rows, including parallel work. Where +writing a row cannot be separated from producing it, that CPU is charged to the +group as well. The commit, durability and any work handed to writer or WAL +queues are outside the guarantee. + +Resource groups account **tracked native query memory**. They do not represent +JVM heap, resident set size, memory-mapped table pages or long-lived engine +caches. Existing process memory protection remains the outer boundary. + +## What each control guarantees + +### Admission is a hard gate + +```questdb-sql title="At most four running, 32 more waiting, and no wait longer than 15 seconds" +CREATE RESOURCE GROUP reporting WITH ( + max_active_queries = 4, + max_queued_queries = 32, + queue_timeout = '15s' +); +``` + +`max_active_queries` is an exact count. A group at its limit queues the next +query until a slot frees, up to `max_queued_queries`; beyond that the query is +rejected immediately. A queued query that waits longer than `queue_timeout` +fails, and the instance-wide `query.timeout` keeps running while it waits, so +whichever of the two expires first ends the wait. + +A slot is held only while the query is running on a worker. A protocol cursor +that is suspended between pages releases its slot and passes through the gate +again when the client asks for more rows, so a paging client does not hold +capacity while the application thinks. The consequence is that admission can be +refused on a later page: a client that received its first rows may still see the +queue-full or timeout error when it asks for more, and the connection stays +usable. + +### CPU weight is a share, not a reservation + +```questdb-sql title="Dashboards get twice the CPU of ad-hoc work while both are busy" +CREATE RESOURCE GROUP dashboards WITH (cpu_weight = 100); +CREATE RESOURCE GROUP adhoc WITH (cpu_weight = 50); +``` + +Weights only matter when groups compete. A group that is alone on the instance +uses everything it can, regardless of its weight. When two groups both have +work, the scheduler hands out CPU so that measured CPU divided by `cpu_weight` +stays balanced: weights 100 and 50 converge to a 2:1 split of query CPU. A query +that was running alone starts sharing at its next checkpoint. + +Weights are relative. 100 and 50 are the same as 2 and 1. A group that becomes +active starts level with the groups already running, so it neither banks the CPU +it did not use while idle nor is punished for having been busy. + +Shares are between groups, not between queries. Within a group, work is served +in arrival order, and a parallel query can hold several places in that order, so +there is no promise of equal CPU between individual queries. + +### Memory limits use batched accounting + +```questdb-sql title="Everything this group runs at once must fit in 8 GiB" +CREATE RESOURCE GROUP reporting WITH (memory_limit = '8G'); +``` + +Accounting has three levels: query, group and process. Allocation and release +deltas accumulate on the executing worker and are published to the shared +counters in batches. Exceeding a checked limit fails the query with +`query memory limit exceeded`; it does not queue the allocation until memory +becomes available. + +The single-query ceiling starts with the principal's effective query memory +limit, when set, or the instance default `cairo.query.memory.limit.bytes`. Any +group `memory_limit` and process memory budget further cap that ceiling. The +group budget also bounds the total tracked memory held by its queries; the +process budget covers tracked native query memory across groups. + +An unset group `memory_limit`, or one set to `0` or `UNLIMITED`, adds no group +ceiling. A process budget of `0` adds no process ceiling. Existing single-query +limits still apply, and memory accounting remains enabled even when all limits +are unlimited. + +The counters can temporarily omit worker-local deltas, so a group can briefly +overshoot its limit by less than 64 KiB per worker running its queries. These +budgets are not byte-exact, instantaneous ceilings. + +## How cooperative CPU scheduling works + +QuestDB does not preempt a running query. The scheduler grants a query a short +slice of CPU on a worker and expects it to reach a cooperative checkpoint, which +is the same circuit breaker check that makes queries cancellable. At that point +the query either renews its grant or yields the worker to another group. + +A yielded worker runs other queries and, within a bounded window, returns to +accepting connections; the query that yielded resumes later. A long +single-threaded query that reaches these checkpoints therefore shares its worker +before finishing, which keeps the instance responsive while heavy queries run. + +Two consequences follow. + +The guarantee is statistical over a short window. Between checkpoints a query +holds its worker, so instantaneous CPU can deviate from the configured share. +The CPU actually used is charged either way, so a query that overran repays it +and the average is preserved. + +A query that cannot reach a checkpoint keeps its worker. While CPU scheduling is +engaged its CPU is still charged when the slice ends, but no cooperative limit +can shorten that stretch. + +Faults are contained rather than escalated. If the scheduler itself hits an +internal fault it degrades: queries keep running without CPU grants, admission +and memory limits stay enforced, and +`questdb_resource_groups_cpu_scheduler_degraded` reports `1` until the instance +restarts. A fault in one query affects only that query, leaving other queries +and other groups alone, and its CPU is still charged to its group. + +### When scheduling engages + +Slicing happens only while CPU scheduling is engaged, and that requires a second +resource group to exist. While `DEFAULT` is the only resource group, scheduling +stays disengaged: no CPU is sampled, no query yields, and every query holds its +worker exactly as it does with the feature disabled. Registration, admission and +memory accounting still run, so this is not free, but the cost is a fixed few +microseconds per query. + +Scheduling engages when a second group is created and disengages once the last +other group has been dropped and its queries have finished. A query that is +already running when a group is created stays outside scheduling until it next +suspends or finishes; the new policy applies to queries that start or resume +after the change. + +Once scheduling is engaged, a query whose group is alone still keeps its worker: +at each checkpoint it renews its grant in place and yields only when another +query is waiting for its worker. + +Queries are assigned to a group throughout, whether or not scheduling is +engaged. Disengaged scheduling means no CPU slicing, not that the feature +stepped aside, so +[`current_resource_group()`](/docs/query/functions/meta/#current_resource_group) +returns `DEFAULT` on such an instance rather than `null`. `null` is reserved for +queries that were never admitted to a group at all, which happens when the +feature is disabled and on a replica whose catalog has not arrived. What does go +quiet is the CPU accounting: `cpu_nanos_total` and `cpu_wait_nanos_total` both +stay at `0` while scheduling is disengaged, because no CPU is sampled and no +query waits for a grant. Admission and memory counters keep working throughout. + +## Replication and catalog lag {#behaviour-under-failure-and-on-replicas} + +Group definitions and mappings live in a replicated system catalog, so a replica +receives them through normal replication. Three things follow while a replica is +behind. + +**A replica that has never received the catalog runs its queries unmanaged.** No +admission slot, no CPU grant, no group memory budget, and +[`current_resource_group()`](/docs/query/functions/meta/#current_resource_group) +returns `null`. They are neither rejected nor quietly run under `DEFAULT`, and +they are counted in +`questdb_resource_groups_catalog_lag_unmanaged_queries_total`. + +**A replica that is behind on a policy change keeps applying the policy it +has.** Once it has the catalog, its queries are managed against the snapshot it +holds, so a group whose weight you just changed keeps the old weight on that +replica until the change arrives. Nothing blocks or errors. + +**A lagging replica cannot be promoted while the feature is enabled.** Promotion +validates the catalog before writes are admitted, because a node about to accept +writes must not enforce a policy it cannot see. The switch is refused and the +node keeps serving reads; wait for replication and retry. With the feature +disabled the condition is logged and the promotion proceeds. A primary that +predated resource groups may never have created the catalog table at all, in +which case the promoted node creates it and continues. + +For the recovery steps, see +[troubleshooting](/docs/operations/resource-groups/#troubleshooting), and for how +a refused switch behaves in general, see +[refusals and the torn state](/docs/high-availability/failover/#refusals-and-the-torn-state). + +## See also + +- [Configure and use resource groups](/docs/operations/resource-groups/) +- [Resource groups configuration](/docs/configuration/resource-groups/) + +Managing groups: + +- [CREATE RESOURCE GROUP](/docs/query/sql/acl/create-resource-group/) +- [ALTER RESOURCE GROUP](/docs/query/sql/acl/alter-resource-group/) +- [DROP RESOURCE GROUP](/docs/query/sql/acl/drop-resource-group/) + +Mapping principals: + +- [ALTER USER SET RESOURCE GROUP](/docs/query/sql/acl/alter-user-set-resource-group/) + and [UNSET](/docs/query/sql/acl/alter-user-unset-resource-group/) +- [ALTER GROUP SET RESOURCE GROUP](/docs/query/sql/acl/alter-group-set-resource-group/) + and [UNSET](/docs/query/sql/acl/alter-group-unset-resource-group/) +- [ALTER SERVICE ACCOUNT SET RESOURCE GROUP](/docs/query/sql/acl/alter-service-account-set-resource-group/) + and [UNSET](/docs/query/sql/acl/alter-service-account-unset-resource-group/) + +Inspecting: + +- [`resource_groups()`](/docs/query/functions/meta/#resource_groups) +- [`current_resource_group()`](/docs/query/functions/meta/#current_resource_group) +- [Resource group metrics](/docs/operations/logging-metrics/#resource-group-metrics) +- [Role-based access control](/docs/security/rbac/) diff --git a/documentation/configuration/cairo-engine.md b/documentation/configuration/cairo-engine.md index d4b1b2262..36615c47a 100644 --- a/documentation/configuration/cairo-engine.md +++ b/documentation/configuration/cairo-engine.md @@ -67,6 +67,11 @@ When `false`, disables the `reload_config()` SQL function. A global timeout for long-running queries, given as a duration: `500ms`, `120s`, `2m` and `1h` are all valid, and a plain number is read as milliseconds. +The timer starts when the server receives the statement and includes any time +the query spends queued for Resource Group admission. Over PGWire each `Execute` +message restarts it, so a client that fetches a cursor in batches is timed per +batch rather than across the whole result. + This key replaces `query.timeout.sec`. When both are set, `query.timeout` takes precedence; when neither is set, queries time out after 60 seconds. diff --git a/documentation/configuration/http-server.md b/documentation/configuration/http-server.md index cb8971ee4..d94598855 100644 --- a/documentation/configuration/http-server.md +++ b/documentation/configuration/http-server.md @@ -63,6 +63,22 @@ worker count. Number of threads in the private HTTP worker pool. When `0`, the HTTP server uses the shared worker pool. Values above `0` enable a private pool. +### http.worker.fiber.enabled + +- **Default**: `true` +- **Reloadable**: no + +Runs the private HTTP worker pool in Fiber mode, where a query can suspend and +release its worker instead of holding it until it finishes. Setting this to +`false` puts the pool in legacy mode. + +This applies only when `http.worker.count` is above `0`. With the default of +`0`, HTTP runs on the shared network pool and +[`shared.network.worker.fiber.enabled`](/docs/configuration/shared-workers/#sharednetworkworkerfiberenabled) +is the setting that governs it instead. QuestDB Enterprise +[resource groups](/docs/concepts/resource-groups/) require Fiber mode on +whichever pool actually serves HTTP. + ### http.worker.haltOnError - **Default**: `false` diff --git a/documentation/configuration/materialized-views.md b/documentation/configuration/materialized-views.md index 92182160b..5a445e983 100644 --- a/documentation/configuration/materialized-views.md +++ b/documentation/configuration/materialized-views.md @@ -85,6 +85,19 @@ Comma-separated list of numerical CPU core indexes. Number of dedicated worker threads assigned to refresh materialized views. When `0`, uses the shared worker pool. +## mat.view.refresh.worker.fiber.enabled + +- **Default**: `true` +- **Reloadable**: no + +Runs the materialized view refresh pool in Fiber mode, where work can suspend +and release its worker instead of holding it until it finishes. Setting this to +`false` puts the pool in legacy mode. + +Unlike the pools that serve user queries, this one is not a prerequisite for +QuestDB Enterprise [resource groups](/docs/concepts/resource-groups/), because +materialized view refresh runs outside them. + ## mat.view.refresh.worker.haltOnError - **Default**: `false` diff --git a/documentation/configuration/postgres-wire-protocol.md b/documentation/configuration/postgres-wire-protocol.md index 129f91507..4d28b700e 100644 --- a/documentation/configuration/postgres-wire-protocol.md +++ b/documentation/configuration/postgres-wire-protocol.md @@ -31,6 +31,22 @@ Comma-separated list of CPU core indexes to pin worker threads to. Example: Number of dedicated worker threads for PostgreSQL wire protocol queries. When `0`, uses the shared worker pool. +### pg.worker.fiber.enabled + +- **Default**: `true` +- **Reloadable**: no + +Runs the dedicated PostgreSQL worker pool in Fiber mode, where a query can +suspend and release its worker instead of holding it until it finishes. Setting +this to `false` puts the pool in legacy mode. + +This applies only when `pg.worker.count` is above `0`. With the default of `0`, +PostgreSQL runs on the shared network pool and +[`shared.network.worker.fiber.enabled`](/docs/configuration/shared-workers/#sharednetworkworkerfiberenabled) +governs it instead. QuestDB Enterprise +[resource groups](/docs/concepts/resource-groups/) require Fiber mode on +whichever pool actually serves PostgreSQL. + ### pg.daemon.pool - **Default**: `true` diff --git a/documentation/configuration/resource-groups.md b/documentation/configuration/resource-groups.md new file mode 100644 index 000000000..838eef819 --- /dev/null +++ b/documentation/configuration/resource-groups.md @@ -0,0 +1,79 @@ +--- +title: Resource groups +sidebar_label: Resource groups +description: + Configuration settings for QuestDB Enterprise resource groups, covering the + master switch and the process memory ceiling. +--- + +:::note + +Resource groups are [Enterprise](/enterprise/) only. + +::: + +[Resource groups](/docs/concepts/resource-groups/) isolate competing query +workloads inside one instance. These settings are instance-wide. The per-group +policy that decides admission, CPU share and memory budgets is set in SQL, not +here. See [Configure and use resource groups](/docs/operations/resource-groups/) +for those statements. + +None of these settings are reloadable: changing any of them requires a restart. + +Resource groups also require access control to be enabled (`acl.enabled=true`) +before principals can be mapped to a group, and every pool that executes SQL +must run in Fiber mode, which is the default. The +[requirements](/docs/operations/resource-groups/#requirements) list the Fiber +setting that governs each pool. What happens when a pool is in +legacy mode depends on how the feature was turned on. Left at its default, it +turns itself off and logs an error naming the pool and the setting to change. +Asked for explicitly, it fails startup with the same error, because an explicit +request and a legacy pool cannot both be honoured. + +## General + +### resource.groups.enabled + +- **Default**: `true` +- **Reloadable**: no + +Master switch. When `false`, resource group admission, CPU scheduling and group +memory accounting are disabled. Group definitions and principal mappings remain +in the catalog, so turning the feature back on restores the policies that were +already there. + +`true` also makes the catalog a hard dependency: an instance whose catalog +cannot be read does not start, and a replica whose catalog is not current is not +promoted. With `false`, both conditions are logged and ignored. See +[Behaviour under failure and on replicas](/docs/concepts/resource-groups/#behaviour-under-failure-and-on-replicas). + +Existing principal-specific and instance-default single-query memory limits +continue to apply when resource groups are disabled. + +Left unset, this resolves to `false` on an instance whose SQL pools are in +legacy mode, so upgrading such an instance does not turn the feature on and does +not stop the instance from starting. `SHOW PARAMETERS` then reports `false`, +which is the value that took effect. Set it to `true` explicitly and a legacy +pool becomes a startup error instead. + +### resource.groups.process.memory.limit.bytes + +- **Default**: `0` +- **Reloadable**: no + +Ceiling for tracked native query memory across all groups. `0` leaves the +instance without a process ceiling, which is the default. When set, it bounds +every group and every query, so no group policy can grant more than this. + +An unlimited process budget does not disable memory accounting or remove an +existing single-query limit. The group-level SQL parameter `memory_limit` treats +`0` and `UNLIMITED` as no ceiling, as does `RESET (memory_limit)`. + +This is not a process RSS limit. It covers tracked query memory only, not JVM +heap, memory-mapped table pages or long-lived engine caches. + +## See also + +- [Resource groups concept](/docs/concepts/resource-groups/) +- [Configure and use resource groups](/docs/operations/resource-groups/) +- [Identity and Access Management configuration](/docs/configuration/iam/) diff --git a/documentation/configuration/shared-workers.md b/documentation/configuration/shared-workers.md index e585b5765..be064d648 100644 --- a/documentation/configuration/shared-workers.md +++ b/documentation/configuration/shared-workers.md @@ -45,6 +45,22 @@ Number of worker threads for the network pool, which handles HTTP, PostgreSQL, and ILP server I/O. Increasing this value raises network I/O parallelism at the expense of CPU resources available to queries and writes. +## shared.network.worker.fiber.enabled + +- **Default**: `true` +- **Reloadable**: no + +Runs the network pool in Fiber mode, where a query can suspend and release its +worker instead of holding it until it finishes. Setting this to `false` puts the +pool in legacy mode. + +QuestDB Enterprise [resource groups](/docs/concepts/resource-groups/) require +Fiber mode on every pool that executes SQL. HTTP and PostgreSQL run on this pool +whenever their own worker counts are zero, which is the default, so this is +usually the setting that matters. With the pool in legacy mode, resource groups +left at their default disable themselves at startup and log the pool and setting +responsible, while `resource.groups.enabled=true` set explicitly fails startup. + ## shared.query.worker.affinity - **Default**: none @@ -63,6 +79,20 @@ operations such as filters and group-by. Increasing this value raises query parallelism at the expense of CPU resources available to network I/O and writes. +## shared.query.worker.fiber.enabled + +- **Default**: `true` +- **Reloadable**: no + +Runs the query pool in Fiber mode, where parallel query work can suspend and +release its worker instead of holding it until it finishes. Setting this to +`false` puts the pool in legacy mode. + +QuestDB Enterprise [resource groups](/docs/concepts/resource-groups/) require +Fiber mode here whenever this pool has workers and HTTP or PostgreSQL is +enabled. A query pool set to zero workers turns parallel SQL off and needs no +check of its own. + ## shared.worker.haltOnError - **Default**: `false` diff --git a/documentation/high-availability/overview.md b/documentation/high-availability/overview.md index e49745503..93ba08a5d 100644 --- a/documentation/high-availability/overview.md +++ b/documentation/high-availability/overview.md @@ -134,6 +134,17 @@ The remote stages behave differently from the local ones. [Cold storage](/docs/c See [Operating cold storage](/docs/operations/cold-storage/) for the manager handoff procedure and its preconditions. +## Resource groups in a replicated cluster + +[Resource group](/docs/concepts/resource-groups/) policies and their principal +mappings live in a WAL-backed system catalog, so they replicate to every +instance through the same pipeline as user data. Enforcement then runs locally +on each instance, against the catalog snapshot that instance holds. + +What that means while a replica is behind, including why a lagging replica +cannot be promoted, is covered in +[Replication and catalog lag](/docs/concepts/resource-groups/#behaviour-under-failure-and-on-replicas). + ## Bring Your Own Cloud (BYOC) QuestDB Enterprise can be self-managed or operated by QuestDB's team under the diff --git a/documentation/operations/logging-metrics.md b/documentation/operations/logging-metrics.md index 240647f3c..b0db855ac 100644 --- a/documentation/operations/logging-metrics.md +++ b/documentation/operations/logging-metrics.md @@ -1,10 +1,12 @@ --- title: Logging and metrics -description: Configure and understand QuestDB logging and metrics, including log levels, configuration options, and Prometheus integration. +description: + Configure and understand QuestDB logging and metrics, including log levels, + configuration options, and Prometheus integration. --- - -This page outlines logging in QuestDB. It covers how to configure logs via `log.conf` and expose metrics via Prometheus. +This page outlines logging in QuestDB. It covers how to configure logs via +`log.conf` and expose metrics via Prometheus. - [Logging](/docs/operations/logging-metrics/#logging) - [Metrics](/docs/operations/logging-metrics/#metrics) @@ -206,19 +208,20 @@ For configuration options, see the :::warning On systems with -[8 Cores and less](/docs/getting-started/capacity-planning/#cpu-cores), contention -for threads might increase the latency of health check service responses. If you -use a load balancer, and it thinks the QuestDB service is dead with nothing -apparent in the QuestDB logs, you may need to configure a dedicated thread pool -for the health check service. To do so, increase `http.min.worker.count` to `1`. +[8 Cores and less](/docs/getting-started/capacity-planning/#cpu-cores), +contention for threads might increase the latency of health check service +responses. If you use a load balancer, and it thinks the QuestDB service is dead +with nothing apparent in the QuestDB logs, you may need to configure a dedicated +thread pool for the health check service. To do so, increase +`http.min.worker.count` to `1`. ::: #### Lifecycle endpoint `GET /lifecycle` on the same port returns the startup and shutdown state of -every server component as JSON, for probes and coordinators that need more -than the `200` of the health check: +every server component as JSON, for probes and coordinators that need more than +the `200` of the health check: ```shell curl http://127.0.0.1:9003/lifecycle @@ -338,23 +341,23 @@ When [cold storage](/docs/concepts/cold-storage/) is enabled, the endpoint exposes fifteen additional metrics under the `questdb_cold_chunk_` prefix, covering the chunk cache and the range reads that serve remote partitions: -| Metric | Type | Description | -| ------ | ---- | ----------- | -| `questdb_cold_chunk_acquire_full_hit_total` | counter | Reads that found every chunk already resident | -| `questdb_cold_chunk_acquire_partial_hit_total` | counter | Reads that found some chunks and fetched the rest | -| `questdb_cold_chunk_acquire_full_miss_total` | counter | Reads where every chunk had to be fetched | -| `questdb_cold_chunk_acquire_hit_chunks_total` | counter | Chunk lookups served from the cache | -| `questdb_cold_chunk_acquire_miss_chunks_total` | counter | Chunk lookups that had to be fetched | -| `questdb_cold_chunk_download_started_total` | counter | Range requests dispatched, one per coalesced group | -| `questdb_cold_chunk_download_finished_total` | counter | Range requests that returned data | -| `questdb_cold_chunk_download_failed_total` | counter | Range requests that failed after retries | -| `questdb_cold_chunk_download_coalesced_total` | counter | Readers that attached to an in-flight download instead of starting a new one | -| `questdb_cold_chunk_release_evictions_total` | counter | Chunks evicted when their last lease was released | -| `questdb_cold_chunk_in_flight_downloads` | gauge | Range requests dispatched but not yet complete | -| `questdb_cold_chunk_pending_batches` | gauge | Batches the read coordinator is tracking | -| `questdb_cold_chunk_busy_leases` | gauge | Currently allocated leases | -| `questdb_cold_chunk_ready_chunks` | gauge | Chunks resident in the ready cache | -| `questdb_cold_chunk_pinned_bytes` | gauge | Compressed bytes resident in the ready cache | +| Metric | Type | Description | +| ---------------------------------------------- | ------- | ---------------------------------------------------------------------------- | +| `questdb_cold_chunk_acquire_full_hit_total` | counter | Reads that found every chunk already resident | +| `questdb_cold_chunk_acquire_partial_hit_total` | counter | Reads that found some chunks and fetched the rest | +| `questdb_cold_chunk_acquire_full_miss_total` | counter | Reads where every chunk had to be fetched | +| `questdb_cold_chunk_acquire_hit_chunks_total` | counter | Chunk lookups served from the cache | +| `questdb_cold_chunk_acquire_miss_chunks_total` | counter | Chunk lookups that had to be fetched | +| `questdb_cold_chunk_download_started_total` | counter | Range requests dispatched, one per coalesced group | +| `questdb_cold_chunk_download_finished_total` | counter | Range requests that returned data | +| `questdb_cold_chunk_download_failed_total` | counter | Range requests that failed after retries | +| `questdb_cold_chunk_download_coalesced_total` | counter | Readers that attached to an in-flight download instead of starting a new one | +| `questdb_cold_chunk_release_evictions_total` | counter | Chunks evicted when their last lease was released | +| `questdb_cold_chunk_in_flight_downloads` | gauge | Range requests dispatched but not yet complete | +| `questdb_cold_chunk_pending_batches` | gauge | Batches the read coordinator is tracking | +| `questdb_cold_chunk_busy_leases` | gauge | Currently allocated leases | +| `questdb_cold_chunk_ready_chunks` | gauge | Chunks resident in the ready cache | +| `questdb_cold_chunk_pinned_bytes` | gauge | Compressed bytes resident in the ready cache | Watch rates and ratios rather than raw totals. Sustained `download_failed_total`, `pending_batches` sitting at its configured cap, or @@ -369,10 +372,56 @@ _Enterprise only._ Two gauges describe the state of an in-place [role switch](/docs/high-availability/failover/): -| Metric | Type | Description | -| ------ | ---- | ----------- | +| Metric | Type | Description | +| ---------------------------------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `questdb_replication_pending_upload_txn` | gauge | Committed transactions not yet uploaded to the object store, summed over the replicated tables. Poll it before demoting a primary: a demote that cannot bring it to zero within its timeout is not completed | -| `questdb_backup_active_at_last_demote` | gauge | `1` if a backup was still running when the node was last demoted, `0` otherwise. Cleared by the next promotion | +| `questdb_backup_active_at_last_demote` | gauge | `1` if a backup was still running when the node was last demoted, `0` otherwise. Cleared by the next promotion | + +### Resource group metrics + +_Enterprise only._ + +When [resource groups](/docs/concepts/resource-groups/) are enabled, the +endpoint exposes one series per group, labelled with `resource_group`: + +| Metric | Type | Description | +| --------------------------------------------------- | ------- | ------------------------------------------------------------ | +| `questdb_resource_group_active_queries` | gauge | Queries holding an admission slot | +| `questdb_resource_group_queued_queries` | gauge | Queries waiting for a slot | +| `questdb_resource_group_oldest_queue_wait_millis` | gauge | How long the longest waiting query has waited | +| `questdb_resource_group_memory_bytes` | gauge | Tracked query memory in use | +| `questdb_resource_group_memory_limit_bytes` | gauge | Effective group memory ceiling, `0` when the group sets none | +| `questdb_resource_group_cpu_nanos_total` | counter | CPU measured under managed scheduling | +| `questdb_resource_group_cpu_wait_nanos_total` | counter | Time the group spent waiting for CPU | +| `questdb_resource_group_admission_rejections_total` | counter | Queries rejected because the queue was full | +| `questdb_resource_group_admission_timeouts_total` | counter | Queries that timed out while queued | + +`cpu_nanos_total` counts only CPU that managed scheduling measured, so treat it +as a floor rather than a full account. While `DEFAULT` is the only group, CPU +scheduling is disengaged and nothing is sampled, so the series reads `0` however +busy the instance is. It also undercounts once scheduling is engaged: a query +already running when a second group was created stays outside scheduling until +it next suspends, and a degraded scheduler stops sampling altogether. + +A flat counter therefore does not imply an idle group. Check +`questdb_resource_groups_cpu_managed_dispatch` and +`questdb_resource_groups_cpu_scheduler_degraded` before drawing that conclusion, +along with query activity. Memory gauges show published accounting and can lag +worker-local deltas. + +Instance-wide series describe the feature itself: + +| Metric | Type | Description | +| ------------------------------------------------------------- | ------- | --------------------------------------------------------------------- | +| `questdb_resource_groups_enabled` | gauge | `1` when the feature is on | +| `questdb_resource_groups_catalog_current` | gauge | `1` when the group catalog is current; `0` while a replica catches up | +| `questdb_resource_groups_catalog_lag_unmanaged_queries_total` | counter | Queries that ran unmanaged because the catalog was not current yet | +| `questdb_resource_groups_cpu_managed_dispatch` | gauge | `1` while managed CPU scheduling is engaged | +| `questdb_resource_groups_cpu_scheduler_degraded` | gauge | `1` after an internal fault has disengaged CPU scheduling | + +A non-zero `questdb_resource_groups_cpu_scheduler_degraded` means CPU shares are +no longer enforced until the instance restarts. Admission and memory limits stay +enforced. ### Prometheus Alertmanager diff --git a/documentation/operations/resource-groups.md b/documentation/operations/resource-groups.md new file mode 100644 index 000000000..46410f604 --- /dev/null +++ b/documentation/operations/resource-groups.md @@ -0,0 +1,399 @@ +--- +title: Resource groups +sidebar_label: Resource groups +description: + Create resource groups, map users and ACL groups to them, and tune admission, + CPU and memory limits so one workload cannot starve another. +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +This page covers running the feature: what an instance needs, how to choose +limits, what to watch, and what to do when something looks wrong. For what the +limits guarantee, read [the concept page](/docs/concepts/resource-groups/). For +statement syntax, see +[`CREATE RESOURCE GROUP`](/docs/query/sql/acl/create-resource-group/) and its +siblings. + +## Requirements + +- QuestDB Enterprise. +- Access control enabled (`acl.enabled=true`). Groups can be created without it, + but mapping statements require it, since mappings attach to ACL principals. +- The worker pools that execute SQL must run in Fiber mode, which is the + default. +- The [`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission for + group management, mappings and instance-wide inspection through + `resource_groups()`. Ordinary users need no permission to call + `current_resource_group()` and check their own query's group. + +Mapping a principal to a resource group grants it nothing by itself. The user +still needs `HTTP` or `PGWIRE` to connect and `SELECT` on the tables it queries, +as described in [RBAC](/docs/security/rbac/). + +### Fiber mode + +Every Fiber setting defaults to `true`, so a stock instance already satisfies +this requirement and there is nothing to check. It matters only on an instance +whose worker pools were tuned by hand. + +If a SQL pool is in legacy mode, resource groups left at their default turn +themselves off at startup and log an error naming the pool and the setting to +change. Setting `resource.groups.enabled=true` explicitly on such an instance +fails startup with that same error. + +Each protocol is governed by the setting for the pool that actually serves it: + +- HTTP and PostgreSQL run on their own pool when `http.worker.count` or + `pg.worker.count` is above zero, governed by + [`http.worker.fiber.enabled`](/docs/configuration/http-server/#httpworkerfiberenabled) + and + [`pg.worker.fiber.enabled`](/docs/configuration/postgres-wire-protocol/#pgworkerfiberenabled). +- With those counts at their default of zero, both run on the shared network + pool, governed by + [`shared.network.worker.fiber.enabled`](/docs/configuration/shared-workers/#sharednetworkworkerfiberenabled). +- Parallel query work follows + [`shared.query.worker.fiber.enabled`](/docs/configuration/shared-workers/#sharedqueryworkerfiberenabled) + whenever the shared query pool has workers. A pool set to zero workers turns + parallel SQL off and needs no check of its own. + +The two instance-wide settings, `resource.groups.enabled` and +`resource.groups.process.memory.limit.bytes`, are described in the +[resource groups configuration reference](/docs/configuration/resource-groups/). +Neither is reloadable, so turning the feature off is a restart. Definitions and +mappings stay in the catalog either way, so nothing is lost while it is off. + +## Changing a live instance + +Three things are worth knowing before you change policy on a running system: + +- **Changes apply online.** An `ALTER` does not cancel anything running at that + moment. Slots already held are kept, an already queued request keeps its + deadline, and only subsequent allocations check a new memory budget. See + [effect on queries already running](/docs/query/sql/acl/alter-resource-group/#effect-on-queries-already-running). +- **A drop is refused while principals are still mapped.** Unmap them first. Once + unmapped, the group can be dropped while its queries are still running, and + they carry on under the settings it had. See + [what happens to queries still using it](/docs/query/sql/acl/drop-resource-group/#what-happens-to-queries-still-using-it). +- **`DEFAULT` is alterable but cannot be dropped or renamed.** Giving it limits + is how you bound everything that is not explicitly mapped. + +Mapping changes affect queries that start after the change, never one already +running. For the resolution order, including `ASSUME SERVICE ACCOUNT`, see +[who a resource group applies to](/docs/concepts/resource-groups/#who-a-resource-group-applies-to). + +On a replicated cluster a change takes effect on each instance as the catalog +reaches it, so a replica that is behind keeps applying the previous policy. See +[Replication and catalog lag](/docs/concepts/resource-groups/#behaviour-under-failure-and-on-replicas). + +## Choosing limits + +The accepted values and their defaults are in the +[`CREATE RESOURCE GROUP` parameter table](/docs/query/sql/acl/create-resource-group/#parameters). +What matters when picking them: + +- **An unset parameter is not always "unlimited".** `cpu_weight` falls back to + 100 and `queue_timeout` to 30 seconds, while the two admission counts really + are unlimited when unset. +- **Weights are only meaningful relative to other groups.** A group at + `cpu_weight = 50` gets a third of query CPU while `DEFAULT` also has work, and + all of it when `DEFAULT` is idle. Setting a weight on a lone group does + nothing. +- **No group memory ceiling does not mean unlimited memory.** A principal's own + limit, `cairo.query.memory.limit.bytes` and the process budget all still + apply, and a group ceiling only ever lowers the result. See + [memory limits](/docs/concepts/resource-groups/#memory-limits-use-batched-accounting). + +Start permissive, watch the counters in [inspecting](#inspecting), then tighten. +Policy changes apply online, so there is no need to get this right first time. + +## Common scenarios + +### The instance stops answering while CPU looks idle + +Every HTTP or PGWire worker is occupied by a large query, new requests are not +picked up, and instance CPU is low because those queries run on one core each. +Clients time out and retry, which produces more of the same queries. + +**1. Separate the workloads into groups.** While `DEFAULT` is the only group, +queries hold their workers exactly as they do with the feature disabled. CPU +scheduling engages as soon as a second group exists, and a query then yields its +worker at the checkpoints that already make it cancellable, so a long +single-threaded scan releases the worker while it is still running, the instance +keeps accepting connections, and CPU is split by weight: + +```questdb-sql +CREATE RESOURCE GROUP dashboards WITH (cpu_weight = 400); +CREATE RESOURCE GROUP adhoc WITH (cpu_weight = 100); + +ALTER USER app SET RESOURCE GROUP dashboards; +ALTER USER analyst SET RESOURCE GROUP adhoc; +``` + +`questdb_resource_groups_cpu_managed_dispatch` reports `1` while scheduling is +engaged, which is whenever a group besides `DEFAULT` exists. A query that +started before the second group was created keeps its worker until it next +suspends or finishes, and a query that never reaches a checkpoint holds its +worker either way, so this does not remove every cause of an unresponsive +instance. + +**2. Bound concurrent requests with admission.** + +```questdb-sql +ALTER RESOURCE GROUP adhoc SET ( + max_active_queries = 4, + max_queued_queries = 8, + queue_timeout = '5s' +); +``` + +The fifth concurrent query waits instead of running, and it does not hold a +worker while it waits. The thirteenth fails immediately with +`Resource Group admission queue is full`, so a client that keeps resending gets +a clear answer in seconds instead of adding to the pile. + +Admission directly bounds the number of concurrent queries; weights do not, so +the two are complementary. Clients should use bounded retries with backoff after +admission failures. + +Afterwards the symptom is also diagnosable rather than mysterious. Low instance +CPU together with a high `queued_queries` and a rising +`oldest_queue_wait_millis` on one group says the work is being held at the +admission gate, not that the machine is busy. `query_activity()` shows which +group each running query was admitted to. + +What resource groups do not do here: they do not make the slow plan faster, and +they do not bound how long one query may run. Wall-clock limits still come from +the instance-wide +[`query.timeout`](/docs/configuration/cairo-engine/#querytimeout), and that +clock includes time spent waiting in the admission queue: a query can time out +before it starts, and the error then says +`while queued for Resource Group admission`. + +### Dashboards must stay responsive while analysts run heavy queries + +Use weights. Shares are per group, not per query, so a group running fifty +queries does not outvote a group running one: + +```questdb-sql +CREATE RESOURCE GROUP dashboards WITH (cpu_weight = 400); +CREATE RESOURCE GROUP analysts WITH (cpu_weight = 100); +``` + +When these are the only competing groups and both can use their shares, weights +target a 4:1 split of managed query CPU. Actual use also depends on runnable +work and available parallelism. When analysts are idle, dashboards can use the +available query CPU. Weights are integers from 1 to 10000 and every group starts +at 100, so a group left alone keeps an equal share against any group you do not +change. + +### A background job must yield to everything else + +Give it a small weight and a small concurrency limit: + +```questdb-sql +CREATE RESOURCE GROUP exports WITH (cpu_weight = 10, max_active_queries = 1); +``` + +Under contention the job receives a tenth of the CPU that a group at the default +weight of 100 receives, and it runs one query at a time. When nothing else has +work it uses the CPU that would otherwise be idle; resource groups do not hold +CPU back from a group that is alone. + +### One workload must not exhaust query memory + +Bound the group rather than each query, so the limit holds however many queries +the workload starts: + +```questdb-sql +CREATE RESOURCE GROUP reporting WITH (memory_limit = '8G'); +``` + +A query that would push the group over its budget fails with +`query memory limit exceeded` reporting `scope=group`, and releases what it +held. + +### An ingestion or automation account runs queries too + +A service account cannot belong to an ACL group, so it has no mapping to +inherit. Without a direct mapping it runs in `DEFAULT`, so map it explicitly: + +```questdb-sql +CREATE RESOURCE GROUP automation WITH (cpu_weight = 50, max_active_queries = 2); + +ALTER SERVICE ACCOUNT ingest_bot SET RESOURCE GROUP automation; +``` + +This governs the queries the account runs. It does not throttle ingestion +itself, which resource groups do not manage. + +### Many teams share one instance + +Map ACL groups rather than individual users, and use `MAPPING PRIORITY` to +decide what happens to someone who belongs to more than one: + +```questdb-sql +ALTER GROUP analysts SET RESOURCE GROUP adhoc MAPPING PRIORITY 10; +ALTER GROUP oncall SET RESOURCE GROUP dashboards MAPPING PRIORITY 20; +``` + +Someone in both groups resolves to `dashboards`, because the higher priority +wins. If two mappings tie on priority, the one whose resource group was created +first wins, so keep priorities distinct. A direct mapping on the user beats +every group mapping regardless of priority, which is the way to make one person +an exception without touching the groups: + +```questdb-sql +ALTER USER lead_analyst SET RESOURCE GROUP dashboards; +``` + +`MAPPING PRIORITY` is rejected on user and service account mappings, because +those are one-to-one and have nothing to break a tie between. Confirm any of +this from the client's own session with `SELECT current_resource_group();`. + +## Inspecting + +[`resource_groups()`](/docs/query/functions/meta/#resource_groups) returns one +row per group, combining the configured policy with live counters. The columns +that matter day to day: + +- `active_queries` and `queued_queries` for live admission state, and + `oldest_queue_wait_millis` for how long the longest waiter has waited. A high + queue with a rising wait means work is held at the gate, not that the machine + is busy. +- `admission_rejections` and `admission_timeouts` for work already turned away. +- `memory_used_bytes` against `memory_limit_bytes` for headroom. + +Mappings are attributes of the principals themselves. `SHOW USERS` and +`SHOW SERVICE ACCOUNTS` carry a `resource_group` column, and `SHOW GROUPS` +carries `resource_group` and `resource_group_priority`; all are `NULL` for an +unmapped principal. Each statement can be used as a subquery, so +`SELECT name FROM (SHOW GROUPS) WHERE resource_group = 'reporting'` lists the +ACL groups mapped to one resource group. + +`current_resource_group()` returns the calling query's group, which is the +quickest way to confirm a mapping from the client's own connection: + +```questdb-sql +SELECT current_resource_group(); +``` + +An unmapped principal gets `DEFAULT`, not `NULL`. `NULL` means the query was +never admitted to a group at all, which happens when the feature is disabled and +on a replica whose group catalog is not ready. Note that `DEFAULT` is returned +even while CPU scheduling is disengaged, because assignment and CPU slicing are +separate things. See the +[function reference](/docs/query/functions/meta/#current_resource_group) for +permissions and return values. + +`query_activity()` carries a `resource_group` column, so you can see which group +each running query was admitted to. It follows the same rule, `DEFAULT` for an +unmapped principal and `NULL` only when no group was assigned: + +```questdb-sql +SELECT resource_group, username, query_start, query +FROM query_activity() +WHERE resource_group IS NOT NULL +ORDER BY query_start; +``` + +## Monitoring + +Metrics require `metrics.enabled=true`, which is off by default. The Prometheus +endpoint then exposes one series per group, labelled with `resource_group`, plus +five instance-wide series describing the feature itself. Every name, type and +meaning is in the +[metrics reference](/docs/operations/logging-metrics/#resource-group-metrics). + +Two signals are worth alerting on: a non-zero +`questdb_resource_groups_cpu_scheduler_degraded`, which means CPU shares are no +longer enforced until the next restart, and a steadily growing +`questdb_resource_group_admission_timeouts_total`, which means a group's queue +settings are rejecting work the application expects to succeed. + +## Errors clients see + +| Message | Cause | Usual fix | +| ----------------------------------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | +| `Resource Group admission queue is full` | The group is at `max_active_queries` and its queue is at `max_queued_queries` | Raise the limits, or let the client retry | +| `Resource Group admission queue timeout` | The query waited longer than `queue_timeout` | Raise `queue_timeout` or `max_active_queries`, or reduce concurrency | +| Either admission error while fetching a later page | A suspended cursor re-enters admission when the client asks for more rows | Adjust admission limits or retry the query with backoff; the failed cursor cannot continue | +| `query memory limit exceeded` | A single-query, group or process memory limit rejected an allocation | `scope` in the message names the level: `query`, `group` or `process`. Reduce memory use or raise that limit | +| `Resource Group is assigned to an ACL entity` | `DROP RESOURCE GROUP` while principals are still mapped | `UNSET RESOURCE GROUP` on those principals first; the message names one of them in `[entity=...]` | +| `built-in Resource Group cannot be dropped` / `cannot be renamed` | `DROP` or `RENAME` on `DEFAULT` | Alter it instead | + +## Troubleshooting + +**A group's CPU share is not what I configured.** Weights only apply while +groups compete. Check `active_queries` on both groups at the same moment: if one +is idle, the other is expected to use everything. A query that started before +the second group was created stays outside CPU scheduling until it next suspends +or finishes; every other query starts sharing at its next checkpoint. Also +confirm the work you are watching is managed at all, since ingestion, WAL apply +and view refresh are outside the feature. `query_activity()` shows the group +each running query belongs to, which is the quickest way to tell whether the +load you are watching is attributed where you expect. + +**Queries on a fresh replica are not limited.** Until the catalog has +replicated, a replica runs queries unmanaged and counts them in +`questdb_resource_groups_catalog_lag_unmanaged_queries_total`. The counter stops +growing once `questdb_resource_groups_catalog_current` reaches 1. + +**Promotion fails naming the resource group catalog.** With the feature enabled, +`SWITCH ROLE TO PRIMARY` does not admit writes over a catalog that is unreadable +or that the replica has not received yet. The node lands in the `UNKNOWN` role +and still serves reads; the server log names +`RESOURCE_GROUP_CATALOG_UNAVAILABLE` with the reason. When the reason is +`Resource Group catalog table is not locally available`, replication has not +delivered the catalog table yet: wait for it and run `SWITCH ROLE TO PRIMARY` +again. Any other reason means the table cannot be read, and retrying does not +help: promote another replica, or restart this node as primary with +`resource.groups.enabled=false`, which turns the check into a logged error. See +[Refusals and the torn state](/docs/high-availability/failover/#refusals-and-the-torn-state). + +**Startup fails naming the resource group catalog.** The catalog table cannot be +created or read while the feature is enabled; the startup error names the +Resource Group catalog and the reason. Starting with +`resource.groups.enabled=false` logs the condition instead of failing. + +**Startup fails naming a worker pool.** A pool that executes SQL is in legacy +mode while `resource.groups.enabled=true` was set explicitly. Either restore the +default Fiber mode for that pool or stop setting the property, which lets the +instance start with resource groups off. + +**The feature is off although the default is on.** Check the log at startup for +an error naming a worker pool, and check `SHOW PARAMETERS` for the value that +took effect. A legacy SQL pool turns the feature off when the property is left +unset. `SHOW PARAMETERS` reporting `resource.groups.enabled` as `true`, together +with `questdb_resource_groups_enabled` at `1`, confirms the feature is actually +running. + +## Limitations + +- Only query statements are managed. See + [which statements are managed](/docs/concepts/resource-groups/#which-statements-are-managed). +- Memory accounting covers tracked native query memory, not JVM heap, resident + set size or memory-mapped table pages. See + [memory limits](/docs/concepts/resource-groups/#memory-limits-use-batched-accounting). +- CPU control is cooperative, so shares hold over a short window rather than + instantaneously, and a query that cannot reach a cooperative checkpoint holds + its worker until it does. See + [cooperative CPU scheduling](/docs/concepts/resource-groups/#how-cooperative-cpu-scheduling-works). +- Principal mapping changes affect new queries. Group budgets change online; + dropping a group retains its runtime state for existing queries until they + finish. See [changing a live instance](#changing-a-live-instance). + +## See also + +- [Resource groups concept](/docs/concepts/resource-groups/) +- [Resource groups configuration](/docs/configuration/resource-groups/) +- [CREATE RESOURCE GROUP](/docs/query/sql/acl/create-resource-group/) +- [ALTER RESOURCE GROUP](/docs/query/sql/acl/alter-resource-group/) +- [DROP RESOURCE GROUP](/docs/query/sql/acl/drop-resource-group/) +- [Role-based access control](/docs/security/rbac/) +- [Logging and metrics](/docs/operations/logging-metrics/) diff --git a/documentation/query/functions/meta.md b/documentation/query/functions/meta.md index bd0f2e4c2..0ac2f6f59 100644 --- a/documentation/query/functions/meta.md +++ b/documentation/query/functions/meta.md @@ -36,7 +36,7 @@ SELECT build(); | ---------------------------------------------------------------------------------------------- | | Build Information: QuestDB 9.0.0, JDK 25, Commit Hash 460b817b0a3705c5633619a8ef9efb5163f1569c | -## current_data_id() +## current_data_id Returns the data ID: a UUID that identifies this database, generated on first start and stored in the database root. Replication and restore use it to tell @@ -61,34 +61,141 @@ SELECT current_data_id(); | ------------------------------------ | | 75f5a084-7a53-5d7f-941a-c2b4c4c6f127 | -## current database, schema, or user +## current_database -`current_database()`, `current_schema()`, `current_user()`, and -`session_user()` are standard SQL functions that return information about the -current database, schema, and user. +Returns the name of the database the session is connected to. + +**Arguments:** + +- `current_database()` does not require arguments. + +**Return value:** + +Returns a `string`. + +**Examples:** ```questdb-sql --- Get the current database SELECT current_database(); +``` --- Get the current schema -SELECT current_schema(); +| current_database | +| ---------------- | +| qdb | --- Get the current user -SELECT current_user(); +## current_resource_group --- Get the authenticated user of the current session -SELECT session_user(); +:::note + +Resource groups and the `current_resource_group()` function are available in +**QuestDB Enterprise** only. + +::: + +`current_resource_group()` returns the name of the +[resource group](/docs/concepts/resource-groups/) the calling query was admitted +to. Run it from a client session to confirm which group that session's queries +land in, which is the quickest way to check that a principal mapping took +effect. Any authenticated session can call it, with no permission of its own, +unlike [`resource_groups()`](#resource_groups) which requires +[`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions). An ordinary user can +therefore check their own group but not read anyone else's policy. + +**Arguments:** + +- `current_resource_group()` does not require arguments. + +**Return value:** + +Returns a string naming the group. A principal with no mapping of its own +resolves to `DEFAULT`, so on an instance where nobody has been mapped yet every +session sees `DEFAULT`: + +```questdb-sql title="Check which group the calling session's queries run in" +SELECT current_resource_group(); ``` -Each of these functions returns a single value, so you can use them in a SELECT -statement without any arguments. +| current_resource_group | +| ---------------------- | +| DEFAULT | + +It returns `null` only when the query was never admitted to a group at all: when +`resource.groups.enabled` is `false`, or on a replica that has not yet received +the group catalog. A group name and `null` therefore mean +different things: `DEFAULT` says the feature placed the query in the built-in +group, while `null` says the feature was not in force for that query. -`current_user()` and `session_user()` both return the authenticated principal -and are interchangeable in QuestDB. Both report the user that authenticated on -the current connection, whichever protocol it arrived on. +The group is resolved when the query starts and stays fixed for its lifetime, +including across the pages of a suspended cursor. Changing a mapping affects the +next query, never one already running. + +**Confirming a mapping end to end:** + +```questdb-sql title="Map a user, then confirm from that user's own session" +ALTER USER analyst SET RESOURCE GROUP reporting; +``` + +Reconnect as `analyst` and run: + +```questdb-sql title="Confirm the mapping took effect" +SELECT current_resource_group(); +``` + +| current_resource_group | +| ---------------------- | +| reporting | + +A principal with `SQL ENGINE ADMIN` can check the same thing without +reconnecting, because [`query_activity()`](#query_activity) carries a +`resource_group` column for every running query. + +## current_schema + +Returns the name of the current schema. + +**Arguments:** + +- `current_schema()` does not require arguments. + +**Return value:** + +Returns a `string`. + +**Examples:** + +```questdb-sql +SELECT current_schema(); +``` + +| current_schema | +| -------------- | +| public | + +## current_user + +Returns the user that authenticated on the current connection, whichever +protocol it arrived on. [`session_user()`](#session_user) is a compatibility +alias for it, and the two are interchangeable in QuestDB. + +**Arguments:** + +- `current_user()` does not require arguments. + +**Return value:** -## flush_query_cache() +Returns a `string`. + +**Examples:** + +```questdb-sql +SELECT current_user(); +``` + +| current_user | +| ------------ | +| admin | + +## flush_query_cache `flush_query_cache' invalidates cached query execution plans. @@ -128,7 +235,7 @@ functions(); | and | and(TT) | and(boolean, boolean) | FALSE | STANDARD | | not | not(T) | not(boolean) | FALSE | STANDARD | -## hydrate_table_metadata('table1', 'table2' ...) +## hydrate_table_metadata `hydrate_table_metadata' re-reads table metadata from disk to update the static metadata cache. @@ -316,7 +423,6 @@ materialized_views(); | trades_OHLC_15m | immediate | trades | 2025-05-30T16:40:37.562421Z | 2025-05-30T16:40:37.568800Z | SELECT timestamp, symbol, first(price) AS open, max(price) as high, min(price) as low, last(price) AS close, sum(amount) AS volume FROM trades SAMPLE BY 15m | trades_OHLC_15m~27 | null | valid | 55141609 | 55141609 | 0 | null | null | 0 | null | | trades_latest_1d | immediate | trades | 2025-05-30T16:40:37.554274Z | 2025-05-30T16:40:37.562049Z | SELECT timestamp, symbol, side, last(price) AS price, last(amount) AS amount, last(timestamp) as latest FROM trades SAMPLE BY 1d | trades_latest_1d~28 | null | valid | 55141609 | 55141609 | 0 | null | null | 0 | null | - ## memory_metrics **Arguments:** @@ -390,10 +496,10 @@ SELECT node_role(); :::warning `node_role()` cannot be used in a materialized view or a live view. Avoid it in -`UPDATE` on a WAL table as well: the statement is re-executed on every node of -a replicated cluster and each node evaluates its own role, so the primary and -its replicas would write different values. Tagging rows on `INSERT` is safe, -because inserted rows replicate as data. +`UPDATE` on a WAL table as well: the statement is re-executed on every node of a +replicated cluster and each node evaluates its own role, so the primary and its +replicas would write different values. Tagging rows on `INSERT` is safe, because +inserted rows replicate as data. ::: @@ -417,21 +523,29 @@ Returns metadata on running SQL queries, with the following columns: - query_start - timestamp of when query started - state_change - timestamp of latest query state change, such as a cancellation - state - state of running query, can be `active` or `cancelled` -- is_wal - `true` when the SQL is being applied by the WAL apply job, such as - an `UPDATE` on a WAL table. Such queries cannot be cancelled +- is_wal - `true` when the SQL is being applied by the WAL apply job, such as an + `UPDATE` on a WAL table. Such queries cannot be cancelled - query - text of sql query - memory_used - native memory currently allocated by the query, in bytes, as tracked by the [per-query memory limit](/docs/configuration/cairo-engine/#memory-limits) - memory_limit - effective native memory limit for the query, in bytes, or - `null` when the query runs unlimited. On QuestDB Enterprise this is the + `null` when the query runs unlimited. On QuestDB Enterprise it starts from the principal's [memory limit](/docs/security/rbac/#memory-limits) when one is - set, otherwise the workload limit. Unlike the `memory_limit` column of - `SHOW USERS`, it includes the workload limit + set, otherwise the workload limit, and is then capped by the + [resource group](/docs/concepts/resource-groups/) budget and the process + budget. Unlike the `memory_limit` column of `SHOW USERS`, it includes the + workload limit and the group budget, so a principal with no limit of its own + still reports a value here when its group sets one +- resource_group - the [resource group](/docs/concepts/resource-groups/) the + query was admitted to on QuestDB Enterprise. A query whose principal has no + mapping reports `DEFAULT`; `null` means the query was never admitted to a + group at all, which happens when resource groups are disabled and when a + replica has not yet received the group catalog `memory_used` is a live gauge with no peak value, and it is reported even when -`memory_limit` is `null`. Both memory columns are `null` for SQL that runs -under a background workload's tracker, such as the `SELECT` a materialized view +`memory_limit` is `null`. Both memory columns are `null` for SQL that runs under +a background workload's tracker, such as the `SELECT` a materialized view refresh runs or an `UPDATE` applied by the WAL apply job, because that SQL charges the workload's budget instead of acquiring its own. @@ -442,10 +556,26 @@ SELECT query_id, username, state, memory_used, memory_limit, query FROM query_activity(); ``` -| query_id | username | state | memory_used | memory_limit | query | -| -------- | -------- | ------ | ----------- | ------------ | ------------------------------------------------------------------------------------- | +| query_id | username | state | memory_used | memory_limit | query | +| -------- | -------- | ------ | ----------- | ------------ | ---------------------------------------------------------------------------------------- | | 62179 | john | active | 262144 | 536870912 | SELECT query_id, username, state, memory_used, memory_limit, query FROM query_activity() | -| 57777 | john | active | 8388608 | 536870912 | SELECT symbol, approx_percentile(price, 0.5, 2) FROM trades | +| 57777 | john | active | 8388608 | 536870912 | SELECT symbol, approx_percentile(price, 0.5, 2) FROM trades | + +To inspect query memory and resource group assignment in QuestDB Enterprise: + +```questdb-sql title="See which group each running query was admitted to" +SELECT query_id, username, resource_group, memory_used, memory_limit +FROM query_activity(); +``` + +| query_id | username | resource_group | memory_used | memory_limit | +| -------- | -------- | -------------- | ----------- | ------------ | +| 47 | analyst | reporting | 0 | 2147483648 | + +`analyst` has no memory limit of its own here, so the 2 GiB ceiling is the one +the `reporting` group sets. Joining this against +[`resource_groups()`](#resource_groups) shows how much of a group's budget its +running queries are actually holding. ## reader_pool @@ -472,7 +602,7 @@ SELECT * FROM reader_pool(); | ---------- | --------------- | --------------------------- | ----------- | | sensors | null | 2023-12-01T19:28:14.311703Z | 1 | -## reload_config() +## reload_config `reload_config' reloads server configuration file's contents (`server.conf`) without server restart. The list of reloadable settings can be found @@ -495,15 +625,160 @@ Edit `server.conf` and run `reload_config`: SELECT reload_config(); ``` -## sleep() +## resource_groups + +:::note + +Resource groups and the `resource_groups()` function are available in **QuestDB +Enterprise** only. + +::: + +`resource_groups()` returns one row per +[resource group](/docs/concepts/resource-groups/) that currently exists, +including the built-in `DEFAULT`, pairing each group's policy with its live +counters. It is the main way to see what a group is configured to do and what it +is doing right now. Groups stay visible when `resource.groups.enabled` is +`false`; only their counters sit at zero. + +Calling it requires the database-level +[`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission, the same +permission that gates creating, altering and mapping groups. A principal without +it gets `Access denied for [RESOURCE GROUP ADMIN]`. + +**Arguments:** + +- `resource_groups()` does not require arguments. + +**Return value:** + +Returns a table with these columns: + +| Column | Type | Description | +| -------------------------- | --------- | -------------------------------------------------------------------------------------------- | +| `name` | `VARCHAR` | Group name | +| `memory_limit_bytes` | `LONG` | Effective memory ceiling in bytes, after the process budget caps it; `0` means no group ceiling | +| `max_active_queries` | `INT` | Concurrent admission limit as configured; `null` when the group sets none | +| `max_queued_queries` | `INT` | Queue capacity as configured; `null` when the group sets none, `0` to disable queueing | +| `queue_timeout_millis` | `LONG` | Effective admission timeout in milliseconds, falling back to the instance default | +| `cpu_weight` | `INT` | Effective relative scheduling weight; `100` when the group sets none | +| `active_queries` | `LONG` | Queries currently holding admission slots | +| `queued_queries` | `LONG` | Queries waiting for admission | +| `oldest_queue_wait_millis` | `LONG` | Age of the oldest admission waiter in milliseconds; `0` when none | +| `memory_used_bytes` | `LONG` | Published tracked native query memory in bytes | +| `cpu_nanos_total` | `LONG` | CPU nanoseconds measured by managed scheduling | +| `cpu_wait_nanos_total` | `LONG` | Cumulative time queries spent waiting for CPU, in nanoseconds | +| `admission_rejections` | `LONG` | Cumulative queue-full rejections | +| `admission_timeouts` | `LONG` | Cumulative admission timeouts | + +The policy columns do not all report the same thing. `max_active_queries` and +`max_queued_queries` show what the group itself sets, and are `null` when it sets +nothing. `memory_limit_bytes`, `queue_timeout_millis` and `cpu_weight` show the +effective value after instance defaults and the process budget have been applied, +so they are never `null`. + +**Inspecting group policy:** + +Given a group created like this: + +```questdb-sql title="Create a group with an explicit policy" +CREATE RESOURCE GROUP reporting WITH ( + cpu_weight = 50, + max_active_queries = 4, + max_queued_queries = 32, + memory_limit = '2G' +); +``` + +```questdb-sql title="Compare each group's policy against its live admission state" +SELECT name, cpu_weight, max_active_queries, max_queued_queries, + memory_limit_bytes, active_queries, queued_queries +FROM resource_groups() +ORDER BY name; +``` + +| name | cpu_weight | max_active_queries | max_queued_queries | memory_limit_bytes | active_queries | queued_queries | +| --------- | ---------- | ------------------ | ------------------ | ------------------ | -------------- | -------------- | +| DEFAULT | 100 | null | null | 0 | 1 | 0 | +| reporting | 50 | 4 | 32 | 2147483648 | 0 | 0 | + +`DEFAULT` sets no admission limits of its own, so both columns are `null`, while +its `cpu_weight` of 100 is the effective default rather than something anyone +configured. Its `active_queries` counts the introspection query itself, which is +why the group you are querying from is never idle in its own output. + +**Finding groups under admission pressure:** + +```questdb-sql title="List groups where queries are waiting or being turned away" +SELECT name, active_queries, queued_queries, oldest_queue_wait_millis, + admission_rejections, admission_timeouts +FROM resource_groups() +WHERE queued_queries > 0 + OR admission_rejections > 0 + OR admission_timeouts > 0; +``` + +A high `queued_queries` with a rising `oldest_queue_wait_millis` means work is +being held at the admission gate rather than by a busy machine, which is the +signal that distinguishes a concurrency limit from a slow query. + +**Constraints and edge cases:** + +- Counters describe the current runtime. They reset when the instance restarts, + and recreating a group under the same name starts it from zero. +- `cpu_nanos_total` counts only CPU that managed scheduling actually measured, so + it is a floor rather than a full account of query CPU. While `DEFAULT` is the + only group, scheduling is disengaged and nothing is sampled, so the column sits + at `0` however busy the instance is. It also undercounts once scheduling is + engaged: a query already running when a second group was created stays outside + scheduling until it next suspends or finishes, and a scheduler that has + degraded after an internal fault stops sampling from then on, which + `questdb_resource_groups_cpu_scheduler_degraded` reports as `1`. +- `memory_used_bytes` can lag real usage, because workers accumulate allocation + deltas locally and publish them to the shared counter in batches. +- A dropped group disappears from this table immediately, while its running and + queued queries finish under the settings it had. + +The same values are published as +[Prometheus metrics](/docs/operations/logging-metrics/#resource-group-metrics), +where a group with no memory ceiling also reports `0`. -Pauses the query for the given number of seconds, then returns the timestamp -at which it resumed. Intended for testing and demonstration, for example to -hold a query open while inspecting -[`query_activity()`](#query_activity) from another session. +No group ceiling does not mean unlimited memory. A principal-specific limit, the +instance default `cairo.query.memory.limit.bytes` and the process budget +`resource.groups.process.memory.limit.bytes` all still apply. -`sleep()` does not hold a worker thread while it waits, so many concurrent -calls can be parked at once without exhausting the shared worker pool. +## session_user + +Compatibility alias for [`current_user()`](#current_user). The two are +interchangeable in QuestDB. + +**Arguments:** + +- `session_user()` does not require arguments. + +**Return value:** + +Returns a `string`. + +**Examples:** + +```questdb-sql +SELECT session_user(); +``` + +| session_user | +| ------------ | +| admin | + +## sleep + +Pauses the query for the given number of seconds, then returns the timestamp at +which it resumed. Intended for testing and demonstration, for example to hold a +query open while inspecting [`query_activity()`](#query_activity) from another +session. + +`sleep()` does not hold a worker thread while it waits, so many concurrent calls +can be parked at once without exhausting the shared worker pool. **Arguments:** @@ -527,8 +802,8 @@ SELECT * FROM sleep(1); :::note -Storage policies — and the `storage_policies` view — are available in -**QuestDB Enterprise** only. +Storage policies — and the `storage_policies` view — are available in **QuestDB +Enterprise** only. ::: @@ -557,9 +832,9 @@ SELECT * FROM storage_policies; - TTL values are rendered in two units: `h` for hours and `m` for **months**. Hour-, day-, and week-based durations are stored as hours (e.g. `3 DAYS` → `72h`, `1 WEEK` → `168h`). Month- and year-based durations are stored as - months (e.g. `1 MONTH` → `1m`, `1 YEAR` → `12m`). Despite the visual - collision with "minute", `m` in this view is **months**; QuestDB's duration - shorthand has no unit for minutes. + months (e.g. `1 MONTH` → `1m`, `1 YEAR` → `12m`). Despite the visual collision + with "minute", `m` in this view is **months**; QuestDB's duration shorthand + has no unit for minutes. - An unset stage renders as `0h`, not blank. **Example:** @@ -582,11 +857,15 @@ stage set and has been temporarily disabled. Every unset stage renders as `0h`. :::note -[Cold storage](/docs/concepts/cold-storage/) and the `table_cold_partitions()` function are available in **QuestDB Enterprise** only. +[Cold storage](/docs/concepts/cold-storage/) and the `table_cold_partitions()` +function are available in **QuestDB Enterprise** only. ::: -`table_cold_partitions('tableName')` returns one row per partition in the table's remote manifest, with the state of its object in the store. Use it to follow a partition through upload and sealing, and to find partitions that are not progressing. +`table_cold_partitions('tableName')` returns one row per partition in the +table's remote manifest, with the state of its object in the store. Use it to +follow a partition through upload and sealing, and to find partitions that are +not progressing. **Arguments:** @@ -634,9 +913,16 @@ WHERE state = 'pending'; **Notes:** -- The cold storage manager answers from its own in-memory view. A refresher answers from its mirrored copy, which it updates when the catalog generation changes, so the two can differ briefly. -- While an instance is transitioning between the manager and refresher roles, the function returns zero rows rather than blocking. Check the live role with [`SWITCH COLD STORAGE STATUS`](/docs/query/sql/switch-cold-storage-role/). -- The function reflects the remote manifest, not local partition state. Use [`SHOW PARTITIONS`](/docs/query/sql/show/#show-partitions) or [`table_partitions()`](#table_partitions) to see whether a partition is actually being served remotely. +- The cold storage manager answers from its own in-memory view. A refresher + answers from its mirrored copy, which it updates when the catalog generation + changes, so the two can differ briefly. +- While an instance is transitioning between the manager and refresher roles, + the function returns zero rows rather than blocking. Check the live role with + [`SWITCH COLD STORAGE STATUS`](/docs/query/sql/switch-cold-storage-role/). +- The function reflects the remote manifest, not local partition state. Use + [`SHOW PARTITIONS`](/docs/query/sql/show/#show-partitions) or + [`table_partitions()`](#table_partitions) to see whether a partition is + actually being served remotely. ## table_columns @@ -659,14 +945,14 @@ Returns a `table` with the following columns: - `symbolCached` - whether this `symbol` column is cached - `symbolCapacity` - how many distinct values this column of `symbol` type is expected to have -- `symbolTableSize` - current number of distinct values stored in this - `symbol` column's table +- `symbolTableSize` - current number of distinct values stored in this `symbol` + column's table - `designated` - if this is set as the designated timestamp column for this table - `upsertKey` - if this column is a part of UPSERT KEYS list for table [deduplication](/docs/concepts/deduplication) -- `indexType` - the [index type](/docs/concepts/deep-dive/indexes/) - (`POSTING`, `POSTING DELTA`, `POSTING EF`, `BITMAP`, or empty) +- `indexType` - the [index type](/docs/concepts/deep-dive/indexes/) (`POSTING`, + `POSTING DELTA`, `POSTING EF`, `BITMAP`, or empty) - `indexInclude` - comma-separated names of columns included in a [posting index's](/docs/concepts/deep-dive/posting-index/) covering sidecar @@ -744,16 +1030,16 @@ Returns a table with the following columns: partition will contain the `.detached` extension) - `attachable` - _BOOLEAN_, true if the partition is detached and can be attached (`name` of the partition will contain the `.attachable` extension) -- `hasParquetGenerated` - _BOOLEAN_, true if a Parquet copy of the partition - has been generated. Set by either +- `hasParquetGenerated` - _BOOLEAN_, true if a Parquet copy of the partition has + been generated. Set by either [manual Parquet conversion](/docs/concepts/parquet/#in-place-conversion) (`ALTER TABLE ... CONVERT PARTITION TO PARQUET`) or by a [storage policy](/docs/concepts/storage-policy/)'s `TO PARQUET` stage (Enterprise) - `isParquet` - _BOOLEAN_, true if the partition is stored in Parquet format: - the native files have been removed and reads are served from the Parquet - file. Set the same way as `hasParquetGenerated`: either manually or by a - storage policy's `TO PARQUET` stage + the native files have been removed and reads are served from the Parquet file. + Set the same way as `hasParquetGenerated`: either manually or by a storage + policy's `TO PARQUET` stage - `parquetFileSize` - _LONG_, size in bytes of the partition's `data.parquet` file when `hasParquetGenerated` or `isParquet` is true; `-1` otherwise - `seqTxn` - _LONG_, WAL transaction version the partition was last written at @@ -914,7 +1200,7 @@ Returns a `table` with the following columns: ::: -### Table metrics (table_* prefix) +### Table metrics (table\_\* prefix) | Column | Type | Description | | ----------------------------- | --------- | ----------------------------------------------------------------------------------------- | @@ -937,16 +1223,18 @@ Returns a `table` with the following columns: | `table_merge_rate_p99` | LONG | Throughput that 99% of jobs **exceeded** (slowest 1%) | | `table_merge_rate_max` | LONG | Maximum throughput in rows/second | -Write amplification measures O3 (out-of-order) merge overhead as `physicalRowsWritten / logicalRows`. -A ratio of `1.0` means no amplification. Higher values indicate O3 merge overhead. +Write amplification measures O3 (out-of-order) merge overhead as +`physicalRowsWritten / logicalRows`. A ratio of `1.0` means no amplification. +Higher values indicate O3 merge overhead. :::note -Merge rate P99 shows the *lowest* throughput (worst performance), not the highest. +Merge rate P99 shows the _lowest_ throughput (worst performance), not the +highest. ::: -### WAL metrics (wal_* prefix) +### WAL metrics (wal\_\* prefix) | Column | Type | Description | | --------------------------------- | --------- | ------------------------------------------------------------- | @@ -960,9 +1248,10 @@ Merge rate P99 shows the *lowest* throughput (worst performance), not the highes | `wal_tx_size_p99` | LONG | 99th percentile transaction size in rows | | `wal_tx_size_max` | LONG | Maximum transaction size in rows | -### Replica metrics (replica_* prefix) +### Replica metrics (replica\_\* prefix) -These columns are populated on **replicas only** via replication download tracking: +These columns are populated on **replicas only** via replication download +tracking: | Column | Type | Description | | ------------------------ | ------- | ------------------------------------------------------------------------ | @@ -979,17 +1268,31 @@ On primary instances, these columns will be `0` or `false`. These values are approximations, not precise real-time metrics: -- **Null when not tracked**: Values are `null` for tables not written to since server start, or evicted from the tracker -- **Writer stats updated on pool return**: `table_row_count`, `table_last_write_timestamp`, `table_txn` are captured when TableWriter returns to the pool, not on every commit. A writer held for a long time won't update these columns until released. +- **Null when not tracked**: Values are `null` for tables not written to since + server start, or evicted from the tracker +- **Writer stats updated on pool return**: `table_row_count`, + `table_last_write_timestamp`, `table_txn` are captured when TableWriter + returns to the pool, not on every commit. A writer held for a long time won't + update these columns until released. - **WAL stats updated in real-time**: - - On WAL commit: `wal_pending_row_count` (incremented), `wal_txn`, `wal_max_timestamp`, `wal_tx_size_*` histogram - - On WAL apply: `wal_pending_row_count` (decremented), `wal_dedup_row_count_since_start`, `table_min_timestamp`, `table_max_timestamp`, `table_write_amp_*`, `table_merge_rate_*` -- **LRU eviction**: Tracker maintains bounded memory (default 1000 tables). Least recently written tables are evicted when capacity is exceeded -- **Startup hydration**: Values are hydrated from table metadata (`TxReader`) on startup, but diverge as writes occur - -**Non-WAL tables**: `wal_txn`, `wal_max_timestamp`, `wal_pending_row_count`, `wal_dedup_row_count_since_start`, `table_min_timestamp`, `table_max_timestamp`, `table_memory_pressure_level`, and histogram columns are `null` or `0`. - -**WAL tables**: All columns populated when tracked. `wal_max_timestamp` reflects the max data timestamp from the WAL transaction, not wall-clock time. `table_min_timestamp` and `table_max_timestamp` reflect the actual data range in the table after WAL merge. + - On WAL commit: `wal_pending_row_count` (incremented), `wal_txn`, + `wal_max_timestamp`, `wal_tx_size_*` histogram + - On WAL apply: `wal_pending_row_count` (decremented), + `wal_dedup_row_count_since_start`, `table_min_timestamp`, + `table_max_timestamp`, `table_write_amp_*`, `table_merge_rate_*` +- **LRU eviction**: Tracker maintains bounded memory (default 1000 tables). + Least recently written tables are evicted when capacity is exceeded +- **Startup hydration**: Values are hydrated from table metadata (`TxReader`) on + startup, but diverge as writes occur + +**Non-WAL tables**: `wal_txn`, `wal_max_timestamp`, `wal_pending_row_count`, +`wal_dedup_row_count_since_start`, `table_min_timestamp`, `table_max_timestamp`, +`table_memory_pressure_level`, and histogram columns are `null` or `0`. + +**WAL tables**: All columns populated when tracked. `wal_max_timestamp` reflects +the max data timestamp from the WAL transaction, not wall-clock time. +`table_min_timestamp` and `table_max_timestamp` reflect the actual data range in +the table after WAL merge. ### Configuration @@ -1215,7 +1518,7 @@ WHERE view_status = 'invalid'; SELECT * FROM views() ORDER BY view_name; ``` -## wait_wal_table() +## wait_wal_table Blocks until the WAL writer for a table has applied its transactions up to a target sequencer transaction, then returns `true`. @@ -1231,9 +1534,8 @@ concurrent waiters is not bounded by the shared worker pool. **Arguments:** -- `tableName` (`string`): name of the table to wait for. Must be a constant, - not a column reference. On a non-WAL table the call returns `true` - immediately. +- `tableName` (`string`): name of the table to wait for. Must be a constant, not + a column reference. On a non-WAL table the call returns `true` immediately. - `seqTxn` (optional, `long`): the sequencer transaction to wait for. When omitted, the call captures the table's current `seqTxn` when it starts and waits for that, which is what you want after your own write. @@ -1243,8 +1545,8 @@ concurrent waiters is not bounded by the shared worker pool. Returns `boolean`. `true` once the writer has caught up. Throws if the table is dropped while the call is waiting, and if the table -becomes [suspended](/docs/query/sql/alter-table-resume-wal/), since a -suspended table would otherwise never catch up. +becomes [suspended](/docs/query/sql/alter-table-resume-wal/), since a suspended +table would otherwise never catch up. **Examples:** @@ -1268,12 +1570,12 @@ SELECT wait_wal_table('trades', 42); :::note -For monitoring and observability, use [`tables()`](#tables) instead. -`tables()` provides the same status information plus additional metrics -(pending rows, memory pressure, deduplication stats, throughput histograms), -and is fully in-memory. `wal_tables()` reads from disk and is less suitable -for frequent polling, but it is the only function that reports the `errorTag` -and `errorMessage` of a suspended table. +For monitoring and observability, use [`tables()`](#tables) instead. `tables()` +provides the same status information plus additional metrics (pending rows, +memory pressure, deduplication stats, throughput histograms), and is fully +in-memory. `wal_tables()` reads from disk and is less suitable for frequent +polling, but it is the only function that reports the `errorTag` and +`errorMessage` of a suspended table. ::: @@ -1291,15 +1593,17 @@ Returns a `table` including the following information: - `name` - table or materialized view name - `suspended` - suspended status flag -- `writerTxn` - the last committed transaction in TableWriter (equivalent to `table_txn` in `tables()`) +- `writerTxn` - the last committed transaction in TableWriter (equivalent to + `table_txn` in `tables()`) - `bufferedTxnSize` - the number of transactions that are kept invisible when writing to the table; these transactions will be eventually moved to the table data and become visible for readers (equivalent to `wal_txn - table_txn`) -- `sequencerTxn` - the last committed transaction in the sequencer (equivalent to `wal_txn` in `tables()`) +- `sequencerTxn` - the last committed transaction in the sequencer (equivalent + to `wal_txn` in `tables()`) - `errorTag` - short classification of the error that suspended the table, such as `OUT OF MEMORY` when a WAL apply batch breached its - [memory limit](/docs/configuration/cairo-engine/#memory-limits), or empty - when the table is not suspended + [memory limit](/docs/configuration/cairo-engine/#memory-limits), or empty when + the table is not suspended - `errorMessage` - full text of the error that suspended the table, or empty when the table is not suspended - `memoryPressure` - memory pressure level of the table writer: `0` for none, diff --git a/documentation/query/sql/acl/alter-group-drop-external-alias.md b/documentation/query/sql/acl/alter-group-drop-external-alias.md new file mode 100644 index 000000000..936c506a9 --- /dev/null +++ b/documentation/query/sql/acl/alter-group-drop-external-alias.md @@ -0,0 +1,48 @@ +--- +title: ALTER GROUP DROP EXTERNAL ALIAS reference +sidebar_label: DROP EXTERNAL ALIAS +description: + "ALTER GROUP DROP EXTERNAL ALIAS removes an external OIDC or LDAP group + mapping. Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER GROUP ... DROP EXTERNAL ALIAS` removes an external OIDC or LDAP group +mapping. + +--- + +## Syntax + +```questdb-sql +ALTER GROUP groupName DROP EXTERNAL ALIAS externalAlias; +``` + +## Description + +Removing an alias requires the `REMOVE EXTERNAL ALIAS` permission. Quote the +alias when it contains commas, spaces, or `=`, as LDAP distinguished names do. + +Members of the external group stop inheriting this group's permissions on their +next login. The QuestDB group itself, and any user explicitly added to it, are +unaffected. + +## Examples + +```questdb-sql +ALTER GROUP analysts DROP EXTERNAL ALIAS 'CN=Analysts,OU=Users,DC=example,DC=com'; +``` + +[`SHOW GROUPS`](/docs/query/sql/show/#show-groups) then reports an empty +`external_alias` column for the group. + +## See also + +- [ALTER GROUP WITH EXTERNAL ALIAS](/docs/query/sql/acl/alter-group-with-external-alias/) +- [DROP GROUP](/docs/query/sql/acl/drop-group/) +- [OpenID Connect (OIDC) integration](/docs/security/oidc/#mapping-user-permissions) diff --git a/documentation/query/sql/acl/alter-group-set-memory-limit.md b/documentation/query/sql/acl/alter-group-set-memory-limit.md new file mode 100644 index 000000000..beeeb7f4c --- /dev/null +++ b/documentation/query/sql/acl/alter-group-set-memory-limit.md @@ -0,0 +1,64 @@ +--- +title: ALTER GROUP SET MEMORY LIMIT reference +sidebar_label: SET MEMORY LIMIT +description: + "ALTER GROUP SET MEMORY LIMIT caps the native memory each query run by a + member of the group may allocate. Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER GROUP ... SET MEMORY LIMIT` caps the native memory each query run by a +member of the group may allocate. + +--- + +## Syntax + +```questdb-sql +ALTER GROUP groupName SET MEMORY LIMIT { size | UNLIMITED }; +``` + +## Description + +- `ALTER GROUP groupName SET MEMORY LIMIT size` caps the native memory that each + query run by a member of the group may allocate. `size` is a byte count or a + size with a `K`, `M`, or `G` suffix, such as `512M` or `2G`. +- `ALTER GROUP groupName SET MEMORY LIMIT UNLIMITED` clears the group's limit. + Members without a limit of their own then fall back to the most restrictive + limit among their other groups, or to the workload limit + (`cairo.query.memory.limit.bytes`). `SET MEMORY LIMIT 0` does the same. + +A group limit applies to a member only when that member has no limit of its own. +When several of a user's groups set a limit, the most restrictive one applies. + +Setting a group limit requires the `SET MEMORY LIMIT` permission. See +[memory limits](/docs/security/rbac/#memory-limits) for how a group limit +interacts with the +[`cairo.query.memory.limit.bytes`](/docs/configuration/cairo-engine/#cairoquerymemorylimitbytes) +workload limit. + +This is a per-query ceiling for the group's members. To bound what a whole +workload may hold at once instead, use a +[resource group](/docs/query/sql/acl/alter-group-set-resource-group/). + +## Examples + +```questdb-sql +-- cap queries of the group's members at 2 GiB of native memory +ALTER GROUP analysts SET MEMORY LIMIT 2G; +-- remove the limit +ALTER GROUP analysts SET MEMORY LIMIT UNLIMITED; +``` + +The configured value can be verified with +[`SHOW GROUPS`](/docs/query/sql/show/#show-groups), which reports it in the +`memory_limit` column. + +## See also + +- [Memory limits](/docs/security/rbac/#memory-limits) diff --git a/documentation/query/sql/acl/alter-group-set-resource-group.md b/documentation/query/sql/acl/alter-group-set-resource-group.md new file mode 100644 index 000000000..fb5eaf70d --- /dev/null +++ b/documentation/query/sql/acl/alter-group-set-resource-group.md @@ -0,0 +1,84 @@ +--- +title: ALTER GROUP SET RESOURCE GROUP reference +sidebar_label: SET RESOURCE GROUP +description: + "ALTER GROUP SET RESOURCE GROUP places every member's queries under a resource + group, with MAPPING PRIORITY breaking ties. Applies to QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +`ALTER GROUP ... SET RESOURCE GROUP` places the queries of every user in the +group under a [resource group](/docs/concepts/resource-groups/), which governs +their admission, CPU share and memory budget. + +--- + +## Syntax + +```questdb-sql +ALTER GROUP groupName SET RESOURCE GROUP resourceGroupName + [MAPPING PRIORITY priority]; +``` + +## Description + +Mapping an ACL group is how you cover a team without naming each member. It +requires the [`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission, +and affects queries that start after the change rather than one already running. + +This is workload mapping: it decides how much of the instance the group's +queries may consume. It is unrelated to +[`WITH EXTERNAL ALIAS`](/docs/query/sql/acl/alter-group-with-external-alias/), +which decides which QuestDB group an external identity lands in. + +### MAPPING PRIORITY + +A user can belong to several ACL groups, so each ACL group mapping carries a +priority and the highest one wins. `MAPPING PRIORITY` is a non-negative integer +that defaults to 0. + +If two carry the same priority, the tie falls to whichever resource group was +created first. That is rarely intended, so give them distinct priorities when +the order matters. + +The clause is accepted only here. It is rejected on +[`ALTER USER`](/docs/query/sql/acl/alter-user-set-resource-group/) and +[`ALTER SERVICE ACCOUNT`](/docs/query/sql/acl/alter-service-account-set-resource-group/), +which are one-to-one and have nothing to break a tie between. + +A direct mapping on a user beats every ACL group mapping regardless of priority. + +## Examples + +```questdb-sql +-- every member's queries run under the adhoc workload policy +ALTER GROUP analysts SET RESOURCE GROUP adhoc MAPPING PRIORITY 10; +-- oncall wins for anyone who is in both groups +ALTER GROUP oncall SET RESOURCE GROUP dashboards MAPPING PRIORITY 20; +``` + +Verify with [`SHOW GROUPS`](/docs/query/sql/show/#show-groups), which reports the +mapping in its `resource_group` and `resource_group_priority` columns: + +```questdb-sql +SELECT name, resource_group, resource_group_priority FROM (SHOW GROUPS); +``` + +| name | resource_group | resource_group_priority | +| -------- | -------------- | ----------------------- | +| analysts | adhoc | 10 | +| oncall | dashboards | 20 | + +Both columns are `null` for a group that is not mapped. + +## See also + +- [ALTER GROUP UNSET RESOURCE GROUP](/docs/query/sql/acl/alter-group-unset-resource-group/) +- [CREATE RESOURCE GROUP](/docs/query/sql/acl/create-resource-group/) +- [Resource groups](/docs/concepts/resource-groups/) diff --git a/documentation/query/sql/acl/alter-group-unset-resource-group.md b/documentation/query/sql/acl/alter-group-unset-resource-group.md new file mode 100644 index 000000000..d3b267b77 --- /dev/null +++ b/documentation/query/sql/acl/alter-group-unset-resource-group.md @@ -0,0 +1,66 @@ +--- +title: ALTER GROUP UNSET RESOURCE GROUP reference +sidebar_label: UNSET RESOURCE GROUP +description: + "ALTER GROUP UNSET RESOURCE GROUP removes an ACL group's resource group + mapping. Applies to QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +`ALTER GROUP ... UNSET RESOURCE GROUP` removes the ACL group's +[resource group](/docs/concepts/resource-groups/) mapping, along with its +priority. + +--- + +## Syntax + +```questdb-sql +ALTER GROUP groupName UNSET RESOURCE GROUP; +``` + +## Description + +Members stop inheriting the mapping from this group. Each one falls back to the +highest-priority mapping among its remaining ACL groups, or to `DEFAULT`. A +member with a direct mapping of its own is unaffected, because a direct mapping +always won anyway. + +The priority is removed with the mapping; there is no way to clear one while +keeping the other. + +The statement requires the +[`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission, and affects +queries that start after the change rather than one already running. + +Unmapping is also the prerequisite for dropping a resource group: +[`DROP RESOURCE GROUP`](/docs/query/sql/acl/drop-resource-group/) is refused +while any principal is still mapped to it. + +## Examples + +```questdb-sql +ALTER GROUP analysts UNSET RESOURCE GROUP; +``` + +`SHOW GROUPS` then reports `null` in both columns: + +```questdb-sql +SELECT name, resource_group, resource_group_priority FROM (SHOW GROUPS); +``` + +| name | resource_group | resource_group_priority | +| -------- | -------------- | ----------------------- | +| analysts | null | null | + +## See also + +- [ALTER GROUP SET RESOURCE GROUP](/docs/query/sql/acl/alter-group-set-resource-group/) +- [DROP RESOURCE GROUP](/docs/query/sql/acl/drop-resource-group/) +- [Resource groups](/docs/concepts/resource-groups/) diff --git a/documentation/query/sql/acl/alter-group-with-external-alias.md b/documentation/query/sql/acl/alter-group-with-external-alias.md new file mode 100644 index 000000000..0428a4ebe --- /dev/null +++ b/documentation/query/sql/acl/alter-group-with-external-alias.md @@ -0,0 +1,60 @@ +--- +title: ALTER GROUP WITH EXTERNAL ALIAS reference +sidebar_label: WITH EXTERNAL ALIAS +description: + "ALTER GROUP WITH EXTERNAL ALIAS maps an external OIDC or LDAP group to a + QuestDB group. Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER GROUP ... WITH EXTERNAL ALIAS` maps an external OIDC or LDAP group to a +QuestDB group, so members of the external group inherit its permissions on +login. + +--- + +## Syntax + +```questdb-sql +ALTER GROUP groupName WITH EXTERNAL ALIAS externalAlias; +``` + +## Description + +Adding an alias requires the `ADD EXTERNAL ALIAS` permission. Quote the alias +when it contains commas, spaces, or `=`, as LDAP distinguished names do. + +For the external group mapping flow, see the +[OpenID Connect (OIDC) integration](/docs/security/oidc/#mapping-user-permissions) +guide. To create a group and its alias in one statement, use +[`CREATE GROUP ... WITH EXTERNAL ALIAS`](/docs/query/sql/acl/create-group/). + +:::note + +This is identity mapping: it decides which QuestDB group an external identity +lands in. It is unrelated to +[`SET RESOURCE GROUP`](/docs/query/sql/acl/alter-group-set-resource-group/), +which decides how much of the instance that group's queries may consume. + +::: + +## Examples + +```questdb-sql +ALTER GROUP analysts WITH EXTERNAL ALIAS 'CN=Analysts,OU=Users,DC=example,DC=com'; +``` + +The alias can be verified with +[`SHOW GROUPS`](/docs/query/sql/show/#show-groups), which reports it in the +`external_alias` column. + +## See also + +- [ALTER GROUP DROP EXTERNAL ALIAS](/docs/query/sql/acl/alter-group-drop-external-alias/) +- [CREATE GROUP](/docs/query/sql/acl/create-group/) +- [OpenID Connect (OIDC) integration](/docs/security/oidc/#mapping-user-permissions) diff --git a/documentation/query/sql/acl/alter-group.md b/documentation/query/sql/acl/alter-group.md deleted file mode 100644 index 4ba801339..000000000 --- a/documentation/query/sql/acl/alter-group.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -title: ALTER GROUP reference -sidebar_label: ALTER GROUP -description: - "ALTER GROUP sets a per-group query memory limit or maps an external OIDC or - LDAP group alias. Applies to RBAC in QuestDB Enterprise." ---- - -import { EnterpriseNote } from "@site/src/components/EnterpriseNote" - - - RBAC provides fine-grained database permissions management. - - -`ALTER GROUP` modifies group settings. - -For full documentation of the Access Control List and Role-based Access Control, -see the [RBAC operations](/docs/security/rbac) page. - ---- - -## Syntax - -```questdb-sql title="Set or clear memory limit" -ALTER GROUP groupName SET MEMORY LIMIT { size | UNLIMITED }; -``` - -```questdb-sql title="Add or remove external alias" -ALTER GROUP groupName { WITH | DROP } EXTERNAL ALIAS externalAlias; -``` - -## Description - -- `ALTER GROUP groupName SET MEMORY LIMIT size` - caps the native memory that - each query run by a member of the group may allocate. `size` is a byte count - or a size with a `K`, `M`, or `G` suffix, such as `512M` or `2G`. -- `ALTER GROUP groupName SET MEMORY LIMIT UNLIMITED` - clears the group's limit. - Members without a limit of their own then fall back to the most restrictive - limit among their other groups, or to the workload limit - (`cairo.query.memory.limit.bytes`). `SET MEMORY LIMIT 0` does the same. -- `ALTER GROUP groupName WITH EXTERNAL ALIAS externalAlias` - maps an external - OIDC or LDAP group to this group. -- `ALTER GROUP groupName DROP EXTERNAL ALIAS externalAlias` - removes an external - group mapping. - -A group limit applies to a member only when that member has no limit of its own. -When several of a user's groups set a limit, the most restrictive one applies. -Setting a group limit requires the `SET MEMORY LIMIT` permission. See -[memory limits](/docs/security/rbac/#memory-limits) for how a group limit -interacts with the -[`cairo.query.memory.limit.bytes`](/docs/configuration/cairo-engine/#cairoquerymemorylimitbytes) -workload limit. - -Adding an alias requires the `ADD EXTERNAL ALIAS` permission and removing one -requires `REMOVE EXTERNAL ALIAS`. Quote the alias when it contains commas, -spaces, or `=`, as LDAP distinguished names do. For external group mapping with -OIDC or LDAP, see the -[OpenID Connect (OIDC) integration](/docs/security/oidc/#mapping-user-permissions) -guide. - -## Examples - -### Set memory limit - -```questdb-sql --- cap queries of the group's members at 2 GiB of native memory -ALTER GROUP analysts SET MEMORY LIMIT 2G; --- remove the limit -ALTER GROUP analysts SET MEMORY LIMIT UNLIMITED; -``` - -The configured value can be verified with -[`SHOW GROUPS`](/docs/query/sql/show/#show-groups), which reports it in the -`memory_limit` column. - -### Map an external group - -```questdb-sql -ALTER GROUP analysts WITH EXTERNAL ALIAS 'CN=Analysts,OU=Users,DC=example,DC=com'; -ALTER GROUP analysts DROP EXTERNAL ALIAS 'CN=Analysts,OU=Users,DC=example,DC=com'; -``` diff --git a/documentation/query/sql/acl/alter-resource-group.md b/documentation/query/sql/acl/alter-resource-group.md new file mode 100644 index 000000000..a659915e1 --- /dev/null +++ b/documentation/query/sql/acl/alter-resource-group.md @@ -0,0 +1,108 @@ +--- +title: ALTER RESOURCE GROUP reference +sidebar_label: ALTER RESOURCE GROUP +description: + "ALTER RESOURCE GROUP changes a group's admission, CPU weight and memory + policy, clears parameters, or renames the group. Applies to QuestDB + Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +`ALTER RESOURCE GROUP` changes an existing query workload policy. + +For what the limits actually guarantee, see +[resource groups](/docs/concepts/resource-groups/). + +--- + +## Syntax + +```questdb-sql title="Set parameters" +ALTER RESOURCE GROUP groupName + SET ( parameter = value [, parameter = value ...] ); +``` + +```questdb-sql title="Clear parameters back to the instance defaults" +ALTER RESOURCE GROUP groupName RESET ( parameter [, parameter ...] ); +``` + +```questdb-sql title="Rename" +ALTER RESOURCE GROUP groupName RENAME TO newName; +``` + +The parameters are the same ones +[`CREATE RESOURCE GROUP`](/docs/query/sql/acl/create-resource-group/#parameters) +accepts: `cpu_weight`, `max_active_queries`, `max_queued_queries`, +`queue_timeout` and `memory_limit`. + +## Description + +`SET` changes only the parameters named; anything else the group already sets is +left alone. `RESET` clears a parameter so it falls back to the instance default, +which is not always "unlimited": `RESET (cpu_weight)` returns the group to a +weight of 100, and `RESET (queue_timeout)` returns it to 30 seconds. + +The statement requires the +[`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission. + +`DEFAULT` can be altered but not renamed. `ALTER RESOURCE GROUP DEFAULT RENAME TO` +fails with `built-in Resource Group cannot be renamed`. + +Renaming preserves the policy and every principal mapping, because mappings +attach to the group's identity rather than its name. + +### Effect on queries already running + +A policy change applies online. It does not cancel anything running at the +moment `ALTER` executes: + +| Change | Effect on existing work | +| ---------------------- | ------------------------------------------------------------------------------------------------- | +| `cpu_weight` | Subsequent scheduling uses the new policy | +| `max_active_queries` | Existing slots are retained; subsequent admission, including a resumed cursor, uses the new limit | +| `max_queued_queries`, `queue_timeout` | New admission requests use the new settings; an already queued request keeps its deadline | +| `memory_limit` | Subsequent allocations check the new budget; existing memory is released normally | + +Lowering `memory_limit` below current usage does not fail running queries +retroactively, but it can make their subsequent allocations fail. The +single-query ceiling is captured when a query starts, so changing the group +budget does not replace a principal's own limit. + +## Examples + +```questdb-sql title="Raise the weight and cap concurrency" +ALTER RESOURCE GROUP analytics SET (cpu_weight = 300, max_active_queries = 8); +``` + +```questdb-sql title="Remove the group's memory ceiling" +ALTER RESOURCE GROUP reporting SET (memory_limit = UNLIMITED); +``` + +`RESET (memory_limit)` and `SET (memory_limit = 0)` do the same thing. + +```questdb-sql title="Clear parameters back to the instance defaults" +ALTER RESOURCE GROUP analytics RESET (memory_limit, max_active_queries); +``` + +```questdb-sql title="Rename" +ALTER RESOURCE GROUP analytics RENAME TO reporting; +``` + +`DEFAULT` takes limits like any other group, which is how you bound everything +that is not explicitly mapped: + +```questdb-sql title="Cap the default workload" +ALTER RESOURCE GROUP DEFAULT SET (max_active_queries = 16); +``` + +## See also + +- [CREATE RESOURCE GROUP](/docs/query/sql/acl/create-resource-group/) +- [DROP RESOURCE GROUP](/docs/query/sql/acl/drop-resource-group/) +- [Configure and use resource groups](/docs/operations/resource-groups/) diff --git a/documentation/query/sql/acl/alter-service-account-create-token.md b/documentation/query/sql/acl/alter-service-account-create-token.md new file mode 100644 index 000000000..129a10845 --- /dev/null +++ b/documentation/query/sql/acl/alter-service-account-create-token.md @@ -0,0 +1,89 @@ +--- +title: ALTER SERVICE ACCOUNT CREATE TOKEN reference +sidebar_label: CREATE TOKEN +description: + "ALTER SERVICE ACCOUNT CREATE TOKEN adds a JWK or REST API token to a service + account, with an optional TTL and REFRESH. Applies to QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER SERVICE ACCOUNT ... CREATE TOKEN` adds a JSON Web Key or a REST API token +to a service account. + +--- + +## Syntax + +```questdb-sql +ALTER SERVICE ACCOUNT serviceAccountName CREATE TOKEN TYPE + { JWK | REST WITH TTL timeUnit [REFRESH] }; +``` + +## Description + +- `ALTER SERVICE ACCOUNT serviceAccountName CREATE TOKEN TYPE JWK` adds a JSON + Web Key to the service account. It returns the public key (x, y) and the + private key. **The private key is not stored in QuestDB**, so capture it when + it is returned. +- `ALTER SERVICE ACCOUNT serviceAccountName CREATE TOKEN TYPE REST WITH TTL timeUnit [REFRESH]` + adds a REST API token to the service account. + +### TTL and REFRESH + +The TTL value is an integer and a unit, such as `1m`. The supported units are: + +- `s` for second +- `m` for minute +- `h` for hour +- `d` for day + +The minimum allowable TTL value is 1 minute and the maximum is 10 years (10 \* +365 days). + +`REFRESH` is optional. When specified, the token's expiration timestamp is +refreshed on each successful authentication. + +### REST API tokens and database replication + +Many [QuestDB Enterprise](/enterprise/) instances run within active +[database replication](/docs/high-availability/setup/) clusters. With replication +enabled, the REST API token is refreshed on successful authentication to the +**primary** node. The token is **not** refreshed during successful +authentications to **replica** nodes. + +Therefore, tokens with the `REFRESH` modifier are for use only on the **primary** +node. + +## Examples + +```questdb-sql title="Add a JSON Web Key" +ALTER SERVICE ACCOUNT client_app CREATE TOKEN TYPE JWK; +``` + +```questdb-sql title="Add a REST API token" +-- generate a token with no TTL refresh +ALTER SERVICE ACCOUNT client_app CREATE TOKEN TYPE REST WITH TTL '1m'; +-- generate a token with TTL refresh +ALTER SERVICE ACCOUNT client_app CREATE TOKEN TYPE REST WITH TTL '1m' REFRESH; +``` + +Verify with: + +```questdb-sql +SHOW SERVICE ACCOUNT client_app; +``` + +| auth_type | enabled | +| ---------- | ------- | +| Password | false | +| JWK Token | true | +| REST Token | false | + +## See also + +- [DROP TOKEN](/docs/query/sql/acl/alter-service-account-drop-token/) diff --git a/documentation/query/sql/acl/alter-service-account-disable.md b/documentation/query/sql/acl/alter-service-account-disable.md new file mode 100644 index 000000000..f1746e931 --- /dev/null +++ b/documentation/query/sql/acl/alter-service-account-disable.md @@ -0,0 +1,47 @@ +--- +title: ALTER SERVICE ACCOUNT DISABLE reference +sidebar_label: DISABLE +description: + "ALTER SERVICE ACCOUNT DISABLE turns off a service account without deleting + it. Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER SERVICE ACCOUNT ... DISABLE` turns off a service account without deleting +it. + +--- + +## Syntax + +```questdb-sql +ALTER SERVICE ACCOUNT serviceAccountName DISABLE; +``` + +## Description + +A disabled service account keeps its permissions and tokens; it simply cannot +authenticate, and cannot be assumed, until it is enabled again. + +To remove one permanently instead, use +[`DROP SERVICE ACCOUNT`](/docs/query/sql/acl/drop-service-account/). + +## Examples + +```questdb-sql +ALTER SERVICE ACCOUNT client_app DISABLE; +``` + +Verify with +[`SHOW SERVICE ACCOUNTS`](/docs/query/sql/show/#show-service-accounts), which +reports `false` in its `enabled` column. + +## See also + +- [ALTER SERVICE ACCOUNT ENABLE](/docs/query/sql/acl/alter-service-account-enable/) +- [DROP SERVICE ACCOUNT](/docs/query/sql/acl/drop-service-account/) diff --git a/documentation/query/sql/acl/alter-service-account-drop-token.md b/documentation/query/sql/acl/alter-service-account-drop-token.md new file mode 100644 index 000000000..0a93028bb --- /dev/null +++ b/documentation/query/sql/acl/alter-service-account-drop-token.md @@ -0,0 +1,63 @@ +--- +title: ALTER SERVICE ACCOUNT DROP TOKEN reference +sidebar_label: DROP TOKEN +description: + "ALTER SERVICE ACCOUNT DROP TOKEN removes a JWK or one or all REST API tokens + from a service account. Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER SERVICE ACCOUNT ... DROP TOKEN` removes a JSON Web Key or a REST API +token from a service account. + +--- + +## Syntax + +```questdb-sql +ALTER SERVICE ACCOUNT serviceAccountName DROP TOKEN TYPE + { JWK | REST [token] }; +``` + +## Description + +- `ALTER SERVICE ACCOUNT serviceAccountName DROP TOKEN TYPE JWK` removes the JSON + Web Key from the service account. +- `ALTER SERVICE ACCOUNT serviceAccountName DROP TOKEN TYPE REST token` removes + that REST token from the service account. +- `ALTER SERVICE ACCOUNT serviceAccountName DROP TOKEN TYPE REST` with no token + removes **all** of the service account's REST tokens. + +## Examples + +```questdb-sql title="Remove the JSON Web Key" +ALTER SERVICE ACCOUNT client_app DROP TOKEN TYPE JWK; +``` + +```questdb-sql title="Remove REST API tokens" +-- drop a single REST API token +ALTER SERVICE ACCOUNT client_app DROP TOKEN TYPE REST 'qt1cNK6s2t79f76GmTBN9k7XTWm5wwOtF7C0UBxiHGPn44'; +-- drop all REST API tokens for the given service account +ALTER SERVICE ACCOUNT client_app DROP TOKEN TYPE REST; +``` + +Verify with: + +```questdb-sql +SHOW SERVICE ACCOUNT client_app; +``` + +| auth_type | enabled | +| ---------- | ------- | +| Password | true | +| JWK Token | false | +| REST Token | false | + +## See also + +- [CREATE TOKEN](/docs/query/sql/acl/alter-service-account-create-token/) diff --git a/documentation/query/sql/acl/alter-service-account-enable.md b/documentation/query/sql/acl/alter-service-account-enable.md new file mode 100644 index 000000000..4ebe6158d --- /dev/null +++ b/documentation/query/sql/acl/alter-service-account-enable.md @@ -0,0 +1,46 @@ +--- +title: ALTER SERVICE ACCOUNT ENABLE reference +sidebar_label: ENABLE +description: + "ALTER SERVICE ACCOUNT ENABLE turns a disabled service account back on. + Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER SERVICE ACCOUNT ... ENABLE` turns a disabled service account back on. + +--- + +## Syntax + +```questdb-sql +ALTER SERVICE ACCOUNT serviceAccountName ENABLE; +``` + +## Description + +The service account can authenticate and be assumed again, with the permissions +and tokens it had before it was disabled. + +A service account is enabled when created, so this is only needed after +[`DISABLE`](/docs/query/sql/acl/alter-service-account-disable/). + +## Examples + +```questdb-sql +ALTER SERVICE ACCOUNT client_app ENABLE; +``` + +Verify with +[`SHOW SERVICE ACCOUNTS`](/docs/query/sql/show/#show-service-accounts), which +reports `true` in its `enabled` column. + +## See also + +- [ALTER SERVICE ACCOUNT DISABLE](/docs/query/sql/acl/alter-service-account-disable/) +- [CREATE SERVICE ACCOUNT](/docs/query/sql/acl/create-service-account/) diff --git a/documentation/query/sql/acl/alter-service-account-set-memory-limit.md b/documentation/query/sql/acl/alter-service-account-set-memory-limit.md new file mode 100644 index 000000000..8db6d5533 --- /dev/null +++ b/documentation/query/sql/acl/alter-service-account-set-memory-limit.md @@ -0,0 +1,65 @@ +--- +title: ALTER SERVICE ACCOUNT SET MEMORY LIMIT reference +sidebar_label: SET MEMORY LIMIT +description: + "ALTER SERVICE ACCOUNT SET MEMORY LIMIT caps the native memory each of a + service account's queries may allocate. Applies to QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER SERVICE ACCOUNT ... SET MEMORY LIMIT` caps the native memory each of the +service account's queries may allocate. + +--- + +## Syntax + +```questdb-sql +ALTER SERVICE ACCOUNT serviceAccountName SET MEMORY LIMIT { size | UNLIMITED }; +``` + +## Description + +- `ALTER SERVICE ACCOUNT serviceAccountName SET MEMORY LIMIT size` caps the + native memory each of the service account's queries may allocate. `size` is a + byte count or a size with a `K`, `M`, or `G` suffix, such as `512M` or `2G`. +- `ALTER SERVICE ACCOUNT serviceAccountName SET MEMORY LIMIT UNLIMITED` clears + the service account's limit. The workload limit + (`cairo.query.memory.limit.bytes`) then applies. `SET MEMORY LIMIT 0` does the + same. + +A user who assumes the service account runs under its memory limit. No group +limit can apply, because a service account cannot belong to an ACL group, so +either it has a limit of its own or only the workload limit applies. + +Setting it requires the `SET MEMORY LIMIT` permission. See +[memory limits](/docs/security/rbac/#memory-limits) for how the limit interacts +with the +[`cairo.query.memory.limit.bytes`](/docs/configuration/cairo-engine/#cairoquerymemorylimitbytes) +workload limit. + +This is a per-query ceiling. To bound what a whole workload may hold at once +instead, use a +[resource group](/docs/query/sql/acl/alter-service-account-set-resource-group/). + +## Examples + +```questdb-sql +-- cap the service account's queries at 1 GiB of native memory +ALTER SERVICE ACCOUNT client_app SET MEMORY LIMIT 1G; +-- remove the limit +ALTER SERVICE ACCOUNT client_app SET MEMORY LIMIT UNLIMITED; +``` + +The configured value can be verified with +[`SHOW SERVICE ACCOUNTS`](/docs/query/sql/show/#show-service-accounts), which +reports it in the `memory_limit` column. + +## See also + +- [Memory limits](/docs/security/rbac/#memory-limits) diff --git a/documentation/query/sql/acl/alter-service-account-set-resource-group.md b/documentation/query/sql/acl/alter-service-account-set-resource-group.md new file mode 100644 index 000000000..b5312cdb7 --- /dev/null +++ b/documentation/query/sql/acl/alter-service-account-set-resource-group.md @@ -0,0 +1,67 @@ +--- +title: ALTER SERVICE ACCOUNT SET RESOURCE GROUP reference +sidebar_label: SET RESOURCE GROUP +description: + "ALTER SERVICE ACCOUNT SET RESOURCE GROUP places a service account's queries + under a resource group. Applies to QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +`ALTER SERVICE ACCOUNT ... SET RESOURCE GROUP` places the service account's +queries under a [resource group](/docs/concepts/resource-groups/), which governs +their admission, CPU share and memory budget. + +--- + +## Syntax + +```questdb-sql +ALTER SERVICE ACCOUNT serviceAccountName SET RESOURCE GROUP resourceGroupName; +``` + +## Description + +A session that assumes the service account keeps the resource group of the +principal that logged in. The account's own mapping applies to sessions that +authenticate as it. + +`MAPPING PRIORITY` is **not** accepted here, because a service account has +exactly one mapping and there is nothing to break a tie between. It belongs to +[`ALTER GROUP SET RESOURCE GROUP`](/docs/query/sql/acl/alter-group-set-resource-group/). + +The statement requires the +[`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission, and affects +queries that start after the change rather than one already running. + +This governs the queries the account runs. It does not throttle ingestion, which +resource groups do not manage. + +## Examples + +```questdb-sql +ALTER SERVICE ACCOUNT client_app SET RESOURCE GROUP automation; +``` + +Verify with +[`SHOW SERVICE ACCOUNTS`](/docs/query/sql/show/#show-service-accounts), which +reports the mapping in its `resource_group` column: + +```questdb-sql +SELECT name, resource_group FROM (SHOW SERVICE ACCOUNTS); +``` + +| name | resource_group | +| ---------- | -------------- | +| client_app | automation | + +## See also + +- [ALTER SERVICE ACCOUNT UNSET RESOURCE GROUP](/docs/query/sql/acl/alter-service-account-unset-resource-group/) +- [CREATE RESOURCE GROUP](/docs/query/sql/acl/create-resource-group/) +- [Resource groups](/docs/concepts/resource-groups/) diff --git a/documentation/query/sql/acl/alter-service-account-unset-resource-group.md b/documentation/query/sql/acl/alter-service-account-unset-resource-group.md new file mode 100644 index 000000000..38098b860 --- /dev/null +++ b/documentation/query/sql/acl/alter-service-account-unset-resource-group.md @@ -0,0 +1,62 @@ +--- +title: ALTER SERVICE ACCOUNT UNSET RESOURCE GROUP reference +sidebar_label: UNSET RESOURCE GROUP +description: + "ALTER SERVICE ACCOUNT UNSET RESOURCE GROUP removes a service account's + resource group mapping. Applies to QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +`ALTER SERVICE ACCOUNT ... UNSET RESOURCE GROUP` removes the service account's +[resource group](/docs/concepts/resource-groups/) mapping. + +--- + +## Syntax + +```questdb-sql +ALTER SERVICE ACCOUNT serviceAccountName UNSET RESOURCE GROUP; +``` + +## Description + +The service account returns to `DEFAULT`. Unlike a user, it has no ACL group +mapping to fall back to, because a service account cannot belong to a group. +Unsetting therefore leaves it governed only by `DEFAULT`'s policy. + +The statement requires the +[`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission, and affects +queries that start after the change rather than one already running. + +Unmapping is also the prerequisite for dropping a resource group: +[`DROP RESOURCE GROUP`](/docs/query/sql/acl/drop-resource-group/) is refused +while any principal is still mapped to it. + +## Examples + +```questdb-sql +ALTER SERVICE ACCOUNT client_app UNSET RESOURCE GROUP; +``` + +`SHOW SERVICE ACCOUNTS` then reports `null` in the `resource_group` column, +which for a service account means `DEFAULT`: + +```questdb-sql +SELECT name, resource_group FROM (SHOW SERVICE ACCOUNTS); +``` + +| name | resource_group | +| ---------- | -------------- | +| client_app | null | + +## See also + +- [ALTER SERVICE ACCOUNT SET RESOURCE GROUP](/docs/query/sql/acl/alter-service-account-set-resource-group/) +- [DROP RESOURCE GROUP](/docs/query/sql/acl/drop-resource-group/) +- [Resource groups](/docs/concepts/resource-groups/) diff --git a/documentation/query/sql/acl/alter-service-account-with-no-password.md b/documentation/query/sql/acl/alter-service-account-with-no-password.md new file mode 100644 index 000000000..7c0f9e482 --- /dev/null +++ b/documentation/query/sql/acl/alter-service-account-with-no-password.md @@ -0,0 +1,58 @@ +--- +title: ALTER SERVICE ACCOUNT WITH NO PASSWORD reference +sidebar_label: WITH NO PASSWORD +description: + "ALTER SERVICE ACCOUNT WITH NO PASSWORD removes a service account's password. + Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER SERVICE ACCOUNT ... WITH NO PASSWORD` removes a service account's +password. + +--- + +## Syntax + +```questdb-sql +ALTER SERVICE ACCOUNT serviceAccountName WITH NO PASSWORD; +``` + +## Description + +This is the only way to clear a password. `WITH PASSWORD ''` does not work, +because empty passwords are rejected. + +Removing the password does not disable the account. If it still holds a JWK or +REST token it can continue to authenticate with that; to stop access entirely, +use [`DISABLE`](/docs/query/sql/acl/alter-service-account-disable/) or drop its +tokens with +[`DROP TOKEN`](/docs/query/sql/acl/alter-service-account-drop-token/). + +## Examples + +```questdb-sql +ALTER SERVICE ACCOUNT client_app WITH NO PASSWORD; +``` + +Verify with: + +```questdb-sql +SHOW SERVICE ACCOUNT client_app; +``` + +| auth_type | enabled | +| ---------- | ------- | +| Password | false | +| JWK Token | true | +| REST Token | false | + +## See also + +- [ALTER SERVICE ACCOUNT WITH PASSWORD](/docs/query/sql/acl/alter-service-account-with-password/) +- [ALTER SERVICE ACCOUNT DISABLE](/docs/query/sql/acl/alter-service-account-disable/) diff --git a/documentation/query/sql/acl/alter-service-account-with-password.md b/documentation/query/sql/acl/alter-service-account-with-password.md new file mode 100644 index 000000000..a97e3308c --- /dev/null +++ b/documentation/query/sql/acl/alter-service-account-with-password.md @@ -0,0 +1,53 @@ +--- +title: ALTER SERVICE ACCOUNT WITH PASSWORD reference +sidebar_label: WITH PASSWORD +description: + "ALTER SERVICE ACCOUNT WITH PASSWORD sets a service account's password. + Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER SERVICE ACCOUNT ... WITH PASSWORD` sets a service account's password. + +--- + +## Syntax + +```questdb-sql +ALTER SERVICE ACCOUNT serviceAccountName WITH PASSWORD password; +``` + +## Description + +Setting a password replaces any existing one. Empty passwords are rejected, so +`WITH PASSWORD ''` cannot be used to clear one; use +[`WITH NO PASSWORD`](/docs/query/sql/acl/alter-service-account-with-no-password/) +instead. + +## Examples + +```questdb-sql +ALTER SERVICE ACCOUNT client_app WITH PASSWORD '1m@re@lh@cker'; +``` + +Verify with: + +```questdb-sql +SHOW SERVICE ACCOUNT client_app; +``` + +| auth_type | enabled | +| ---------- | ------- | +| Password | true | +| JWK Token | false | +| REST Token | false | + +## See also + +- [ALTER SERVICE ACCOUNT WITH NO PASSWORD](/docs/query/sql/acl/alter-service-account-with-no-password/) +- [CREATE SERVICE ACCOUNT](/docs/query/sql/acl/create-service-account/) diff --git a/documentation/query/sql/acl/alter-service-account.md b/documentation/query/sql/acl/alter-service-account.md deleted file mode 100644 index ca73247ac..000000000 --- a/documentation/query/sql/acl/alter-service-account.md +++ /dev/null @@ -1,200 +0,0 @@ ---- -title: ALTER SERVICE ACCOUNT reference -sidebar_label: ALTER SERVICE ACCOUNT -description: - "ALTER SERVICE ACCOUNT enables or disables a service account, manages - passwords and tokens, and sets its query memory limit. Applies to RBAC in - QuestDB Enterprise." ---- - -import { EnterpriseNote } from "@site/src/components/EnterpriseNote" - - - RBAC provides fine-grained database permissions management. - - -`ALTER SERVICE ACCOUNT` modifies service account settings. - -For full documentation of the Access Control List and Role-based Access Control, -see the [RBAC operations](/docs/security/rbac) page. - ---- - -## Syntax - -```questdb-sql title="Enable / disable" -ALTER SERVICE ACCOUNT serviceAccountName { ENABLE | DISABLE }; -``` - -```questdb-sql title="Set or remove password" -ALTER SERVICE ACCOUNT serviceAccountName WITH { PASSWORD password | NO PASSWORD }; -``` - -```questdb-sql title="Create token" -ALTER SERVICE ACCOUNT serviceAccountName CREATE TOKEN TYPE - { JWK | REST WITH TTL timeUnit [REFRESH] }; -``` - -```questdb-sql title="Drop token" -ALTER SERVICE ACCOUNT serviceAccountName DROP TOKEN TYPE - { JWK | REST [token] }; -``` - -```questdb-sql title="Set or clear memory limit" -ALTER SERVICE ACCOUNT serviceAccountName SET MEMORY LIMIT { size | UNLIMITED }; -``` - -## Description - -- `ALTER SERVICE ACCOUNT serviceAccountName ENABLE` - enables service account. -- `ALTER SERVICE ACCOUNT serviceAccountName DISABLE` - disables service account. -- `ALTER SERVICE ACCOUNT serviceAccountName WITH PASSWORD password` - sets - password for the service account. -- `ALTER SERVICE ACCOUNT serviceAccountName WITH NO PASSWORD` - removes password - for the service account. -- `ALTER SERVICE ACCOUNT serviceAccountName CREATE TOKEN TYPE JWK` - adds Json - Web Key to the service account. Returns public key (x, y) and private key. The - private key is not stored in QuestDB. -- `ALTER SERVICE ACCOUNT serviceAccountName DROP TOKEN TYPE JWK` - removes Json - Web Key from the service account. -- `ALTER USER serviceAccountName CREATE TOKEN TYPE REST WITH TTL timeUnit REFRESH` - - adds REST token to the service account. -- `ALTER USER serviceAccountName DROP TOKEN TYPE REST token` - removes REST - token from the service account. -- `ALTER SERVICE ACCOUNT serviceAccountName SET MEMORY LIMIT size` - caps the - native memory each of the service account's queries may allocate. `size` is a - byte count or a size with a `K`, `M`, or `G` suffix, such as `512M` or `2G`. -- `ALTER SERVICE ACCOUNT serviceAccountName SET MEMORY LIMIT UNLIMITED` - clears - the service account's limit. The workload limit - (`cairo.query.memory.limit.bytes`) then applies. `SET MEMORY LIMIT 0` does the - same. - -A user who assumes the service account runs under its memory limit. Group limits -are never merged into a service account. Setting it requires the -`SET MEMORY LIMIT` permission. See -[memory limits](/docs/security/rbac/#memory-limits) for how the limit interacts -with the -[`cairo.query.memory.limit.bytes`](/docs/configuration/cairo-engine/#cairoquerymemorylimitbytes) -workload limit. - -## Examples - -### Enable service account - -```questdb-sql -ALTER SERVICE ACCOUNT client_app ENABLE; -``` - -### Disable service account - -```questdb-sql -ALTER SERVICE ACCOUNT client_app DISABLE; -``` - -### Set password - -```questdb-sql -ALTER SERVICE ACCOUNT client_app WITH PASSWORD '1m@re@lh@cker'; -``` - -### Remove password - -```questdb-sql -ALTER SERVICE ACCOUNT client_app WITH NO PASSWORD; -``` - -Removing a password is not possible using `WITH PASSWORD ''` as the database -will reject empty passwords. - -### Add Json Web Key - -```questdb-sql -ALTER SERVICE ACCOUNT client_app CREATE TOKEN TYPE JWK; -``` - -### Remove Json Web Key - -```questdb-sql -ALTER SERVICE ACCOUNT client_app DROP TOKEN TYPE JWK; -``` - -Result of commands above can be verified with `SHOW USER`, e.g. - -```questdb-sql -SHOW SERVICE ACCOUNT client_app; -``` - -| auth_type | enabled | -| ---------- | ------- | -| Password | false | -| JWK Token | true | -| REST Token | false | - -### Add REST API token - -```questdb-sql --- generate a token with no TTL refresh -ALTER SERVICE ACCOUNT client_app CREATE TOKEN TYPE REST WITH TTL '1m'; --- generate a token with TTL refresh -ALTER SERVICE ACCOUNT client_app CREATE TOKEN TYPE REST WITH TTL '1m' REFRESH; -``` - -Here, the TTL (Time-to-Live) value should contain an integer and a unit, such as -`1m`. The supported units are: - -- `s` - second -- `m` - minute -- `h` - hour -- `d` - day - -The minimum allowable TTL value is 1 minute and the maximum value is 10 years -(10 \* 365 days). - -The `REFRESH` modifier is optional. When the `REFRESH` modifier is specified, -the token's expiration timestamp will be refreshed on each successful -authentication. - -#### Rest API tokens and database replication - -Many [QuestDB Enterprise](/enterprise/) instances run within active -[database replication](/docs/high-availability/setup/) clusters. With replication -enabled, the REST API token will be refreshed on successful authentication to -the **primary** node. The token will **not** be refreshed during successful -authentications to **replica** nodes. - -Therefore, tokens with the `REFRESH` modifier are for use only on the -**primary** node. - -### Remove REST API token - -```questdb-sql --- drop single REST API token -ALTER SERVICE ACCOUNT client_app DROP TOKEN TYPE REST 'qt1cNK6s2t79f76GmTBN9k7XTWm5wwOtF7C0UBxiHGPn44'; --- drop all REST API tokens for the given service account -ALTER SERVICE ACCOUNT client_app DROP TOKEN TYPE REST; -``` - -The result of the above commands can be verified with `SHOW SERVICE ACCOUNT`: - -```questdb-sql -SHOW SERVICE ACCOUNT client_app; -``` - -| auth_type | enabled | -| ---------- | ------- | -| Password | true | -| JWK Token | false | -| REST Token | false | - -### Set memory limit - -```questdb-sql --- cap the service account's queries at 1 GiB of native memory -ALTER SERVICE ACCOUNT client_app SET MEMORY LIMIT 1G; --- remove the limit -ALTER SERVICE ACCOUNT client_app SET MEMORY LIMIT UNLIMITED; -``` - -The configured value can be verified with -[`SHOW SERVICE ACCOUNTS`](/docs/query/sql/show/#show-service-accounts), which -reports it in the `memory_limit` column. diff --git a/documentation/query/sql/acl/alter-user-create-token.md b/documentation/query/sql/acl/alter-user-create-token.md new file mode 100644 index 000000000..6143aa9c7 --- /dev/null +++ b/documentation/query/sql/acl/alter-user-create-token.md @@ -0,0 +1,85 @@ +--- +title: ALTER USER CREATE TOKEN reference +sidebar_label: CREATE TOKEN +description: + "ALTER USER CREATE TOKEN adds a JWK or REST API token to a user account, with + an optional TTL and REFRESH. Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER USER ... CREATE TOKEN` adds a JSON Web Key or a REST API token to a user +account. + +--- + +## Syntax + +```questdb-sql +ALTER USER userName CREATE TOKEN TYPE + { JWK | REST WITH TTL timeUnit [REFRESH] }; +``` + +## Description + +- `ALTER USER username CREATE TOKEN TYPE JWK` adds a JSON Web Key to the user + account. It returns the public key (x, y) and the private key. **The private + key is not stored in QuestDB**, so capture it when it is returned. +- `ALTER USER username CREATE TOKEN TYPE REST WITH TTL timeUnit [REFRESH]` adds a + REST API token to the user account. + +### TTL and REFRESH + +The TTL value is an integer and a unit, such as `1m`. The supported units are: + +- `s` for second +- `m` for minute +- `h` for hour +- `d` for day + +The minimum allowed TTL is 1 minute and the maximum is 10 years (10 \* 365 +days). + +`REFRESH` is optional. When specified, the token's expiration timestamp is +refreshed on each successful authentication. + +:::note + +When replication is used, the token is not refreshed on successful +authentication on replicas, only on the primary node. This makes tokens with the +`REFRESH` modifier meaningful for use on the primary node only. + +::: + +## Examples + +```questdb-sql title="Add a JSON Web Key" +ALTER USER john CREATE TOKEN TYPE JWK; +``` + +```questdb-sql title="Add a REST API token" +-- generate a token with no TTL refresh +ALTER USER john CREATE TOKEN TYPE REST WITH TTL '1m'; +-- generate a token with TTL refresh +ALTER USER john CREATE TOKEN TYPE REST WITH TTL '1m' REFRESH; +``` + +Verify with: + +```questdb-sql +SHOW USER john; +``` + +| auth_type | enabled | +| ---------- | ------- | +| Password | true | +| JWK Token | false | +| REST Token | false | + +## See also + +- [DROP TOKEN](/docs/query/sql/acl/alter-user-drop-token/) diff --git a/documentation/query/sql/acl/alter-user-disable.md b/documentation/query/sql/acl/alter-user-disable.md new file mode 100644 index 000000000..166ed9756 --- /dev/null +++ b/documentation/query/sql/acl/alter-user-disable.md @@ -0,0 +1,46 @@ +--- +title: ALTER USER DISABLE reference +sidebar_label: DISABLE +description: + "ALTER USER DISABLE turns off a user account without deleting it. Applies to + RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER USER ... DISABLE` turns off a user account without deleting it. + +--- + +## Syntax + +```questdb-sql +ALTER USER userName DISABLE; +``` + +## Description + +A disabled user keeps its permissions, group memberships and tokens; it simply +cannot authenticate until it is enabled again. Existing sessions are not the +subject of this statement, so disable is not a way to evict a connected user. + +To remove a user permanently instead, use +[`DROP USER`](/docs/query/sql/acl/drop-user/). + +## Examples + +```questdb-sql +ALTER USER john DISABLE; +``` + +Verify with [`SHOW USERS`](/docs/query/sql/show/#show-users), which reports +`false` in its `enabled` column. + +## See also + +- [ALTER USER ENABLE](/docs/query/sql/acl/alter-user-enable/) +- [DROP USER](/docs/query/sql/acl/drop-user/) diff --git a/documentation/query/sql/acl/alter-user-drop-token.md b/documentation/query/sql/acl/alter-user-drop-token.md new file mode 100644 index 000000000..ac6674e0e --- /dev/null +++ b/documentation/query/sql/acl/alter-user-drop-token.md @@ -0,0 +1,63 @@ +--- +title: ALTER USER DROP TOKEN reference +sidebar_label: DROP TOKEN +description: + "ALTER USER DROP TOKEN removes a JWK or one or all REST API tokens from a user + account. Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER USER ... DROP TOKEN` removes a JSON Web Key or a REST API token from a +user account. + +--- + +## Syntax + +```questdb-sql +ALTER USER userName DROP TOKEN TYPE + { JWK | REST [token] }; +``` + +## Description + +- `ALTER USER username DROP TOKEN TYPE JWK` removes the JSON Web Key from the + user account. +- `ALTER USER username DROP TOKEN TYPE REST token` removes that REST token from + the user account. +- `ALTER USER username DROP TOKEN TYPE REST` with no token removes **all** of the + user's REST tokens. + +## Examples + +```questdb-sql title="Remove the JSON Web Key" +ALTER USER john DROP TOKEN TYPE JWK; +``` + +```questdb-sql title="Remove REST API tokens" +-- drop a single REST API token +ALTER USER john DROP TOKEN TYPE REST 'qt1cNK6s2t79f76GmTBN9k7XTWm5wwOtF7C0UBxiHGPn44'; +-- drop all REST API tokens for the given user +ALTER USER john DROP TOKEN TYPE REST; +``` + +Verify with: + +```questdb-sql +SHOW USER john; +``` + +| auth_type | enabled | +| ---------- | ------- | +| Password | true | +| JWK Token | false | +| REST Token | false | + +## See also + +- [CREATE TOKEN](/docs/query/sql/acl/alter-user-create-token/) diff --git a/documentation/query/sql/acl/alter-user-enable.md b/documentation/query/sql/acl/alter-user-enable.md new file mode 100644 index 000000000..85ec70576 --- /dev/null +++ b/documentation/query/sql/acl/alter-user-enable.md @@ -0,0 +1,46 @@ +--- +title: ALTER USER ENABLE reference +sidebar_label: ENABLE +description: + "ALTER USER ENABLE turns a disabled user account back on. Applies to RBAC in + QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER USER ... ENABLE` turns a disabled user account back on. + +--- + +## Syntax + +```questdb-sql +ALTER USER userName ENABLE; +``` + +## Description + +The user can authenticate again, with the permissions, group memberships and +tokens it had before it was disabled. Nothing is restored or re-granted, because +nothing was removed. + +A user is enabled when created, so this is only needed after +[`DISABLE`](/docs/query/sql/acl/alter-user-disable/). + +## Examples + +```questdb-sql +ALTER USER john ENABLE; +``` + +Verify with [`SHOW USERS`](/docs/query/sql/show/#show-users), which reports +`true` in its `enabled` column. + +## See also + +- [ALTER USER DISABLE](/docs/query/sql/acl/alter-user-disable/) +- [CREATE USER](/docs/query/sql/acl/create-user/) diff --git a/documentation/query/sql/acl/alter-user-set-memory-limit.md b/documentation/query/sql/acl/alter-user-set-memory-limit.md new file mode 100644 index 000000000..493cc535c --- /dev/null +++ b/documentation/query/sql/acl/alter-user-set-memory-limit.md @@ -0,0 +1,65 @@ +--- +title: ALTER USER SET MEMORY LIMIT reference +sidebar_label: SET MEMORY LIMIT +description: + "ALTER USER SET MEMORY LIMIT caps the native memory each of a user's queries + may allocate. Applies to RBAC in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER USER ... SET MEMORY LIMIT` caps the native memory each of the user's +queries may allocate. + +--- + +## Syntax + +```questdb-sql +ALTER USER userName SET MEMORY LIMIT { size | UNLIMITED }; +``` + +## Description + +- `ALTER USER username SET MEMORY LIMIT size` caps the native memory each of the + user's queries may allocate. `size` is a byte count or a size with a `K`, `M`, + or `G` suffix, such as `512M` or `2G`. +- `ALTER USER username SET MEMORY LIMIT UNLIMITED` clears the user's own limit. A + group limit or the workload limit (`cairo.query.memory.limit.bytes`) then + applies. `SET MEMORY LIMIT 0` does the same. + +The limit applies to the user's queries on both the primary and replicas. +Setting it requires the `SET MEMORY LIMIT` permission. + +The built-in admin and external (SSO/OIDC) users cannot be given a limit; the +statement is rejected for both. An external user inherits a limit from its +groups instead. + +A limit set here takes priority over the user's groups and over the +[`cairo.query.memory.limit.bytes`](/docs/configuration/cairo-engine/#cairoquerymemorylimitbytes) +workload limit; see [memory limits](/docs/security/rbac/#memory-limits) for how +limits resolve. + +This is a per-query ceiling. To bound what a whole workload may hold at once +instead, use a +[resource group](/docs/query/sql/acl/alter-user-set-resource-group/). + +## Examples + +```questdb-sql +-- cap the user's queries at 512 MiB of native memory +ALTER USER john SET MEMORY LIMIT 512M; +-- remove the limit +ALTER USER john SET MEMORY LIMIT UNLIMITED; +``` + +Use [`SHOW USERS`](/docs/query/sql/show/#show-users) to inspect the user's own or +inherited group limit in the `memory_limit` column. + +## See also + +- [Memory limits](/docs/security/rbac/#memory-limits) diff --git a/documentation/query/sql/acl/alter-user-set-resource-group.md b/documentation/query/sql/acl/alter-user-set-resource-group.md new file mode 100644 index 000000000..a7627b114 --- /dev/null +++ b/documentation/query/sql/acl/alter-user-set-resource-group.md @@ -0,0 +1,68 @@ +--- +title: ALTER USER SET RESOURCE GROUP reference +sidebar_label: SET RESOURCE GROUP +description: + "ALTER USER SET RESOURCE GROUP places a user's queries under a resource group, + overriding any ACL group mapping. Applies to QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +`ALTER USER ... SET RESOURCE GROUP` places the user's queries under a +[resource group](/docs/concepts/resource-groups/), which governs their +admission, CPU share and memory budget. + +--- + +## Syntax + +```questdb-sql +ALTER USER userName SET RESOURCE GROUP resourceGroupName; +``` + +## Description + +A direct mapping on the user beats any mapping it would inherit from an ACL +group, whatever that group's priority. That is how you make one person an +exception without touching the groups. + +`MAPPING PRIORITY` is **not** accepted here, because a user has exactly one +direct mapping and there is nothing to break a tie between. It belongs to +[`ALTER GROUP SET RESOURCE GROUP`](/docs/query/sql/acl/alter-group-set-resource-group/). + +The statement requires the +[`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission, and affects +queries that start after the change rather than one already running. + +## Examples + +```questdb-sql +ALTER USER john SET RESOURCE GROUP reporting; +``` + +Verify with [`SHOW USERS`](/docs/query/sql/show/#show-users), which reports the +direct mapping in its `resource_group` column: + +```questdb-sql +SELECT name, resource_group FROM (SHOW USERS); +``` + +| name | resource_group | +| ----- | -------------- | +| admin | null | +| john | reporting | + +To see the group a session actually resolved to, which may come from an ACL +group rather than a direct mapping, use +[`current_resource_group()`](/docs/query/functions/meta/#current_resource_group). + +## See also + +- [ALTER USER UNSET RESOURCE GROUP](/docs/query/sql/acl/alter-user-unset-resource-group/) +- [CREATE RESOURCE GROUP](/docs/query/sql/acl/create-resource-group/) +- [Resource groups](/docs/concepts/resource-groups/) diff --git a/documentation/query/sql/acl/alter-user-unset-resource-group.md b/documentation/query/sql/acl/alter-user-unset-resource-group.md new file mode 100644 index 000000000..99452b3a6 --- /dev/null +++ b/documentation/query/sql/acl/alter-user-unset-resource-group.md @@ -0,0 +1,66 @@ +--- +title: ALTER USER UNSET RESOURCE GROUP reference +sidebar_label: UNSET RESOURCE GROUP +description: + "ALTER USER UNSET RESOURCE GROUP removes a user's direct resource group + mapping. Applies to QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +`ALTER USER ... UNSET RESOURCE GROUP` removes the user's direct +[resource group](/docs/concepts/resource-groups/) mapping. + +--- + +## Syntax + +```questdb-sql +ALTER USER userName UNSET RESOURCE GROUP; +``` + +## Description + +Removing the direct mapping does not leave the user ungoverned. Its queries fall +back to the highest-priority mapping among its ACL groups, or to `DEFAULT` when +it belongs to none that are mapped. + +The statement requires the +[`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission, and affects +queries that start after the change rather than one already running. + +Unmapping is also the prerequisite for dropping a resource group: +[`DROP RESOURCE GROUP`](/docs/query/sql/acl/drop-resource-group/) is refused +while any principal is still mapped to it. + +## Examples + +```questdb-sql +ALTER USER john UNSET RESOURCE GROUP; +``` + +`SHOW USERS` then reports `null` in the `resource_group` column: + +```questdb-sql +SELECT name, resource_group FROM (SHOW USERS); +``` + +| name | resource_group | +| ----- | -------------- | +| admin | null | +| john | null | + +`null` means no direct mapping, not that the user is ungoverned. Use +[`current_resource_group()`](/docs/query/functions/meta/#current_resource_group) +from the user's own session to see the group its queries actually resolve to. + +## See also + +- [ALTER USER SET RESOURCE GROUP](/docs/query/sql/acl/alter-user-set-resource-group/) +- [DROP RESOURCE GROUP](/docs/query/sql/acl/drop-resource-group/) +- [Resource groups](/docs/concepts/resource-groups/) diff --git a/documentation/query/sql/acl/alter-user-with-no-password.md b/documentation/query/sql/acl/alter-user-with-no-password.md new file mode 100644 index 000000000..3bf930d7f --- /dev/null +++ b/documentation/query/sql/acl/alter-user-with-no-password.md @@ -0,0 +1,56 @@ +--- +title: ALTER USER WITH NO PASSWORD reference +sidebar_label: WITH NO PASSWORD +description: + "ALTER USER WITH NO PASSWORD removes a user's password. Applies to RBAC in + QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER USER ... WITH NO PASSWORD` removes a user's password. + +--- + +## Syntax + +```questdb-sql +ALTER USER userName WITH NO PASSWORD; +``` + +## Description + +This is the only way to clear a password. `WITH PASSWORD ''` does not work, +because empty passwords are rejected. + +Removing the password does not disable the user. If it still holds a JWK or REST +token it can continue to authenticate with that; to stop access entirely, use +[`DISABLE`](/docs/query/sql/acl/alter-user-disable/) or drop its tokens with +[`DROP TOKEN`](/docs/query/sql/acl/alter-user-drop-token/). + +## Examples + +```questdb-sql +ALTER USER john WITH NO PASSWORD; +``` + +Verify with: + +```questdb-sql +SHOW USER john; +``` + +| auth_type | enabled | +| ---------- | ------- | +| Password | false | +| JWK Token | false | +| REST Token | false | + +## See also + +- [ALTER USER WITH PASSWORD](/docs/query/sql/acl/alter-user-with-password/) +- [ALTER USER DISABLE](/docs/query/sql/acl/alter-user-disable/) diff --git a/documentation/query/sql/acl/alter-user-with-password.md b/documentation/query/sql/acl/alter-user-with-password.md new file mode 100644 index 000000000..7bf424ad1 --- /dev/null +++ b/documentation/query/sql/acl/alter-user-with-password.md @@ -0,0 +1,52 @@ +--- +title: ALTER USER WITH PASSWORD reference +sidebar_label: WITH PASSWORD +description: + "ALTER USER WITH PASSWORD sets a user's password. Applies to RBAC in QuestDB + Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER USER ... WITH PASSWORD` sets a user's password. + +--- + +## Syntax + +```questdb-sql +ALTER USER userName WITH PASSWORD password; +``` + +## Description + +Setting a password replaces any existing one. Empty passwords are rejected, so +`WITH PASSWORD ''` cannot be used to clear one; use +[`WITH NO PASSWORD`](/docs/query/sql/acl/alter-user-with-no-password/) instead. + +## Examples + +```questdb-sql +ALTER USER john WITH PASSWORD '1m@re@lh@cker'; +``` + +Verify with: + +```questdb-sql +SHOW USER john; +``` + +| auth_type | enabled | +| ---------- | ------- | +| Password | true | +| JWK Token | false | +| REST Token | false | + +## See also + +- [ALTER USER WITH NO PASSWORD](/docs/query/sql/acl/alter-user-with-no-password/) +- [CREATE USER](/docs/query/sql/acl/create-user/) diff --git a/documentation/query/sql/acl/alter-user.md b/documentation/query/sql/acl/alter-user.md deleted file mode 100644 index 828980e37..000000000 --- a/documentation/query/sql/acl/alter-user.md +++ /dev/null @@ -1,195 +0,0 @@ ---- -title: ALTER USER reference -sidebar_label: ALTER USER -description: - "ALTER USER enables or disables a user, manages passwords and tokens, and sets - a per-user query memory limit. Applies to RBAC in QuestDB Enterprise." ---- - -import { EnterpriseNote } from "@site/src/components/EnterpriseNote" - - - RBAC provides fine-grained database permissions management. - - -`ALTER USER` modifies user settings. - -For full documentation of the Access Control List and Role-based Access Control, -see the [RBAC operations](/docs/security/rbac) page. - ---- - -## Syntax - -```questdb-sql title="Enable / disable" -ALTER USER userName { ENABLE | DISABLE }; -``` - -```questdb-sql title="Set or remove password" -ALTER USER userName WITH { PASSWORD password | NO PASSWORD }; -``` - -```questdb-sql title="Create token" -ALTER USER userName CREATE TOKEN TYPE - { JWK | REST WITH TTL timeUnit [REFRESH] }; -``` - -```questdb-sql title="Drop token" -ALTER USER userName DROP TOKEN TYPE - { JWK | REST [token] }; -``` - -```questdb-sql title="Set or clear memory limit" -ALTER USER userName SET MEMORY LIMIT { size | UNLIMITED }; -``` - -## Description - -- `ALTER USER username ENABLE` - enables user account. -- `ALTER USER username DISABLE` - disables user account. -- `ALTER USER username WITH PASSWORD password` - sets password for the user - account. -- `ALTER USER username WITH NO PASSWORD` - removes password for the user - account. -- `ALTER USER username CREATE TOKEN TYPE JWK` - adds Json Web Key to user - account. Returns public key (x, y) and private key. The private key is not - stored in QuestDB. -- `ALTER USER username DROP TOKEN TYPE JWK` - removes Json Web Key from user - account. -- `ALTER USER username CREATE TOKEN TYPE REST WITH TTL timeUnit REFRESH` - adds - REST token to user account. -- `ALTER USER username DROP TOKEN TYPE REST token` - removes REST token from - user account. -- `ALTER USER username SET MEMORY LIMIT size` - caps the native memory each of - the user's queries may allocate. `size` is a byte count or a size with a `K`, - `M`, or `G` suffix, such as `512M` or `2G`. -- `ALTER USER username SET MEMORY LIMIT UNLIMITED` - clears the user's own - limit. A group limit or the workload limit (`cairo.query.memory.limit.bytes`) - then applies. `SET MEMORY LIMIT 0` does the same. - -The limit applies to the user's queries on both the primary and replicas. -Setting it requires the `SET MEMORY LIMIT` permission. The built-in admin and -external (SSO/OIDC) users cannot be given a limit; the statement is rejected for -both. An external user inherits a limit from its groups instead. A set limit -takes priority over the user's groups and over the -[`cairo.query.memory.limit.bytes`](/docs/configuration/cairo-engine/#cairoquerymemorylimitbytes) -workload limit; see [memory limits](/docs/security/rbac/#memory-limits) for how -limits resolve. - -## Examples - -### Enable user - -```questdb-sql -ALTER USER john ENABLE; -``` - -### Disable user - -```questdb-sql -ALTER USER john DISABLE; -``` - -### Set password - -```questdb-sql -ALTER USER john WITH PASSWORD '1m@re@lh@cker'; -``` - -### Remove password - -```questdb-sql -ALTER USER john WITH NO PASSWORD; -``` - -Removing user's password is not possible with `WITH PASSWORD ''` because it -rejects empty passwords. - -### Add Json Web Key - -```questdb-sql -ALTER USER john CREATE TOKEN TYPE JWK; -``` - -### Remove Json Web Key - -```questdb-sql -ALTER USER john DROP TOKEN TYPE JWK; -``` - -Result of commands above can be verified with `SHOW USER`, e.g. - -```questdb-sql -SHOW USER john; -``` - -| auth_type | enabled | -| ---------- | ------- | -| Password | true | -| JWK Token | false | -| REST Token | false | - -### Add REST API token - -```questdb-sql --- generate a token with no TTL refresh -ALTER USER john CREATE TOKEN TYPE REST WITH TTL '1m'; --- generate a token with TTL refresh -ALTER USER john CREATE TOKEN TYPE REST WITH TTL '1m' REFRESH; -``` - -Here, the TTL (Time-to-Live) value should contain an integer and a unit, e.g. -`1m`. The supported units are: - -- `s` - second -- `m` - minute -- `h` - hour -- `d` - day - -The minimal allowed TTL value is 1 minute, the maximum value is 10 years (10 \* -365 days). - -The REFRESH modifier is optional. When the REFRESH modifier is specified, the -token's expiration timestamp will be refreshed on each successful -authentication. - -:::note - -When replication is used, the token will not be refreshed on successful -authentication on replicas, but only on the primary node. This makes tokens with -the REFRESH modifier meaningful for use on the primary node only. - -::: - -### Remove REST API token - -```questdb-sql --- drop single REST API token -ALTER USER john DROP TOKEN TYPE REST 'qt1cNK6s2t79f76GmTBN9k7XTWm5wwOtF7C0UBxiHGPn44'; --- drop all REST API tokens for the given user -ALTER USER john DROP TOKEN TYPE REST; -``` - -Result of commands above can be verified with `SHOW USER`, e.g. - -```questdb-sql -SHOW USER john; -``` - -| auth_type | enabled | -| ---------- | ------- | -| Password | true | -| JWK Token | false | -| REST Token | false | - -### Set memory limit - -```questdb-sql --- cap the user's queries at 512 MiB of native memory -ALTER USER john SET MEMORY LIMIT 512M; --- remove the limit -ALTER USER john SET MEMORY LIMIT UNLIMITED; -``` - -Use [`SHOW USERS`](/docs/query/sql/show/#show-users) to inspect the user's own -or inherited group limit in the `memory_limit` column. diff --git a/documentation/query/sql/acl/create-group.md b/documentation/query/sql/acl/create-group.md index 838bdfcaf..d4ac0f00a 100644 --- a/documentation/query/sql/acl/create-group.md +++ b/documentation/query/sql/acl/create-group.md @@ -38,15 +38,17 @@ OIDC or LDAP group to the new group in one statement, so members of the external group inherit its permissions on login. The group and the mapping are created atomically. `WITH EXTERNAL ALIAS` cannot be combined with `IF NOT EXISTS`. To map or unmap an existing group, use -[`ALTER GROUP`](/docs/query/sql/acl/alter-group/). For the external group -mapping flow, see the +[`ALTER GROUP ... WITH EXTERNAL ALIAS`](/docs/query/sql/acl/alter-group-with-external-alias/) +or +[`DROP EXTERNAL ALIAS`](/docs/query/sql/acl/alter-group-drop-external-alias/). +For the external group mapping flow, see the [OpenID Connect (OIDC) integration](/docs/security/oidc/#mapping-user-permissions) guide. `CREATE GROUP` cannot set a memory limit: a new group has none, and `SHOW GROUPS` reports `null` in its `memory_limit` column. To cap the native memory each query from the group's members may allocate, use -[`ALTER GROUP ... SET MEMORY LIMIT`](/docs/query/sql/acl/alter-group/#set-memory-limit) +[`ALTER GROUP ... SET MEMORY LIMIT`](/docs/query/sql/acl/alter-group-set-memory-limit/) after creating the group. The chosen name must be unique across all users (including the built-in admin), @@ -75,7 +77,11 @@ SHOW GROUPS; that yields: -| name | external_alias | memory_limit | -| -------- | --------------------------------------- | ------------ | -| admins | | null | -| analysts | CN=Analysts,OU=Users,DC=example,DC=com | null | +| name | external_alias | memory_limit | resource_group | resource_group_priority | +| -------- | -------------------------------------- | ------------ | -------------- | ----------------------- | +| admins | | null | null | null | +| analysts | CN=Analysts,OU=Users,DC=example,DC=com | null | null | null | + +`resource_group` and `resource_group_priority` are `null` until the group is +mapped with +[`ALTER GROUP ... SET RESOURCE GROUP`](/docs/query/sql/acl/alter-group-set-resource-group/). diff --git a/documentation/query/sql/acl/create-resource-group.md b/documentation/query/sql/acl/create-resource-group.md new file mode 100644 index 000000000..877fc04f3 --- /dev/null +++ b/documentation/query/sql/acl/create-resource-group.md @@ -0,0 +1,111 @@ +--- +title: CREATE RESOURCE GROUP reference +sidebar_label: CREATE RESOURCE GROUP +description: + "CREATE RESOURCE GROUP creates a query workload policy setting admission, CPU + weight and memory limits. Applies to QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +`CREATE RESOURCE GROUP` creates a query workload policy. + +For what the limits actually guarantee, see +[resource groups](/docs/concepts/resource-groups/). + +--- + +## Syntax + +```questdb-sql +CREATE RESOURCE GROUP [IF NOT EXISTS] groupName + [WITH ( parameter = value [, parameter = value ...] )]; +``` + +## Parameters + +Every parameter is optional. An unset parameter falls back to the instance +default, which is not always "unlimited": + +| Parameter | Accepted values | Unset behaviour | +| -------------------- | --------------------------------------------------------------------- | ---------------- | +| `cpu_weight` | integer, 1 to 10000 | 100 | +| `max_active_queries` | integer, 1 or more | unlimited | +| `max_queued_queries` | integer, 0 or more | unlimited | +| `queue_timeout` | whole milliseconds, or a duration such as `'15s'` or `'2m'` | 30 seconds | +| `memory_limit` | a byte size such as `'8G'`, or `0` or `UNLIMITED` for no group ceiling | no group ceiling | + +## Description + +`CREATE RESOURCE GROUP` adds a policy with no principals mapped to it. Creating +the group alone changes nothing: map a user, ACL group or service account to it +with [`ALTER USER`](/docs/query/sql/acl/alter-user-set-resource-group/), +[`ALTER GROUP`](/docs/query/sql/acl/alter-group-set-resource-group/) or +[`ALTER SERVICE ACCOUNT`](/docs/query/sql/acl/alter-service-account-set-resource-group/) before it +governs anything. + +The name must be unique across resource groups. If it is already taken the +statement fails, unless `IF NOT EXISTS` is included. + +The statement requires the +[`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission. + +Creating a second resource group engages CPU scheduling for the whole instance. +While `DEFAULT` is the only group there is no CPU slicing at all, so weights +have no effect until a second group exists. See +[when scheduling engages](/docs/concepts/resource-groups/#when-scheduling-engages). + +A group ceiling only ever lowers what a query may use. `memory_limit` is capped +by `resource.groups.process.memory.limit.bytes`, and it never raises a limit set +on the principal or by `cairo.query.memory.limit.bytes`. + +## Examples + +```questdb-sql title="A policy with no limits of its own" +CREATE RESOURCE GROUP analytics; +``` + +```questdb-sql title="Half the default CPU share, four concurrent queries, 2 GiB" +CREATE RESOURCE GROUP reporting WITH ( + cpu_weight = 50, + max_active_queries = 4, + memory_limit = '2G' +); +``` + +```questdb-sql title="A background workload that yields to everything else" +CREATE RESOURCE GROUP IF NOT EXISTS exports WITH ( + cpu_weight = 10, + max_active_queries = 1 +); +``` + +Verify with [`resource_groups()`](/docs/query/functions/meta/#resource_groups): + +```questdb-sql +SELECT name, cpu_weight, max_active_queries, memory_limit_bytes +FROM resource_groups() +ORDER BY name; +``` + +| name | cpu_weight | max_active_queries | memory_limit_bytes | +| --------- | ---------- | ------------------ | ------------------ | +| DEFAULT | 100 | null | 0 | +| analytics | 100 | null | 0 | +| exports | 10 | 1 | 0 | +| reporting | 50 | 4 | 2147483648 | + +`max_active_queries` is `null` when the group sets no admission limit, and +`memory_limit_bytes` is `0` when it sets no memory ceiling. `cpu_weight` always +reports an effective value, so a group that sets none shows `100`. + +## See also + +- [ALTER RESOURCE GROUP](/docs/query/sql/acl/alter-resource-group/) +- [DROP RESOURCE GROUP](/docs/query/sql/acl/drop-resource-group/) +- [Configure and use resource groups](/docs/operations/resource-groups/) diff --git a/documentation/query/sql/acl/create-service-account.md b/documentation/query/sql/acl/create-service-account.md index ce9230ec9..97651a2ea 100644 --- a/documentation/query/sql/acl/create-service-account.md +++ b/documentation/query/sql/acl/create-service-account.md @@ -32,7 +32,7 @@ CREATE SERVICE ACCOUNT [IF NOT EXISTS] accountName [OWNED BY ownerName]; `CREATE SERVICE ACCOUNT` cannot set a memory limit. To cap the native memory each of the service account's queries may allocate, use -[`ALTER SERVICE ACCOUNT ... SET MEMORY LIMIT`](/docs/query/sql/acl/alter-service-account/#set-memory-limit) +[`ALTER SERVICE ACCOUNT ... SET MEMORY LIMIT`](/docs/query/sql/acl/alter-service-account-set-memory-limit/) after creating it. Service accounts do not inherit group limits. The chosen name must be unique across all users (including the built-in admin), diff --git a/documentation/query/sql/acl/create-user.md b/documentation/query/sql/acl/create-user.md index 65457ccb0..354374435 100644 --- a/documentation/query/sql/acl/create-user.md +++ b/documentation/query/sql/acl/create-user.md @@ -33,9 +33,9 @@ also be set for the user. `CREATE USER` cannot set a memory limit. To cap the native memory each of the user's queries may allocate, set a limit on the user with -[`ALTER USER ... SET MEMORY LIMIT`](/docs/query/sql/acl/alter-user/#set-memory-limit), +[`ALTER USER ... SET MEMORY LIMIT`](/docs/query/sql/acl/alter-user-set-memory-limit/), or on one of its groups with -[`ALTER GROUP ... SET MEMORY LIMIT`](/docs/query/sql/acl/alter-group/#set-memory-limit). +[`ALTER GROUP ... SET MEMORY LIMIT`](/docs/query/sql/acl/alter-group-set-memory-limit/). See [memory limits](/docs/security/rbac/#memory-limits) for how the two interact. @@ -67,7 +67,7 @@ CREATE USER IF NOT EXISTS john WITH NO PASSWORD; ``` If you want to update the user's password unconditionally, you can use the -[ALTER USER](/docs/query/sql/acl/alter-user/#set-password) command. +[ALTER USER](/docs/query/sql/acl/alter-user-with-password/) command. ## Examples diff --git a/documentation/query/sql/acl/drop-resource-group.md b/documentation/query/sql/acl/drop-resource-group.md new file mode 100644 index 000000000..c898a3dbd --- /dev/null +++ b/documentation/query/sql/acl/drop-resource-group.md @@ -0,0 +1,89 @@ +--- +title: DROP RESOURCE GROUP reference +sidebar_label: DROP RESOURCE GROUP +description: + "DROP RESOURCE GROUP removes a query workload policy once no principal is + mapped to it. Applies to QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Resource groups isolate competing query workloads inside a single QuestDB + instance. + + +`DROP RESOURCE GROUP` removes a query workload policy. + +For what the limits actually guarantee, see +[resource groups](/docs/concepts/resource-groups/). + +--- + +## Syntax + +```questdb-sql +DROP RESOURCE GROUP [IF EXISTS] groupName; +``` + +## Description + +The drop is refused while any principal is still mapped to the group. Unmap them +first with `UNSET RESOURCE GROUP` on each user, ACL group or service account. +The error names one of the blocking principals: + +``` +Resource Group is assigned to an ACL entity [entity=analyst] +``` + +Without `IF EXISTS`, dropping a group that does not exist raises an error. + +The statement requires the +[`RESOURCE GROUP ADMIN`](/docs/security/rbac/#permissions) permission. + +`DEFAULT` cannot be dropped. `DROP RESOURCE GROUP DEFAULT` fails with +`built-in Resource Group cannot be dropped`. + +### What happens to queries still using it + +Once no principal is mapped, the group can be dropped even while its queries are +still running. It disappears from +[`resource_groups()`](/docs/query/functions/meta/#resource_groups) immediately, +but running and queued queries, including suspended cursors, carry on under the +settings the group had, and their memory still counts towards the process +budget. + +Recreating a group with the same name produces a new group with fresh counters. +Queries still running under the dropped group do not move to it. Map principals +to the new group to place their subsequent queries under it. + +Dropping the last group other than `DEFAULT` disengages CPU scheduling for the +instance once those queries finish. See +[when scheduling engages](/docs/concepts/resource-groups/#when-scheduling-engages). + +## Examples + +```questdb-sql title="Unmap the principals, then drop" +ALTER USER analyst UNSET RESOURCE GROUP; +ALTER GROUP analysts UNSET RESOURCE GROUP; + +DROP RESOURCE GROUP reporting; +``` + +```questdb-sql title="Drop only if present" +DROP RESOURCE GROUP IF EXISTS reporting; +``` + +To find every principal mapped to a group before dropping it: + +```questdb-sql title="List the principals blocking a drop" +SELECT name FROM (SHOW USERS) WHERE resource_group = 'reporting'; +SELECT name FROM (SHOW GROUPS) WHERE resource_group = 'reporting'; +SELECT name FROM (SHOW SERVICE ACCOUNTS) WHERE resource_group = 'reporting'; +``` + +## See also + +- [CREATE RESOURCE GROUP](/docs/query/sql/acl/create-resource-group/) +- [ALTER RESOURCE GROUP](/docs/query/sql/acl/alter-resource-group/) +- [Configure and use resource groups](/docs/operations/resource-groups/) diff --git a/documentation/query/sql/show.md b/documentation/query/sql/show.md index 29256373f..38163957d 100644 --- a/documentation/query/sql/show.md +++ b/documentation/query/sql/show.md @@ -39,16 +39,16 @@ SHOW { COLUMNS FROM tableName - `SHOW COLUMNS` returns all the columns and their metadata for the selected table. -- `SHOW CREATE DATABASE` returns DDL statements that recreate every object - in the database, one per row, ordered so dependencies come first. +- `SHOW CREATE DATABASE` returns DDL statements that recreate every object in + the database, one per row, ordered so dependencies come first. - `SHOW CREATE LIVE VIEW` returns a DDL query that allows you to recreate a live view. - `SHOW CREATE MATERIALIZED VIEW` returns a DDL query that allows you to recreate a materialized view. - `SHOW CREATE TABLE` returns a DDL query that allows you to recreate the table. - `SHOW CREATE VIEW` returns a DDL query that allows you to recreate a view. -- `SHOW GROUPS` lists all groups, or the groups a user belongs to, with each - group's external alias and memory limit (enterprise-only) +- `SHOW GROUPS` lists all groups with their external alias, memory limit and + resource group mapping, or the groups a user belongs to (enterprise-only) - `SHOW PARAMETERS` shows configuration keys and their matching `env_var_name`, their values and the source of the value - `SHOW PARTITIONS` returns the partition information for the selected table. @@ -56,13 +56,13 @@ SHOW { COLUMNS FROM tableName (enterprise-only) - `SHOW SERVER_VERSION` displays PostgreSQL compatibility version - `SHOW SERVICE ACCOUNT` displays details of a service account (enterprise-only) -- `SHOW SERVICE ACCOUNTS` lists all service accounts with their enabled flag and - memory limit, or those a user or group can assume with the grant option - (enterprise-only) +- `SHOW SERVICE ACCOUNTS` lists all service accounts with their enabled flag, + memory limit and resource group, or those a user or group can assume with the + grant option (enterprise-only) - `SHOW TABLES` returns all the tables. - `SHOW USER` shows user secret (enterprise-only) -- `SHOW USERS` lists all users with their enabled flag and memory limit - (enterprise-only) +- `SHOW USERS` lists all users with their enabled flag, memory limit and + resource group (enterprise-only) ## Examples @@ -72,6 +72,7 @@ SHOW { COLUMNS FROM tableName SHOW COLUMNS FROM trades; ``` + | column | type | indexed | indexBlockCapacity | symbolCached | symbolCapacity | symbolTableSize | designated | upsertKey | indexType | indexInclude | | --------- | --------- | ------- | ------------------ | ------------ | -------------- | --------------- | ---------- | --------- | --------- | ------------ | | symbol | SYMBOL | false | 0 | true | 256 | 42 | false | false | | | @@ -81,10 +82,10 @@ SHOW COLUMNS FROM trades; | timestamp | TIMESTAMP | false | 0 | false | 0 | 0 | true | false | | | The `indexType` column shows the index type (`POSTING`, `POSTING DELTA`, -`POSTING EF`, `BITMAP`, or empty for non-indexed columns). The -`indexInclude` column lists the names of columns included in a -[posting index's](/docs/concepts/deep-dive/posting-index/) covering -sidecar, as a comma-separated string. +`POSTING EF`, `BITMAP`, or empty for non-indexed columns). The `indexInclude` +column lists the names of columns included in a +[posting index's](/docs/concepts/deep-dive/posting-index/) covering sidecar, as +a comma-separated string. ### SHOW CREATE DATABASE @@ -115,10 +116,9 @@ all valid. Called without a clause, the statement dumps the whole database: SHOW CREATE DATABASE; ``` -The result set has a single `ddl` column with one self-contained statement -per row. Run against a database holding the -[demo](https://demo.questdb.io) tables and materialized views, it returns one -row per object: +The result set has a single `ddl` column with one self-contained statement per +row. Run against a database holding the [demo](https://demo.questdb.io) tables +and materialized views, it returns one row per object: | ddl | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | @@ -203,10 +203,9 @@ statements carry no password or token, so set these after replaying the dump. The Enterprise ACL categories are `USERS`, `GROUPS`, `SERVICE_ACCOUNTS`, and `PERMISSIONS`, grouped by the `ACL` umbrella. Each requires the matching `LIST` -or `USER DETAILS` permission, -while the schema categories need no access control permission, so a user with -only `SELECT` can still dump the structure. When access control is disabled the -command degrades to a schema-only dump. +or `USER DETAILS` permission, while the schema categories need no access control +permission, so a user with only `SELECT` can still dump the structure. When +access control is disabled the command degrades to a schema-only dump. ### SHOW CREATE LIVE VIEW @@ -242,11 +241,12 @@ materialized view, including its base table, refresh strategy, and partitioning. SHOW CREATE TABLE trades; ``` -| ddl | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| ddl | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | CREATE TABLE trades (symbol SYMBOL CAPACITY 256 CACHE, side SYMBOL CAPACITY 256 CACHE, price DOUBLE, amount DOUBLE, timestamp TIMESTAMP) timestamp(timestamp) PARTITION BY DAY WITH maxUncommittedRows=500000, o3MaxLag=600000000us; | -This is printed with formatting, so when pasted into a text editor that support formatting characters, you will see: +This is printed with formatting, so when pasted into a text editor that support +formatting characters, you will see: ```questdb-sql CREATE TABLE trades ( @@ -261,9 +261,9 @@ WITH maxUncommittedRows=500000, o3MaxLag=600000000us; #### Posting index with covering columns -When a symbol column has a posting index with `INCLUDE`, the DDL reflects -the index type and covered columns. The designated timestamp is appended -to the `INCLUDE` list automatically, so a table created with +When a symbol column has a posting index with `INCLUDE`, the DDL reflects the +index type and covered columns. The designated timestamp is appended to the +`INCLUDE` list automatically, so a table created with `INCLUDE (price, exchange)` round-trips as `INCLUDE (price, exchange, timestamp)`: @@ -318,7 +318,8 @@ policy is not shown in `SHOW CREATE TABLE`. See #### Enterprise variant -[QuestDB Enterprise](/enterprise/) will include an additional `OWNED BY` clause populated with the current user. +[QuestDB Enterprise](/enterprise/) will include an additional `OWNED BY` clause +populated with the current user. For example, @@ -336,8 +337,8 @@ OWNED BY 'admin'; This clause assigns permissions for the table to that user. -If permissions should be assigned to a different user, -please modify this clause appropriately. +If permissions should be assigned to a different user, please modify this clause +appropriately. ### SHOW CREATE VIEW @@ -349,8 +350,8 @@ SHOW CREATE VIEW my_view; | ---------------------------------------------------------------- | | CREATE VIEW 'my_view' AS (SELECT ts, symbol, price FROM trades); | -This returns the `CREATE VIEW` statement that would recreate the view, -including any `DECLARE` parameters if the view is parameterized. +This returns the `CREATE VIEW` statement that would recreate the view, including +any `DECLARE` parameters if the view is parameterized. ### SHOW GROUPS @@ -361,12 +362,21 @@ _Enterprise only._ Requires `LIST USERS`; filtering by another user requires SHOW GROUPS; ``` -| name | external_alias | memory_limit | -| ---------- | -------------- | ------------ | -| management | | 2147483648 | +| name | external_alias | memory_limit | resource_group | resource_group_priority | +| ---------- | -------------- | ------------ | -------------- | ----------------------- | +| management | | null | reporting | 10 | +| analysts | analysts-sso | 1073741824 | null | null | + +`memory_limit` is the group's own query memory limit in bytes (`1073741824` is 1 +GiB), `null` when none is set. `external_alias` is empty when the group is not +mapped to an external group. `resource_group` and `resource_group_priority` are +the group's +[resource group](/docs/query/sql/acl/alter-group-set-resource-group/) mapping, +`null` when the group is not mapped. See +[memory limits](/docs/security/rbac/#memory-limits). -Filtering by a user lists the groups that user belongs to, with the same -columns. Each row's `memory_limit` is that group's own limit: +Filtering by a user lists the groups that user belongs to, without the resource +group columns. Each row's `memory_limit` is that group's own limit: ```questdb-sql SHOW GROUPS john; @@ -374,12 +384,7 @@ SHOW GROUPS john; | name | external_alias | memory_limit | | ---------- | -------------- | ------------ | -| management | | 2147483648 | - -The `memory_limit` column is reported in bytes (`2147483648` is 2 GiB) and is -`null` when the group has no limit of its own. `external_alias` is empty when -the group is not mapped to an external group. See -[memory limits](/docs/security/rbac/#memory-limits). +| management | | null | ### SHOW PARAMETERS @@ -394,18 +399,18 @@ The output demonstrates: - `value`: the current value of the key - `value_source`: how the value is set (default, conf or env) - `sensitive`: if it is a sensitive value (passwords) -- `reloadable`: if the value can be [reloaded without a server restart](/docs/configuration/overview/#reloadable-settings) - -| property_path | env_var_name | value | value_source | sensitive | reloadable | -| -------------------------------------------- | ------------------------------------------------ | ------ | ------------ | --------- | ---------- | -| http.min.net.connection.limit | QDB_HTTP_MIN_NET_CONNECTION_LIMIT | 64 | default | false | false | -| line.http.enabled | QDB_LINE_HTTP_ENABLED | true | default | false | false | -| cairo.parquet.export.row.group.size | QDB_CAIRO_PARQUET_EXPORT_ROW_GROUP_SIZE | 100000 | default | false | false | -| http.security.interrupt.on.closed.connection | QDB_HTTP_SECURITY_INTERRUPT_ON_CLOSED_CONNECTION | true | conf | false | false | -| pg.readonly.user.enabled | QDB_PG_READONLY_USER_ENABLED | true | conf | false | true | -| pg.readonly.password | QDB_PG_READONLY_PASSWORD | **** | default | true | true | -| http.password | QDB_HTTP_PASSWORD | **** | default | true | false | - +- `reloadable`: if the value can be + [reloaded without a server restart](/docs/configuration/overview/#reloadable-settings) + +| property_path | env_var_name | value | value_source | sensitive | reloadable | +| -------------------------------------------- | ------------------------------------------------ | -------- | ------------ | --------- | ---------- | +| http.min.net.connection.limit | QDB_HTTP_MIN_NET_CONNECTION_LIMIT | 64 | default | false | false | +| line.http.enabled | QDB_LINE_HTTP_ENABLED | true | default | false | false | +| cairo.parquet.export.row.group.size | QDB_CAIRO_PARQUET_EXPORT_ROW_GROUP_SIZE | 100000 | default | false | false | +| http.security.interrupt.on.closed.connection | QDB_HTTP_SECURITY_INTERRUPT_ON_CLOSED_CONNECTION | true | conf | false | false | +| pg.readonly.user.enabled | QDB_PG_READONLY_USER_ENABLED | true | conf | false | true | +| pg.readonly.password | QDB_PG_READONLY_PASSWORD | \*\*\*\* | default | true | true | +| http.password | QDB_HTTP_PASSWORD | \*\*\*\* | default | true | false | You can optionally chain `SHOW PARAMETERS` with other clauses: @@ -440,11 +445,15 @@ See [`table_partitions()`](/docs/query/functions/meta/#table_partitions) for the full column list, including `hasParquetGenerated`, `isParquet`, `parquetFileSize`, `seqTxn`, and `isRemotelyServed`. -`isRemotelyServed` is `true` when the partition's data lives in object storage and is fetched with range reads. See [cold storage](/docs/concepts/cold-storage/) (Enterprise). +`isRemotelyServed` is `true` when the partition's data lives in object storage +and is fetched with range reads. See +[cold storage](/docs/concepts/cold-storage/) (Enterprise). :::note -`seqTxn` and `isRemotelyServed` are appended at the end of the result set. Tools that bind `SHOW PARTITIONS` columns by position rather than by name must account for the two new trailing columns. +`seqTxn` and `isRemotelyServed` are appended at the end of the result set. Tools +that bind `SHOW PARTITIONS` columns by position rather than by name must account +for the two new trailing columns. ::: @@ -541,15 +550,21 @@ requires `USER DETAILS`. SHOW SERVICE ACCOUNTS; ``` -| name | enabled | memory_limit | -| ---------- | ------- | ------------ | -| client_app | true | null | -| svc1_admin | true | 268435456 | +| name | enabled | memory_limit | resource_group | +| ---------- | ------- | ------------ | -------------- | +| client_app | true | null | null | +| svc1_admin | true | 268435456 | automation | + +`memory_limit` is the account's own query memory limit in bytes (`268435456` is +256 MiB) and `resource_group` its +[resource group](/docs/query/sql/acl/alter-service-account-set-resource-group/) mapping, +each `null` when not set. Filtering by a user or group instead lists the service accounts that principal -can assume. The result has a `grant_option` column in place of `enabled`, -showing whether the user or group may grant the assumption to others, and -`memory_limit` reports each listed service account's own limit: +can assume, without the `resource_group` column. The result has a `grant_option` +column in place of `enabled`, showing whether the user or group may grant the +assumption to others, and `memory_limit` reports each listed service account's +own limit: ```questdb-sql SHOW SERVICE ACCOUNTS john; @@ -609,10 +624,10 @@ _Enterprise only._ Requires `LIST USERS`. SHOW USERS; ``` -| name | enabled | memory_limit | -| ----- | ------- | ------------ | -| admin | true | null | -| john | true | 536870912 | +| name | enabled | memory_limit | resource_group | +| ----- | ------- | ------------ | -------------- | +| admin | true | null | null | +| john | true | 536870912 | reporting | The `memory_limit` column is reported in bytes (`536870912` is 512 MiB) and is the user's own limit or, when it has none, the most restrictive of its groups'. @@ -620,15 +635,19 @@ the user's own limit or, when it has none, the most restrictive of its groups'. (`cairo.query.memory.limit.bytes`) still applies, unlike the `memory_limit` column of [`query_activity`](/docs/query/functions/meta/#query_activity), which reports the effective limit and includes it. In `SHOW GROUPS` and -`SHOW SERVICE ACCOUNTS` above it is instead the -listed entity's own limit, since neither inherits one. See -[memory limits](/docs/security/rbac/#memory-limits). +`SHOW SERVICE ACCOUNTS` above it is instead the listed entity's own limit, since +neither inherits one. See [memory limits](/docs/security/rbac/#memory-limits). + +`resource_group` is the user's direct +[resource group](/docs/query/sql/acl/alter-user-set-resource-group/) mapping; +a user mapped only through an ACL group shows `null` here. :::note -`memory_limit` is appended as the last column of `SHOW USERS`, `SHOW GROUPS`, -and `SHOW SERVICE ACCOUNTS`, including their filtered forms. Tools that bind -these columns by position rather than by name must account for it. See +`memory_limit` is appended after the original columns of `SHOW USERS`, +`SHOW GROUPS`, and `SHOW SERVICE ACCOUNTS`, including their filtered forms, and +the unfiltered forms then append the resource group columns. Tools that bind +these columns by position rather than by name must account for them. See [upgrading](/docs/security/rbac/#memory-limit-upgrade). ::: diff --git a/documentation/query/sql/switch-role.md b/documentation/query/sql/switch-role.md index 3c9e3b002..9b764b08d 100644 --- a/documentation/query/sql/switch-role.md +++ b/documentation/query/sql/switch-role.md @@ -133,6 +133,11 @@ session that submitted it. The outcome is visible through `SWITCH STATUS`, `GET /lifecycle`, and the server log. See [Refusals and the torn state](/docs/high-availability/failover/#refusals-and-the-torn-state). +With [resource groups](/docs/concepts/resource-groups/) enabled, a promotion is +also refused when the node has not yet received the resource group catalog. The +server log names `RESOURCE_GROUP_CATALOG_UNAVAILABLE` with the reason. See +[Replication and catalog lag](/docs/concepts/resource-groups/#behaviour-under-failure-and-on-replicas). + ## Examples A planned switchover runs on two instances, in this order: diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 51e5d01ab..fb10a470f 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -655,7 +655,7 @@ ALTER GROUP groupName DROP EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=qu ``` See [`CREATE GROUP`](/docs/query/sql/acl/create-group/) and -[`ALTER GROUP`](/docs/query/sql/acl/alter-group/) for the full syntax. +[`ALTER GROUP`](/docs/query/sql/acl/alter-group-with-external-alias/) for the full syntax. External users cannot be given a query memory limit directly; `ALTER USER ... SET MEMORY LIMIT` is rejected for them. Set the limit on the diff --git a/documentation/security/rbac.md b/documentation/security/rbac.md index 62dd03447..17b226a67 100644 --- a/documentation/security/rbac.md +++ b/documentation/security/rbac.md @@ -601,9 +601,10 @@ default. Per-principal limits are available since QuestDB Enterprise 4.0.2; the server-wide [workload limits](/docs/configuration/cairo-engine/#memory-limits) they override are available since QuestDB 10.0.0. -Set a limit with [`ALTER USER`](/docs/query/sql/acl/alter-user/), -[`ALTER GROUP`](/docs/query/sql/acl/alter-group/), or -[`ALTER SERVICE ACCOUNT`](/docs/query/sql/acl/alter-service-account/): +Set a limit with +[`ALTER USER`](/docs/query/sql/acl/alter-user-set-memory-limit/), +[`ALTER GROUP`](/docs/query/sql/acl/alter-group-set-memory-limit/), or +[`ALTER SERVICE ACCOUNT`](/docs/query/sql/acl/alter-service-account-set-memory-limit/): ```questdb-sql ALTER USER john SET MEMORY LIMIT 512M; @@ -838,6 +839,7 @@ SELECT * FROM all_permissions(); | REMOVE STORAGE POLICY | Database | Table | Remove storage policies | | RENAME COLUMN | Database | Table | Column | Rename columns | | RENAME TABLE | Database | Table | Rename tables | +| RESOURCE GROUP ADMIN | Database | Manage and map resource groups, read `resource_groups()` | | RESUME WAL | Database | Table | Resume WAL processing | | SELECT | Database | Table | Column | Read data | | SET STORAGE POLICY | Database | Table | Set storage policies | @@ -900,9 +902,32 @@ replication role, is an ordinary grantable permission. ## SQL commands reference - [ADD USER](/docs/query/sql/acl/add-user/) -- [ALTER GROUP](/docs/query/sql/acl/alter-group/) -- [ALTER SERVICE ACCOUNT](/docs/query/sql/acl/alter-service-account/) -- [ALTER USER](/docs/query/sql/acl/alter-user/) +- ALTER GROUP + - [DROP EXTERNAL ALIAS](/docs/query/sql/acl/alter-group-drop-external-alias/) + - [SET MEMORY LIMIT](/docs/query/sql/acl/alter-group-set-memory-limit/) + - [SET RESOURCE GROUP](/docs/query/sql/acl/alter-group-set-resource-group/) + - [UNSET RESOURCE GROUP](/docs/query/sql/acl/alter-group-unset-resource-group/) + - [WITH EXTERNAL ALIAS](/docs/query/sql/acl/alter-group-with-external-alias/) +- ALTER SERVICE ACCOUNT + - [CREATE TOKEN](/docs/query/sql/acl/alter-service-account-create-token/) + - [DISABLE](/docs/query/sql/acl/alter-service-account-disable/) + - [DROP TOKEN](/docs/query/sql/acl/alter-service-account-drop-token/) + - [ENABLE](/docs/query/sql/acl/alter-service-account-enable/) + - [SET MEMORY LIMIT](/docs/query/sql/acl/alter-service-account-set-memory-limit/) + - [SET RESOURCE GROUP](/docs/query/sql/acl/alter-service-account-set-resource-group/) + - [UNSET RESOURCE GROUP](/docs/query/sql/acl/alter-service-account-unset-resource-group/) + - [WITH NO PASSWORD](/docs/query/sql/acl/alter-service-account-with-no-password/) + - [WITH PASSWORD](/docs/query/sql/acl/alter-service-account-with-password/) +- ALTER USER + - [CREATE TOKEN](/docs/query/sql/acl/alter-user-create-token/) + - [DISABLE](/docs/query/sql/acl/alter-user-disable/) + - [DROP TOKEN](/docs/query/sql/acl/alter-user-drop-token/) + - [ENABLE](/docs/query/sql/acl/alter-user-enable/) + - [SET MEMORY LIMIT](/docs/query/sql/acl/alter-user-set-memory-limit/) + - [SET RESOURCE GROUP](/docs/query/sql/acl/alter-user-set-resource-group/) + - [UNSET RESOURCE GROUP](/docs/query/sql/acl/alter-user-unset-resource-group/) + - [WITH NO PASSWORD](/docs/query/sql/acl/alter-user-with-no-password/) + - [WITH PASSWORD](/docs/query/sql/acl/alter-user-with-password/) - [ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/assume-service-account/) - [CREATE GROUP](/docs/query/sql/acl/create-group/) - [CREATE SERVICE ACCOUNT](/docs/query/sql/acl/create-service-account/) diff --git a/documentation/sidebars.js b/documentation/sidebars.js index e82481098..c679c4efe 100644 --- a/documentation/sidebars.js +++ b/documentation/sidebars.js @@ -288,13 +288,37 @@ module.exports = { label: "ALTER", items: [ { - id: "query/sql/acl/alter-group", - type: "doc", + type: "category", + label: "ALTER GROUP", + // Ordered by the label the sidebar renders, not the doc id + items: [ + "query/sql/acl/alter-group-drop-external-alias", // DROP EXTERNAL ALIAS + "query/sql/acl/alter-group-set-memory-limit", // SET MEMORY LIMIT + "query/sql/acl/alter-group-set-resource-group", // SET RESOURCE GROUP + "query/sql/acl/alter-group-unset-resource-group", // UNSET RESOURCE GROUP + "query/sql/acl/alter-group-with-external-alias", // WITH EXTERNAL ALIAS + ], }, { - id: "query/sql/acl/alter-service-account", + id: "query/sql/acl/alter-resource-group", type: "doc", }, + { + type: "category", + label: "ALTER SERVICE ACCOUNT", + // Ordered by the label the sidebar renders, not the doc id + items: [ + "query/sql/acl/alter-service-account-create-token", // CREATE TOKEN + "query/sql/acl/alter-service-account-disable", // DISABLE + "query/sql/acl/alter-service-account-drop-token", // DROP TOKEN + "query/sql/acl/alter-service-account-enable", // ENABLE + "query/sql/acl/alter-service-account-set-memory-limit", // SET MEMORY LIMIT + "query/sql/acl/alter-service-account-set-resource-group", // SET RESOURCE GROUP + "query/sql/acl/alter-service-account-unset-resource-group", // UNSET RESOURCE GROUP + "query/sql/acl/alter-service-account-with-no-password", // WITH NO PASSWORD + "query/sql/acl/alter-service-account-with-password", // WITH PASSWORD + ], + }, { type: "category", label: "ALTER TABLE", @@ -352,8 +376,20 @@ module.exports = { ], }, { - id: "query/sql/acl/alter-user", - type: "doc", + type: "category", + label: "ALTER USER", + // Ordered by the label the sidebar renders, not the doc id + items: [ + "query/sql/acl/alter-user-create-token", // CREATE TOKEN + "query/sql/acl/alter-user-disable", // DISABLE + "query/sql/acl/alter-user-drop-token", // DROP TOKEN + "query/sql/acl/alter-user-enable", // ENABLE + "query/sql/acl/alter-user-set-memory-limit", // SET MEMORY LIMIT + "query/sql/acl/alter-user-set-resource-group", // SET RESOURCE GROUP + "query/sql/acl/alter-user-unset-resource-group", // UNSET RESOURCE GROUP + "query/sql/acl/alter-user-with-no-password", // WITH NO PASSWORD + "query/sql/acl/alter-user-with-password", // WITH PASSWORD + ], }, "query/sql/alter-view", ], @@ -374,6 +410,10 @@ module.exports = { }, "query/sql/create-live-view", "query/sql/create-mat-view", + { + id: "query/sql/acl/create-resource-group", + type: "doc", + }, { id: "query/sql/acl/create-service-account", type: "doc", @@ -396,6 +436,10 @@ module.exports = { }, "query/sql/drop-live-view", "query/sql/drop-mat-view", + { + id: "query/sql/acl/drop-resource-group", + type: "doc", + }, { id: "query/sql/acl/drop-service-account", type: "doc", @@ -640,6 +684,11 @@ module.exports = { type: "doc", label: "Cold Storage", }, + { + id: "concepts/resource-groups", + type: "doc", + label: "Resource Groups", + }, "concepts/write-ahead-log", ], }, @@ -703,6 +752,11 @@ module.exports = { "configuration/postgres-wire-protocol", "configuration/qwp", "configuration/database-replication", + { + id: "configuration/resource-groups", + type: "doc", + label: "Resource groups", + }, "configuration/shared-workers", "configuration/storage-policy", "configuration/telemetry", @@ -794,6 +848,11 @@ module.exports = { type: "doc", label: "Cold storage", }, + { + id: "operations/resource-groups", + type: "doc", + label: "Resource groups", + }, "operations/logging-metrics", "operations/monitoring-alerting", "operations/data-retention",