From 4ad0f1ae6e3d0fc1de2cc862914c65d1120545a5 Mon Sep 17 00:00:00 2001 From: PedroHenrique0713 Date: Tue, 1 Sep 2026 00:02:49 -0300 Subject: [PATCH] docs: cover the legacy shared.worker.* keys and the three-pool split The split into network, query and write pools (questdb/questdb#5805) is documented for the six new per-pool keys, but the legacy shared.worker.* keys that survived the split were not, and two pages still described the single-pool model. - shared-workers: document shared.worker.count and the four idle-tuning keys, and state that shared.worker.count is the default for *each* pool rather than a total, so a carried-over shared.worker.count=8 yields 24 threads. Also note the per-pool thread priorities. - query-engine: work is spread across three pools, not one. - capacity-planning: the total thread count is the sum of the three pools. Values read from PropServerConfiguration.java (the configureSharedThreadPool calls around L2216-2270) and PropertyKey.java on master. --- documentation/architecture/query-engine.md | 3 +- documentation/configuration/shared-workers.md | 67 +++++++++++++++++++ .../getting-started/capacity-planning.md | 5 +- 3 files changed, 73 insertions(+), 2 deletions(-) diff --git a/documentation/architecture/query-engine.md b/documentation/architecture/query-engine.md index 260d35b61b..9478d07047 100644 --- a/documentation/architecture/query-engine.md +++ b/documentation/architecture/query-engine.md @@ -60,7 +60,8 @@ to process data in table page frames for better CPU use. thread. Other queries, like those involving `GROUP BY` and `SAMPLE BY`, execute a pipeline with some single-threaded stages and some multi-threaded stages to avoid slow downs when groups are unbalanced. - **Worker pools:** QuestDB allows to configure different pools for specialized functions, like -parsing incoming data, applying WAL file changes, handling PostgreSQL-Wire protocol, or responding to HTTP connections. By default, most tasks are handled by a shared worker pool. +parsing incoming data, applying WAL file changes, handling PostgreSQL-Wire protocol, or responding to HTTP connections. By default, work is spread across three shared pools: network I/O, query +execution, and writes. See [shared workers](/docs/configuration/shared-workers/). - **Query plan caching:** The system caches query plans for reuse within the same connection. (Query results are not diff --git a/documentation/configuration/shared-workers.md b/documentation/configuration/shared-workers.md index e585b57651..a188ad0206 100644 --- a/documentation/configuration/shared-workers.md +++ b/documentation/configuration/shared-workers.md @@ -26,6 +26,30 @@ server, each pool gets 62 threads. Adjust these values when you need to shift CPU capacity toward a specific workload, for example reducing the network pool and increasing the query pool on a read-heavy system. +The pools run at different thread priorities so the server stays responsive +under load: the network pool is one step above normal, the query pool is at +normal, and the write pool is one step below. + +## Settings inherited from the legacy shared pool + +Before the split into three pools, QuestDB ran a single shared pool configured +through the `shared.worker.*` keys. Those keys still work and act as the +defaults for all three pools. + +:::note + +`shared.worker.count` sets the default size of **each** pool, not a total to be +divided between them. On a configuration carried over from a single-pool +release, `shared.worker.count=8` yields 8 network threads, 8 query threads and +8 write threads, so 24 in total. Set `shared.network.worker.count`, +`shared.query.worker.count` and `shared.write.worker.count` explicitly when you +need to cap the total. + +::: + +The remaining `shared.worker.*` settings below have no per-pool equivalent: +they apply to all three pools at once. + ## shared.network.worker.affinity - **Default**: none @@ -63,6 +87,16 @@ operations such as filters and group-by. Increasing this value raises query parallelism at the expense of CPU resources available to network I/O and writes. +## shared.worker.count + +- **Default**: see table above +- **Reloadable**: no + +Default number of worker threads for each of the three pools. Overridden per +pool by `shared.network.worker.count`, `shared.query.worker.count` and +`shared.write.worker.count`. Kept for compatibility with single-pool +configurations. + ## shared.worker.haltOnError - **Default**: `false` @@ -72,6 +106,39 @@ When enabled, a worker thread stops if it encounters an unexpected error. Intended for debugging. In production, leave this disabled so that transient errors do not reduce the size of a thread pool. +## shared.worker.nap.threshold + +- **Default**: `7000` +- **Reloadable**: no + +Number of idle cycles after which a worker thread yields to other threads with +a short pause. Applies to all three pools. + +## shared.worker.sleep.threshold + +- **Default**: `10000` +- **Reloadable**: no + +Number of idle cycles after which a worker thread goes to sleep, waking on +`shared.worker.sleep.timeout`. Lower values reduce idle CPU use at the cost of +a slower response to the next task. Applies to all three pools. + +## shared.worker.sleep.timeout + +- **Default**: `10` +- **Reloadable**: no + +Time in milliseconds a sleeping worker thread waits before checking for work +again. Applies to all three pools. + +## shared.worker.yield.threshold + +- **Default**: `10` +- **Reloadable**: no + +Number of idle cycles after which a worker thread yields its CPU time slice. +Applies to all three pools. + ## shared.write.worker.affinity - **Default**: none diff --git a/documentation/getting-started/capacity-planning.md b/documentation/getting-started/capacity-planning.md index 3548de46ba..4b36b0306a 100644 --- a/documentation/getting-started/capacity-planning.md +++ b/documentation/getting-started/capacity-planning.md @@ -188,7 +188,10 @@ for other database processes to use. ### CPU cores -By default, QuestDB tries to use all available CPU cores. +By default, QuestDB tries to use all available CPU cores. Work is spread across +three pools, sized independently: network I/O, query execution, and writes. Each +one defaults to roughly the core count, so the total thread count is the sum of +the three rather than a single pool of that size. [The guide on shared worker configuration](/docs/configuration/shared-workers/) explains how to change the default settings. Assuming that the disk is not bottlenecked on IOPS, the throughput of read-only queries scales proportionally