Skip to content

docs: cover the legacy shared.worker.* keys and the three-pool split - #536

Open
PedroHenrique0713 wants to merge 1 commit into
questdb:mainfrom
PedroHenrique0713:docs/separated-thread-pools
Open

docs: cover the legacy shared.worker.* keys and the three-pool split#536
PedroHenrique0713 wants to merge 1 commit into
questdb:mainfrom
PedroHenrique0713:docs/separated-thread-pools

Conversation

@PedroHenrique0713

Copy link
Copy Markdown
Contributor

Closes #229.

The split into network, query and write pools (questdb/questdb#5805) is already documented for the six new per-pool keys in shared-workers.md. What was missing is the other half: the legacy shared.worker.* keys survived the split and still drive the defaults, and two pages still described the single-pool model.

What changed

configuration/shared-workers.md

  • Documents shared.worker.count plus the four idle-tuning keys that had no entry: shared.worker.nap.threshold, shared.worker.sleep.threshold, shared.worker.sleep.timeout, shared.worker.yield.threshold. Only shared.worker.haltOnError was covered before.
  • Adds a note on the part that surprises people migrating a config: shared.worker.count is the default for each pool, not a total to divide between them. A configuration carried over from a single-pool release with shared.worker.count=8 gets 8 network + 8 query + 8 write threads, so 24.
  • Notes the per-pool thread priorities, since they explain the responsiveness behaviour.

architecture/query-engine.md — said "most tasks are handled by a shared worker pool", singular. Now points at the three pools and links to the config page.

getting-started/capacity-planning.md — the "CPU cores" section did not mention that the total thread count is the sum of three independently sized pools.

Where the values come from

Read from master rather than from the changelog:

  • PropertyKey.java L533-544 for the key names.
  • PropServerConfiguration.java L2216-2270: shared.worker.count is resolved first and then passed as the default argument to each of the three configureSharedThreadPool calls, which do getInt(properties, env, workerCountProp, sharedWorkerCount). That is what makes it per-pool rather than a total. The four idle-tuning values are read once and applied to all three pools with no per-pool override.
  • Priorities in the same block: network NORM_PRIORITY + 1, query NORM_PRIORITY, write NORM_PRIORITY - 1, matching the code comment about keeping the server responsive.
  • Checked DynamicPropServerConfiguration.java: none of these keys are reloadable, so every entry says Reloadable: no.

I also verified the existing default-sizing table on the page against cpuSpare (L1109-1124) and it is correct, so it is untouched.

The issue also lists the networking-layer and data-ingestion architecture pages; those no longer exist in the current structure, so the architecture change lands in query-engine.md.

The split into network, query and write pools (questdb/questdb#5805) is
documented for the six new per-pool keys, but the legacy shared.worker.*
keys that survived the split were not, and two pages still described the
single-pool model.

- shared-workers: document shared.worker.count and the four idle-tuning
  keys, and state that shared.worker.count is the default for *each* pool
  rather than a total, so a carried-over shared.worker.count=8 yields 24
  threads. Also note the per-pool thread priorities.
- query-engine: work is spread across three pools, not one.
- capacity-planning: the total thread count is the sum of the three pools.

Values read from PropServerConfiguration.java (the configureSharedThreadPool
calls around L2216-2270) and PropertyKey.java on master.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add documentation for separated thread pools

1 participant