Skip to content

Optimize protocol round trips for commands and queries - #1356

Open
davidlar wants to merge 5 commits into
typelevel:mainfrom
davidlar:round-trip-reduction
Open

Optimize protocol round trips for commands and queries#1356
davidlar wants to merge 5 commits into
typelevel:mainfrom
davidlar:round-trip-reduction

Conversation

@davidlar

@davidlar davidlar commented Sep 1, 2026

Copy link
Copy Markdown

We run a service whose database is in another country. At ~60 ms round-trip time the driver's message count is the cost of a small statement — execution is single-digit milliseconds, so wall-clock is exchanges × RTT.
Parse+Describe and Bind+Execute are each already pipelined behind one Flush (#1059, #1061),
but the exchanges between those groups, plus the trailing Sync and portal Close, were separate
waits.

Exchanges per operation

Operation before after
Parameterized command 3 1
…first use (cold cache) 4 2
…inside an explicit transaction 3 2
execute(query)(args) 4 1
…inside an explicit transaction 4 2
unique / option 3 1
stream 4 4 — unchanged, deliberately

testOnly tests.simulation.ExchangeCountTest prints this table and asserts every number; it needs no
database.

Results

In production since 31 August — one day so far, but under heavy load against several databases
(Postgres 14 and 16) in different datacentres, on both the ~60 ms link and low-latency ones, with no
protocol errors or session problems. On a transaction that reads and then writes we see about 40%
lower latency
, consistent with 4→2 exchanges for the reads and 3→2 for the writes at 60 ms. That is
the conservative case: inside a transaction the portal outlives Sync, so the Close stays and each
statement saves one round trip rather than two.

With latency injected locally, medians of three runs at 40 ms RTT: command 134→44 ms, unique
128→45, option 127→47, execute(query)(args) 170→48, and stream — the control, whose
exchange count does not change — 128→129. At 100 ms the rows scale as exchanges × RTT predicts. On loopback the difference is within noise, so this buys little if your database is next door.

What changed

  • Commands send Sync in the same write as Bind+Execute. A command never pages, so it has no
    reason to hold the portal open between Executes. Commands not completing until next query/command is run #210 requires that Sync be sent, not that it
    be sent after the completion read.
  • The portal Close is skipped when the backend has already dropped it: Sync ends the implicit
    transaction and takes the portal with it, and ReadyForQuery already reports which case applies.
  • One-shot queries (execute(query)(args), unique, option) get the same treatment — they fetch
    once by construction, so no second Execute can be stranded by ending the transaction.
  • stream, cursor and pipe are untouched: paging needs the portal to outlive one Execute, which
    is why Flush and not Sync is correct there.

Points worth a close look

  1. copy-in — documented: the backend ignores Flush/Sync in copy-in mode, so the pre-sent
    Sync is swallowed and that branch sends its own.
  2. copy-out — not documented either way. The argument is that the backend performs the whole copy
    inside its handling of Execute and only then reads the next frontend message, so it reaches the
    pre-sent Sync afterwards and replies ReadyForQuery, which must be consumed. CopyOutTest
    corroborates it.
  3. suspended portalsunique/option ask for 2 rows to tell whether more exist. When more do,
    the portal suspends rather than completing, and its ReadyForQuery still has to be read or the
    next operation mistakes it for its own. That desynchronises the session rather than failing a test,
    so the tests assert that a follow-up operation still succeeds.

Production covers the ordinary paths under concurrency, but not copy-out, and reaches (3) only when
unique/option errors on extra rows.

Review

Five commits, each self-contained and green on its own — the first is only the measurement harness,
and 2–4 are independent protocol changes. Happy to split into separate PRs if you would prefer.

PreparedQuery.fetchAll is the one real API addition: cursor(args).use(_.fetch(Int.MaxValue)) is
the obvious way to ask for all rows and costs four exchanges where one will do. Happy to drop that
commit if you would rather not grow the API; nothing else depends on it.

An exchange is a send->receive transition, which on a high-latency link
dominates the time a statement takes. Needs no database.
A command never pages, so it has no reason to keep the portal open between
Executes and can end the exchange in the same write. Issue typelevel#210 requires that
Sync be sent, not that it be sent after the completion read.
Sync ends the implicit transaction and takes the portal with it, so outside an
explicit transaction the Close is a round trip spent on a no-op. ReadyForQuery
already reports which case applies.
execute(query)(args), unique and option fetch once, so they have no use for a
portal that outlives the fetch. stream still pages and is unchanged.
cursor(args).use(_.fetch(Int.MaxValue)) is the obvious way to ask for all rows
and costs four exchanges where one will do. Also lets Session.Impl route
execute(query)(args) through it, removing two overrides.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant