Ext proc client flow control - #12956
Conversation
…onformance - Bypass sidecar readiness checks in `request(int)` when observability mode is false and response body mode is not `GRPC`. - Ensure that when response headers, body, or trailers are skipped/disabled, they bypass request draining and stream directly downstream instead of being blocked. - Fix sidecar mocks in tests to properly call `onCompleted()` when the request stream completes, preventing background stream leaks. - Set appropriate processing modes in existing tests to match their flow control expectations under the updated logic.
…ffered use response processing mode GRPC so that buffered requests are actually sent to ext_proc instead of directly upstream in this test.
…send-mode-is-off' into ext-proc-client-flow-control # Conflicts: # xds/src/test/java/io/grpc/xds/ExternalProcessorClientInterceptorTest.java
…RequestsAreNotBuffered to verify that when response_body_mode is NONE (and observability mode is disabled), request(int) calls are passed upstream immediately without being gated by sidecar readiness or buffered.
…ying the current message size, rather than after.
…port readiness - Gated the transmission of `ClientWindowUpdate` messages by transport readiness. If the upstream client call is not ready (`isReady() == false`), the upstream window increment is kept at 0 to enforce backpressure. - Modified `trySendAccumulatedWindowUpdates()` to avoid transmitting redundant, zero-increment window updates when transport readiness prevents replenishment. A `ClientWindowUpdate` is now sent only if at least one window increment is positive (`incrementUpstream > 0 || incrementDownstream > 0`). - Updated `testClientWindowUpdateSentImmediatelyOnWindowExhaustion` to verify that no redundant window updates are sent while the transport is not ready, and that the accumulated `65536` increment is correctly flushed immediately upon calling `onReady()`. - Fixed checkstyle violations (missing line separator) in the test code.
… client interceptor - Queue mutated request body messages received from ext_proc in a new `pendingUpstreamBodyMessages` queue if the upstream transport is not ready (`isReady() == false`). - Drain the queued request body messages and forward them to the transport when the transport becomes ready (triggered in `onReady()`). - Defer request half-close if there are still body messages in the queue, executing the half-close once the queue is fully drained. - Remove the `isReady()` check when merging or sending accumulated client window updates (`accumulatedWindowUpdateSidestreamToUpstream`), ensuring window updates are always propagated. - Add a new unit test `testSidestreamToUpstreamFlowControl_QueuingAndDraining` in `ExternalProcessorClientInterceptorTest` to verify flow control queuing, transport transitions, and draining behavior.
…w-control # Conflicts: # xds/src/test/java/io/grpc/xds/ExternalProcessorClientInterceptorTest.java
…w-control # Conflicts: # xds/src/main/java/io/grpc/xds/ExternalProcessorClientInterceptor.java # xds/src/test/java/io/grpc/xds/ExternalProcessorClientInterceptorTest.java
Revert nit changes causing diffs. Revert nit changes causing diffs. Revert nit changes causing diffs.
399a16d to
98ec5be
Compare
Move the onExternalBody calls in deliverResponseBody and drainPendingMutatedResponseBodies outside of the streamLock synchronized block to prevent deadlocks. TAG=agy CONV=841033be-dc17-46b4-93ef-502e462c23da
…c interceptor - Implement fail-open queue draining for both inbound and outbound directions. - Defer transport half-close during fail-open until all outbound queues are drained. - Refactor DataPlaneClientCall.onReady to release streamLock before calling drain/update methods, avoiding deadlocks. - Consolidate and batch outbound draining in drainPendingDrainingMessages.
…rt readiness Revert the batching optimization in drainPendingUpstreamBodyMessages and drainPendingDrainingMessages back to loop-based implementations. This ensures we check super.isReady() and poll messages one-by-one under lock, correctly respecting synchronous changes in transport readiness and ensuring incremental window updates as required by tests. TAG=agy CONV=841033be-dc17-46b4-93ef-502e462c23da
…ngRequestBodyMessages Remove the redundant synchronized(streamLock) block inside drainPendingRequestBodyMessages, as its only caller already holds the lock. Annotate the method with @GuardedBy("streamLock") to document this requirement. TAG=agy CONV=841033be-dc17-46b4-93ef-502e462c23da
| .setWindowIncrementSidestreamToUpstream(incrementUpstream) | ||
| .setWindowIncrementSidestreamToDownstream(incrementDownstream) | ||
| .build()); | ||
| accumulatedWindowUpdateSidestreamToUpstream -= incrementUpstream; |
There was a problem hiding this comment.
Optional:
The side effect here seems somewhat odd. We are mutating the accumulation while building the request. This assumes that we expect a guarantee that the request will always be build and sent. If some implementation in future adds a conditional around this, we may end up in a scenario where we performed the mutation on accumulation but never sent the update.
While the currently, I can't think of a case where this'd reasonably affect anything, but I am slightly afraid of maintaining this invariant , especially when undocumented.
There was a problem hiding this comment.
I considered changing
void mergeAccumulatedWindowUpdates(ProcessingRequest.Builder requestBuilder)
to
ProcessingRequest mergeAccumulatedWindowUpdates(boolean isRequest, ByteString body, boolean isEndOfStream)
but one is still free to throw away the return value even though calling the method changes the value of the state variables. I don't seem to have a real solution to this while having readable code with methods for specific work items.
There was a problem hiding this comment.
Fair enough. Let's document it as function documentation and that should be good enough.
|
First pass at source done. Mostly okay. I have some concerns about the unbounded receiver buffering and some non-urgent concerns about growing complexity and size of the file, but the latter can potentially be addressed in the future, since it'll likely be a large refactor. Will look at the tests later. |
|
|
||
| assertThat(firstRequest.hasRequestHeaders()).isTrue(); | ||
| assertThat(firstRequest.hasFlowControlInit()).isTrue(); | ||
| assertThat(firstRequest.getFlowControlInit().getInitialWindowDownstreamToSidestream()) |
There was a problem hiding this comment.
Should we be covering all 4 windows here?
| .setWindowIncrementSidestreamToUpstream(incrementUpstream) | ||
| .setWindowIncrementSidestreamToDownstream(incrementDownstream) | ||
| .build()); | ||
| accumulatedWindowUpdateSidestreamToUpstream -= incrementUpstream; |
There was a problem hiding this comment.
Fair enough. Let's document it as function documentation and that should be good enough.
There was a problem hiding this comment.
Let's add unit tests for the fixes made in the latest commit for addressing review comments as applicable.
| () -> delegate().onMessage(bodyByteString.newInput())); | ||
| void drainSavedMessages() { | ||
| synchronized (dataPlaneClientCall.streamLock) { | ||
| while (dataPlaneClientCall.isSidecarReady() |
There was a problem hiding this comment.
IDE shows some missed coverage here.
My likely guess is that we don't have an exit criteria out of the dataPlaneClientCall.upstreamToSidestreamWindow > 0 i.e. we need behavorial tests that cover cases where the window update doesn't empty the entire queue.
Might be worth adding.
|
Looked at the tests. Things seem reasonable. Unfortunately the CI are failing. I'll be OOO for most of next weeek, but feel free to ping me once the remaining comments are addresses and the CI is passing. I'll gladly approve. |
2 participants
No description provided.