Skip to content

fix: Return immediately from a zero-length ReadFullBufferAsync/ReadFullBuffer instead of calling into the stream - #426

Merged
CurtHagenlocher merged 2 commits into
apache:mainfrom
rustyconover:fix/zero-length-read-hang-networkstream
Aug 25, 2026
Merged

fix: Return immediately from a zero-length ReadFullBufferAsync/ReadFullBuffer instead of calling into the stream#426
CurtHagenlocher merged 2 commits into
apache:mainfrom
rustyconover:fix/zero-length-read-hang-networkstream

Conversation

@rustyconover

Copy link
Copy Markdown
Contributor

What's Changed

StreamExtensions.ReadFullBufferAsync/ReadFullBuffer called stream.ReadAsync/stream.Read with a zero-length buffer unconditionally, whenever a message body is legitimately empty (e.g. a RecordBatch built from a zero-column schema, which has no buffers). Over a MemoryStream this is a harmless no-op, but over a real socket-backed NetworkStream a zero-byte read does not complete immediately — it blocks as though waiting for the peer to send more data, instead of trivially returning 0. In a lockstep/RPC-style protocol this blocks indefinitely, since the peer is itself waiting for a response before sending anything further.

Both helpers now short-circuit buffer.Length == 0 and return 0 immediately, before ever touching the stream — matching Go's io.ReadFull, which documents the same special-case for a zero-length buffer.

Full repro (real pyarrow client, real socket, strace evidence pinning the exact hang to this call) is in the linked issue.

Added StreamExtensionsTests.cs: a Stream subclass whose Read/ReadAsync throw if ever invoked confirms the zero-length fast path never touches the underlying stream, plus two tests confirming normal non-empty reads are unaffected. dotnet test test/Apache.Arrow.Tests passes in full (1874 passed, 28 skipped — unrelated Python interop tests, pre-existing).

Closes #425.

🤖 Generated with Claude Code

…llBuffer instead of calling into the stream

A RecordBatch message body is legitimately zero bytes whenever the
batch has no buffers (e.g. a batch built from a zero-column schema).
StreamExtensions.ReadFullBufferAsync/ReadFullBuffer called
stream.ReadAsync/stream.Read with that zero-length buffer
unconditionally.

Over a MemoryStream this is a harmless no-op. Over a real
socket-backed NetworkStream, a zero-byte ReadAsync/Read does not
complete immediately as it logically should -- it blocks as though
waiting for the peer to send more data (or close the connection),
instead of trivially returning 0. In a lockstep/RPC-style protocol
where the peer is itself waiting for a response before sending
anything further, this blocks indefinitely.

Both read helpers now short-circuit buffer.Length == 0 and return 0
immediately, before ever touching the stream -- matching the Go
standard library's documented behavior for io.ReadFull, which
special-cases a zero-length buffer and never issues the read at all.

Closes apache#425.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a hang in Arrow IPC stream reading when a message body is legitimately empty by ensuring StreamExtensions.ReadFullBufferAsync / ReadFullBuffer short-circuit on buffer.Length == 0 and return immediately rather than calling into Stream.ReadAsync / Stream.Read (which can block on socket-backed streams for zero-length reads).

Changes:

  • Add a zero-length fast path to ReadFullBufferAsync to avoid issuing a zero-byte ReadAsync against the underlying stream.
  • Add the same zero-length fast path to the synchronous ReadFullBuffer.
  • Add unit tests validating the zero-length fast path and confirming normal reads remain unchanged.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/Apache.Arrow/Extensions/StreamExtensions.cs Adds zero-length buffer guards to sync/async full-buffer read helpers to prevent socket read hangs.
test/Apache.Arrow.Tests/StreamExtensionsTests.cs Adds coverage for the zero-length fast path and verifies behavior for non-empty reads.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@CurtHagenlocher CurtHagenlocher left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Other than the netstandard2.0 build break, looks good.

Comment thread test/Apache.Arrow.Tests/StreamExtensionsTests.cs
…with #if NETCOREAPP

Stream.ReadAsync(Memory<byte>, CancellationToken) is only a virtual member of
Stream on netcoreapp targets; net462/net472 don't declare it, so overriding it
unconditionally in ThrowsIfReadStream broke the .NET Framework builds (CS0115).

Addresses review comment from @CurtHagenlocher on apache#426.
@CurtHagenlocher
CurtHagenlocher merged commit 07db113 into apache:main Aug 25, 2026
14 checks passed
@rustyconover
rustyconover deleted the fix/zero-length-read-hang-networkstream branch August 25, 2026 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ArrowStreamReader hangs indefinitely reading a RecordBatch with a zero-length body over a NetworkStream

3 participants