Skip to content

Preserve queued TCP writes after peer EOF - #927

Open
radkesvat wants to merge 1 commit into
ithewei:masterfrom
radkesvat:fix/tcp-eof-write-drain
Open

radkesvat wants to merge 1 commit into
ithewei:masterfrom
radkesvat:fix/tcp-eof-write-drain

Conversation

@radkesvat

Copy link
Copy Markdown
Contributor

A TCP client can finish sending its request and still read the server’s response. When libhv has response bytes waiting in its write queue, a clean read EOF can make it close the connection before those bytes reach the client.

How the response gets cut short

  1. The client finishes sending and calls shutdown(SHUT_WR). Its read side stays open.
  2. The server gives libhv a response. The socket accepts only part of it immediately, so hio_write() queues the remaining bytes.
  3. The server reads EOF. hio_close() sees the queued output, starts the close timer and returns so the writes can drain. The socket is still registered for reads.
  4. EOF stays readable, so the event loop can read EOF again before the queue drains. The second hio_close() sees that a close is already in progress and proceeds to close the socket. Queue cleanup discards the bytes that are still waiting.

The same loss can happen on the first EOF if the server already called hio_close() while its response was queued.

Fix

Stop watching the read side when a plain TCP socket reaches EOF. If a local graceful close is already draining the queue, let it continue. Otherwise, start the usual close. Write readiness lets the remaining bytes drain, and the existing close timer still bounds how long libhv waits for a slow peer.

Regression test

tcp_eof_drain_test opens an IPv4 loopback connection, limits its socket buffers, and queues a patterned 512 KiB response. It exercises both orders described above. It checks every byte received and verifies that the server’s close callback runs once.

On Linux before the fix, both cases closed after delivering only 6,144 of 524,288 bytes. After the fix, both delivered all 524,288 bytes in Debug and Release. The Release test also passed five consecutive runs.

The Makefile test passed on Linux. A Windows x64 Release build also passed under Wine with the WSAPoll backend. With wepoll on this Wine installation, neither socket registered its read interest, so native Windows wepoll coverage is still needed.

To run the CMake test locally, use these commands.

cmake -S . -B build -DBUILD_UNITTEST=ON -DBUILD_EXAMPLES=OFF
cmake --build build --target tcp_eof_drain_test
./build/bin/tcp_eof_drain_test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant