Repository navigation
perf(http): reuse connections across cursor windows and ramp up read_ahead - #348
Merged
Merged
Conversation
…ahead Every sl.http job now runs on one process-wide runtime with one pooled client (one for jobs that POST), so the cursor windows of a backfill reuse the connections the first window opened instead of each window starting a runtime and client of its own and handshaking again. read_ahead now ramps up: after a full first page, two pages are requested at a time, then four, eight and so on up to read_ahead, so a window of a few pages sends only a few requests past its end. Pages read ahead that are no longer needed are no longer aborted mid-request, which closed their pooled connection; they stop before any further wait or retry instead.
…e until made Pages read ahead are released to finish only when paging ends normally; a failure, a closed receiver or an aborted job aborts them. A read ahead also checks for the stop before it sends. The shared runtime gets at least two worker threads, and a failure to make it is no longer kept.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HTTP sources reuse their connections across cursor windows, and
read_aheadramps up instead of sending every request at once.sl.httpjob now runs on one process-wide runtime with one pooled client (a second for jobs thatPOST, which follow redirects differently). Before, each job, so each cursor window, started its own runtime and client, opening new connections and repeating the TLS handshake every window. OAuth tokens were already shared per connection and still are.read_aheadramps up. After a full first page, pages are requested two at a time, then four, eight and so on up toread_ahead, which keeps its meaning as the maximum. A window of a few pages now sends only a few requests past its end, rather than up toread_ahead.rate_limitand the order rows load in are unchanged.Numbers
A stand-in for a windowed API backfill: a local HTTPS stub (TLS, 100 ms to connect, 20 ms per request), 6 years of rows in
batch_size = "30d"windows (84 windows, 18,250 rows),sl.offset(limit=50, read_ahead=20), OAuth client credentials.One large window (the same rows in one window, 80 ms per request,
read_ahead=20), three runs each:The ramp costs a fixed 0.2–0.5 s at the start of each window, its four rounds of 2, 4, 8 and 16, and it doesn't grow with the window. A ×4 ramp was within noise on one large window, but sent 1,544 requests on 23 connections in the backfill, so the ramp doubles.
Testing
test_http_connections.py, against a TLS stub with a CA made for the test. Both tests fail on main:read_ahead=20on a table of seven pages must send at most 11 requests; main sent 21.test_http_jobs.pypasses. The page-order test now compares the retried page's last answer, since the ramp sends two pages first.make verifyandmake test-pythonpass on the final head.