Skip to content

perf(http): reuse connections across cursor windows and ramp up read_ahead - #348

Merged
kvlonge merged 4 commits into
mainfrom
http-reuse
Nov 19, 2026
Merged

kvlonge merged 4 commits into
mainfrom
http-reuse

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

HTTP sources reuse their connections across cursor windows, and read_ahead ramps up instead of sending every request at once.

  • One connection pool per load. Every sl.http job now runs on one process-wide runtime with one pooled client (a second for jobs that POST, which follow redirects differently). Before, each job, so each cursor window, started its own runtime and client, opening new connections and repeating the TLS handshake every window. OAuth tokens were already shared per connection and still are.
  • read_ahead ramps up. After a full first page, pages are requested two at a time, then four, eight and so on up to read_ahead, which keeps its meaning as the maximum. A window of a few pages now sends only a few requests past its end, rather than up to read_ahead.
  • Pages read ahead past the end finish. When paging ends normally, requests already sent run to completion so their connection goes back to the pool, instead of being aborted mid-request, which closed it. They take no further rate-limit turn and do no retry. A failure, a closed receiver or a cancelled job still aborts them.
  • Retries, rate_limit and the order rows load in are unchanged.

Numbers

A stand-in for a windowed API backfill: a local HTTPS stub (TLS, 100 ms to connect, 20 ms per request), 6 years of rows in batch_size = "30d" windows (84 windows, 18,250 rows), sl.offset(limit=50, read_ahead=20), OAuth client credentials.

0.74.0 this PR
Wall time 27.3 s 14.8 s
TLS connections 697, plus 129 abandoned handshakes 5, none abandoned
Requests 1,482 582
Pages with rows 365 365
Token requests 1 1

One large window (the same rows in one window, 80 ms per request, read_ahead=20), three runs each:

Pages 0.74.0 this PR
365 3.2–3.5 s 3.7 s
1,825 12.4 s 12.8–12.9 s
365, 10 ms to connect 3.0 s 3.2 s

The ramp costs a fixed 0.2–0.5 s at the start of each window, its four rounds of 2, 4, 8 and 16, and it doesn't grow with the window. A ×4 ramp was within noise on one large window, but sent 1,544 requests on 23 connections in the backfill, so the ramp doubles.

Testing

  • New test_http_connections.py, against a TLS stub with a CA made for the test. Both tests fail on main:
    • 13 one-day windows must take at most 4 handshakes; main took 49.
    • read_ahead=20 on a table of seven pages must send at most 11 requests; main sent 21.
  • test_http_jobs.py passes. The page-order test now compares the retried page's last answer, since the ramp sends two pages first.
  • make verify and make test-python pass on the final head.

…ahead

Every sl.http job now runs on one process-wide runtime with one pooled
client (one for jobs that POST), so the cursor windows of a backfill reuse
the connections the first window opened instead of each window starting a
runtime and client of its own and handshaking again.

read_ahead now ramps up: after a full first page, two pages are requested
at a time, then four, eight and so on up to read_ahead, so a window of a
few pages sends only a few requests past its end. Pages read ahead that are
no longer needed are no longer aborted mid-request, which closed their
pooled connection; they stop before any further wait or retry instead.
…e until made

Pages read ahead are released to finish only when paging ends normally;
a failure, a closed receiver or an aborted job aborts them. A read ahead
also checks for the stop before it sends. The shared runtime gets at least
two worker threads, and a failure to make it is no longer kept.
@kvlonge
kvlonge merged commit ab0ac82 into main Nov 19, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant