Skip to content

Use identity encoding for default HTTP range reads - #2209

Open
SAY-5 wants to merge 1 commit into
fsspec:masterfrom
SAY-5:fix-http-range-identity-encoding
Open

SAY-5 wants to merge 1 commit into
fsspec:masterfrom
SAY-5:fix-http-range-identity-encoding

Conversation

@SAY-5

@SAY-5 SAY-5 commented Oct 1, 2026

Copy link
Copy Markdown

Buffered HTTP range reads can return truncated data or false EOF when size discovery requests identity encoding but the range request negotiates gzip. Use identity encoding for range requests unless the caller supplies an Accept-Encoding header on the request or session. Preserve explicit headers case-insensitively without changing caller mappings.

Refs #1748. The loopback regression uses the reported 81-byte payload and reproduces its truncated final line on unchanged master. It also covers offset reads, HEAD-to-GET metadata fallback, explicit request/session encodings, and full-read/stream behavior. The original remote endpoint was not retested.

Validation on Python 3.11.14:

  • Five regression cases fail on unchanged master and pass with the patch.
  • HTTP tests: 75 passed, 1 skipped (optional NumPy unavailable).
  • Local, memory, cached, cache and cache-thread tests: 515 passed, 51 optional-backend/platform skips.
  • Ruff 0.14.3 check and format-check, and the applicable pinned whitespace, EOF, docstring and codespell hooks pass.

The HTTP run emitted an unclosed-client-session diagnostic whose cause has not been isolated. The full Docker/backend, downstream and platform matrix were not run locally.

AI assistance: OpenAI Codex developed the patch and tests, ran the checks, and performed a separate agent review.

@SAY-5
SAY-5 force-pushed the fix-http-range-identity-encoding branch from 7508f31 to f2abfc5 Compare October 5, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant