Skip to content

refactor(net)!: separate range scheduling from HTTP - #3135

Open
nostalume wants to merge 6 commits into
OpenListTeam:mainfrom
nostalume:refactor/domain5-range-scheduler
Open

nostalume wants to merge 6 commits into
OpenListTeam:mainfrom
nostalume:refactor/domain5-range-scheduler

Conversation

@nostalume

@nostalume nostalume commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary / 摘要

  • Separate byte scheduling from HTTP transport interpretation. The bounded downloader now accepts a part opener; the URL adapter alone handles HTTP request ranges, status/size validation, and retry classification, while native range sources open directly rather than being wrapped as synthetic http.Response values. This removes the old HTTP-shaped native adapter and yields a net 38-line production reduction versus the refactor(link)!: consolidate borrowing and terminal ownership #3131 head. No throughput gain is claimed.

  • Include the downloader completion/cancellation receipt repairs and the BufferStore lifetime synchronization needed for race-safe delivery. The high-concurrency test now checks delivered bytes and released slots instead of racing a worker-side request counter.

  • Depends on refactor(link)!: consolidate borrowing and terminal ownership #3131. This branch descends from its exact reviewed head 4b6315c6. The upstream PR's default diff temporarily contains refactor(link)!: consolidate borrowing and terminal ownership #3131; review only this Domain 5 layer. Parallel review is welcome; merge only after refactor(link)!: consolidate borrowing and terminal ownership #3131. Once it lands, we will update this same PR against landed main and revalidate the final diff and CI.

  • Source compatibility: internal Go callers using HttpRequestParams, HttpRequestFunc, DefaultHttpRequestFunc, GetRangeReaderHttpRequestFunc, or the prior Downloader.Download signature must migrate. No configuration or storage format changes are intended.

  • This PR has breaking changes.
    / 此 PR 包含破坏性变更。

  • This PR changes public API, config, storage format, or migration behavior.
    / 此 PR 修改了公开 API、配置、存储格式或迁移行为。

  • This PR requires corresponding changes in related repositories.
    / 此 PR 需要关联仓库同步修改。

Related repository PRs / 关联仓库 PR: None known.

Related Issues / 关联 Issue

Relates to #3098.

Testing / 测试

Windows, Go 1.27.1, final tree 11c220f7:

  • go test ./internal/net -skip '^TestNewOSSClientUsesEnvironmentHTTPSProxy$' — passed. The unfiltered package run failed on this unchanged, environment-sensitive OSS transport-type assertion.

  • go test ./internal/stream ./internal/op ./server/handles ./server/s3 — passed.

  • go vet ./internal/net ./internal/stream ./internal/hybrid_cache and go build ./... — passed; Go printed a non-fatal module stat-cache access warning in this restricted environment.

  • WSL Tumbleweed, Go 1.27.1: go test -race ./internal/net -run '^(TestDownload|TestHighConcurrency)' -count=10 and go test -race ./internal/net -skip TestNewOSSClientUsesEnvironmentHTTPSProxy plus go test -race ./internal/stream — passed after the cache and test corrections.

  • All six changed committed Go blobs are gofmt-clean; git diff --check passed. The head tree was rechecked after commit-message cleanup and matches the tree exercised by the tests.

  • Full go test ./... was not run in this audit and is not claimed green.

  • go test ./...

  • Manual test / 手动测试:

Checklist / 检查清单

  • I have read CONTRIBUTING.
    / 我已阅读 CONTRIBUTING。
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
    / 我确认此贡献符合仓库许可证、贡献规范和行为准则。
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
    / 我已按适用情况使用 gofmt、go fmt 或 prettier 格式化变更代码。
  • I have requested review from relevant maintainers or code owners where applicable.
    / 我已在适用情况下请求相关维护者或代码所有者审查。

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.
    / 此 PR 包含 AI 辅助内容。

Tools used / 使用工具:

  • Codex

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Refactoring / 重构

  • Documentation / 文档

  • Tests / 测试

  • Translation / 翻译

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.
    / 我已审核并验证此 PR 中的所有 AI 辅助内容。

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.
    / 我已确保所有 AI 辅助提交都包含 Co-Authored-By 归属信息。

  • I can reproduce all AI-assisted content included in this PR without any AI tools.
    / 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。

nostalume and others added 6 commits September 24, 2026 19:14
- Return isolated link borrowers from regular and archive caches while removing duplicate lifecycle policy.

- Transfer link cleanup to proxy and seekable-stream consumers, with bodies released before links.

- Remove redundant wrappers and caller-side cleanup across alias, chunk, strm, and transfer paths.

- Cover cache isolation, stream close order, proxy range behavior, and proxy cleanup.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
- Treat delivered bytes as the completion receipt when a consumer closes after a full read.

- Use cancellation to stop workers without closing shared task queues during unwind.

- Make the read receipt atomic and retain cancellation regressions.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
- Keep HTTP status, range validation, and retry classification inside the URL adapter.

- Open native ranges directly through the transport-neutral scheduler and remove the synthetic HTTP response path.

- Retain focused cancellation and range behavior tests without the exploratory benchmark.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
- Serialize growth and closure against reads, writes, and size observations.

- Remove the test-only append path and clear owned blocks on close.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
- Verify delivered bytes and released slots without racing worker-side request counters.

- Remove a redundant fixed-length branch from the high-concurrency case.

Co-authored-by: Codex <267193182+codex@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant