Skip to content

feat: adopt html2rss default HTTPX strategy and WebMock adapter - #1138

Merged
gildesmarais merged 3 commits into
mainfrom
experiment/httpx-default
Sep 5, 2026
Merged

feat: adopt html2rss default HTTPX strategy and WebMock adapter#1138
gildesmarais merged 3 commits into
mainfrom
experiment/httpx-default

Conversation

@gildesmarais

Copy link
Copy Markdown
Member

Summary

Adopts the new generic :default strategy powered by HTTPX from html2rss and updates WebMock/VCR and error classification accordingly.

Note

This PR is opened in Draft. Pending release of html2rss gem with HTTPX, the Gemfile / Bundler dependencies reflect the experimental sibling integration.

Highlights

  • VCR & WebMock: Configured webmock with HTTPX support in spec_helper.rb.
  • Public Strategies API: Constrained /api/v1/strategies to public strategies (default and botasaurus), filtering out internal aliases.
  • Error Classification:
    • Added HTTPX::ConnectionError, HTTPX::TLSError, and HTTPX::TimeoutError to network error classification.
    • Added explicit mapping of Html2rss::RequestService::PrivateNetworkDenied to 403 FORBIDDEN in ErrorClassifier::SPECIAL_DECISIONS.
  • Review Findings Addressed:
    • Restored strict public strategy list contract and tests.
    • Added specs verifying HTTPX network/timeout classification and SSRF 403 response.

Verification

  • bundle exec rspec passed (345 examples, 0 failures).
  • bundle exec rubocop clean (122 files inspected, 0 offenses).

- Point Gemfile to sibling html2rss path for httpx experiment
- Configure VCR to hook into :webmock with HTTPX adapter in spec_helper
- Update strategy display names for default, faraday, and httpx
- Explicitly require net/http in error_classifier for Net::OpenTimeout/ReadTimeout
@gildesmarais
gildesmarais marked this pull request as ready for review September 5, 2026 23:24
@gildesmarais
gildesmarais merged commit 6b28b57 into main Sep 5, 2026
15 checks passed
@gildesmarais
gildesmarais deleted the experiment/httpx-default branch September 5, 2026 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant