Skip to content

fix(firecrawl): trace v2 extraction batch agent and browser calls - #487

Merged
nik13 merged 2 commits into
devfrom
fix/TH-8325-v2-coverage
Nov 1, 2026
Merged

nik13 merged 2 commits into
devfrom
fix/TH-8325-v2-coverage

Conversation

@nik13

@nik13 nik13 commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Extend the maintained Firecrawl v2 instrumentor to cover extraction, batch scraping, agent jobs and browser sessions. These SDK operations previously ran without a Firecrawl span; the existing scrape/search/map/crawl entry points remain unchanged.

  • Trace the supported start/status/cancel lifecycle methods and async batch waiting, with one span per user operation rather than per internal poll.
  • Correlate supplied/returned job IDs and browser session IDs. Record available credits and completed-page counts without capturing prompts, schemas, extracted data, browser code/output or connection URLs.
  • Mark failed/cancelled blocking jobs, rejected responses and nonzero browser execution exits as errors. Preserve the SDK's original result or exception, including sync/async response differences.
  • Keep suppression, parent context, task cancellation and uninstrumentation behavior. Refresh the README with an executable environment-based setup and remove internal verification prose.

Verification

Real firecrawl-py clients exercise loopback API transports with non-loopback network denied. The full 259-test package suite passes on a clean wheel with Python 3.10 / Firecrawl 4.46.2 / fi-instrumentation-otel 1.1.0, and on a clean sdist with Python 3.13 / Firecrawl 4.54.0. Coverage includes lifecycle correlation, nested/concurrent calls, privacy, errors, cancellation, suppression and facade teardown.

OTLP checks cover JSON and protobuf, each plain and gzip, plus the README snippet through the real default exporter. A package-specific CI matrix checks wheel/sdist installs at the SDK floor and latest supported release. No live vendor credentials are used.

Compatibility and documentation

No dependency range, public instrumentation entry point or package version changes. Job IDs and credits remain absent when the SDK does not expose them; they are not inferred from private transport internals. APIs outside the README's supported-method table are unchanged.

The existing Firecrawl docs PR is future-agi/docs#998. Its expanded coverage page must follow the package release; this PR does not publish those claims early or create a duplicate docs PR.

Related: TH-8325.

E2E coverage: not applicable (this SDK repository has no product e2e harness); real-client loopback and OTLP contract tests cover the integration boundary.

🤖 Generated with Claude Code

nik13 and others added 2 commits October 31, 2026 19:35
Co-Authored-By: Claude Code <noreply@anthropic.com>
Read async batch formats from the effective ScrapeOptions argument and cover the vendor SDK contracts on the supported floor and latest versions.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nik13
nik13 merged commit 979c271 into dev Nov 1, 2026
25 checks passed
@nik13
nik13 deleted the fix/TH-8325-v2-coverage branch November 1, 2026 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant