Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
5936e09
test(harness): add shared OTLP harness for TH-8103 contract tests
nik13 Oct 4, 2026
32f5c9d
test(harness): accept the collector /tracer/v1/traces path
nik13 Oct 4, 2026
7586f64
feat(firecrawl): add traceAI-firecrawl instrumentor for TH-8325
nik13 Oct 4, 2026
3eaadc8
fix(harness): decode chunked OTLP bodies and record request metadata
nik13 Oct 4, 2026
4f7a537
Merge harness 3eaadc8 (chunked bodies, Receiver.requests()) into feat…
nik13 Oct 4, 2026
7b89f6a
test(firecrawl): collector contract through the shared harness
nik13 Oct 4, 2026
c5c1f53
fix(firecrawl): emit one span per call when the async crawl nests sta…
nik13 Oct 5, 2026
4c2d095
fix(firecrawl): read the crawl job id from positional arguments
nik13 Oct 5, 2026
9515703
fix(firecrawl): set firecrawl.status from the returned crawl job
nik13 Oct 5, 2026
cb016d9
fix(firecrawl): isolate instrumentation errors from the caller
nik13 Oct 5, 2026
f78c281
fix(firecrawl): make the TOOL span current during the vendor call
nik13 Oct 5, 2026
5495f98
feat(firecrawl): record cancellation, requested formats and vendor er…
nik13 Oct 5, 2026
6031bcd
fix(firecrawl): redact the API key from where firecrawl-py stores it
nik13 Oct 5, 2026
d466462
build(firecrawl): support firecrawl-py >=4.46.2,<5 and declare tested…
nik13 Oct 5, 2026
febe6e5
test(firecrawl): cover map, async parity, uninstrument, facade bindin…
nik13 Oct 5, 2026
1fff107
docs(firecrawl): document attributes, construction order, watchers an…
nik13 Oct 5, 2026
8c1e79e
style(firecrawl): wrap two long lines in the wrappers
nik13 Oct 5, 2026
8a29398
fix(firecrawl): only the blocking crawl() span takes ERROR from the j…
nik13 Oct 5, 2026
e983153
fix(firecrawl): close privacy, grouped-search and setup review gaps
nik13 Oct 6, 2026
cabf3a6
fix(firecrawl): record the search query in input.value
nik13 Oct 12, 2026
0fc99e5
docs(firecrawl): add the pip install line to the README
nik13 Oct 12, 2026
8c1289f
fix(firecrawl): align span keys with the tool tracer family
nik13 Oct 12, 2026
4c9f68e
docs(firecrawl): live vendor verification 2026-10-17
nik13 Oct 17, 2026
cbe84b0
Merge dev into feat/th-8325-firecrawl
nik13 Oct 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions python/frameworks/firecrawl/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Changelog

All notable changes to `traceAI-firecrawl` will be documented here.

## [0.1.0] - Unreleased

### Added
- `FirecrawlInstrumentor` for the Firecrawl v2 sync and async clients (`firecrawl-py>=4.46.2,<5`):
`scrape`, `search`, `map`, `crawl`, `start_crawl`, `get_crawl_status` and `cancel_crawl`, one span per call.
- `gen_ai.span.kind` is `RETRIEVER` for `search` and `TOOL` for the other calls.
- `search` records the key-redacted query (capped at 1024 characters) in `gen_ai.retrieval.query` and
`input.value`, and the result count in `firecrawl.result_count`; other calls record only the target host.
- Crawl job id, status, page count and credits; a crawl job that ends `failed` or `cancelled` sets ERROR.
- Fixed-vocabulary error metadata: no raw SDK error text or traceback is recorded.
- `TraceConfig` support (`hide_inputs`, `hide_input_text`, PII redaction, `using_session` and friends).
- `uninstrument()` also stops facades built while instrumented.
91 changes: 91 additions & 0 deletions python/frameworks/firecrawl/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# traceAI-firecrawl

OpenTelemetry instrumentation for the Firecrawl v2 client.

## v1 and v2 are different clients

`firecrawl-py` 4.x ships two clients. The v2 client is `from firecrawl import Firecrawl` (and `AsyncFirecrawl`), and it delegates `scrape`, `search`, `map`, and the crawl methods to `firecrawl.v2.client.FirecrawlClient` (and `firecrawl.v2.client_async.AsyncFirecrawlClient`). The v1 client is `FirecrawlApp`, and it is feature-frozen.

This package wraps the v2 client only. It does not wrap `FirecrawlApp`, and it does not import or disable OpenLIT. OpenLIT's `FireCrawlInstrumentor` wraps `FirecrawlApp.scrape_url` on the v1 client. A user who constructs both clients and enables both instrumentors gets two spans. Pick one.

## Installation

```bash
pip install traceAI-firecrawl
```

## Supported versions

`firecrawl-py >=4.46.2,<5` on Python 3.10, 3.11, 3.12 and 3.13. The tests run on `firecrawl-py==4.46.2`. Outside that range `instrument()` logs an error and wraps nothing.

## Usage

```python
from firecrawl import Firecrawl
from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_firecrawl import FirecrawlInstrumentor

tracer_provider = register(project_name="firecrawl", project_type=ProjectType.OBSERVE)
FirecrawlInstrumentor().instrument(tracer_provider=tracer_provider)

# Construct the client after instrument(); see "Client construction order".
client = Firecrawl(api_key="your-firecrawl-api-key")
client.scrape("https://example.com")
```

## What it records

Each call to `scrape`, `search`, `map`, `crawl`, `start_crawl`, `get_crawl_status` or `cancel_crawl`, sync or async, emits one span named `firecrawl.<method>`. `gen_ai.span.kind` is `RETRIEVER` for `search` and `TOOL` for the others, which fetch the pages they are given. A blocking `crawl()` is one span for the whole call: not one per page and not one per status poll. When the async `crawl()` calls `start_crawl()` internally, that inner call adds no span. The span is current while the SDK runs, so HTTP client spans nest under it.

| Attribute | Methods | Value |
|---|---|---|
| `server.address` | `scrape`, `map`, `crawl`, `start_crawl` | Host of the target URL. Never the path or query. |
| `gen_ai.retrieval.query`, `input.value` | `search` | The query, with the API key redacted, capped at 1024 characters. Omitted with `hide_inputs` or `hide_input_text`; shared PII redaction applies when enabled. |
| `firecrawl.result_count` | `search` | Total returned records across SDK `web`, `news`, `images` and `tools` groups. Known empty groups count as zero; unknown or malformed shapes omit the attribute. |
| `firecrawl.formats` | `scrape`, `search`, `crawl`, `start_crawl` | Bounded, deduplicated canonical SDK format names from `formats=` or `scrape_options.formats`. Unknown names, prompts and schemas are omitted. Snake-case aliases use the SDK's API spelling. |
| `firecrawl.job_id` | `crawl`, `start_crawl`, `get_crawl_status`, `cancel_crawl` | The crawl job id, positional or keyword. The start, status and cancel spans of one job share it. |
| `firecrawl.limit` | `crawl`, `start_crawl` | The `limit` argument, when passed. |
| `firecrawl.page_count` | `crawl` | Pages the job completed. |
| `firecrawl.status` | `crawl`, `get_crawl_status` | Job status from the response (`scraping`, `completed`, `failed`, `cancelled`). |
| `firecrawl.cancelled` | see "Status" | `true` when the job or the call was cancelled. |
| `firecrawl.credits_used` | any method whose result carries it | Credits the response reports. Not converted to dollars. |
| `firecrawl.error.status_code`, `firecrawl.error.code` | any, on a Firecrawl API error | HTTP status and the vendor's machine-readable error code, when present. |

`firecrawl.credits_used` caveat: for crawl jobs, firecrawl-py fills `credits_used` with `0` when the API response omits `creditsUsed`, so a recorded `0` on `crawl` or `get_crawl_status` can mean either "no credits" or "not reported".

Format metadata follows SDK 4.46.2's payload: `scrape_options` takes precedence over convenience `formats`. A `ScrapeFormats` container contributes its explicit list and serialized enabled flags, including its default `markdown=true`. The SDK does not serialize its `images` or `json` boolean flags, so those flags add no metadata; explicit format entries still do.

Search counts include `tools` because SDK 4.46.2 returns a list of `DiscoveredTool` discovery records, alongside web/news/image records. The count includes those records, not pages fetched by a tool. The prior generic `data`/`results`/`web` list fallback remains supported; no result contents are exported.

### Status

- The SDK returns a failed or cancelled crawl job without raising. On a blocking `crawl()`, a job with status `failed` sets the span to ERROR `failed`, and a job with status `cancelled` sets ERROR `cancelled` and `firecrawl.cancelled=true`. A `get_crawl_status` poll that gets an answer stays OK and records the job's state in `firecrawl.status`.
- `cancel_crawl` sets `firecrawl.cancelled` to the API's answer; the span itself is OK.
- Cancelling an asyncio task that is awaiting a Firecrawl call ends the span ERROR `cancelled` with `firecrawl.cancelled=true`.
- Any exception the SDK raises adds one safe exception event and sets ERROR with a recognized exception type (otherwise `Exception`). Exception free text and serialized tracebacks are omitted; the original exception is re-raised unchanged.

## What it never records

This package has no content-capture setting. It never records page content (markdown, HTML, screenshots, links, titles, descriptions), URL paths or query strings, JSON-format prompts or schemas, or the Firecrawl API key. The key is redacted from every attribute the package sets, including a key taken from `FIRECRAWL_API_KEY`. The search query is recorded by design. Error events contain a safe type and generic message; status descriptions contain that type or `cancelled`. Only HTTP status and recognized machine codes are retained from API errors; unknown codes are omitted.

Instrumentation errors never reach your code: if reading an argument or a result fails, the call still returns the SDK's result or raises the SDK's exception, and the span still ends.

Pass `config=TraceConfig(...)` from `fi_instrumentation.instrumentation` to `instrument()` to use shared privacy controls. `None` creates `TraceConfig()` using its `FI_*` environment settings; other objects raise `TypeError`. `hide_inputs` or `hide_input_text` omits the search query. `pii_redaction` (or `FI_PII_REDACTION=true`) applies the shared regex redactor to query text before truncation, followed by FITracer attribute masking. Other message, image and output hide flags are not applicable because no messages or page bodies are exported. Shared `suppress_tracing()` emits no Firecrawl spans, and `using_session()` supplies the shared session attribute. If API-key discovery fails, free-text attributes are omitted while safe format/count metadata remains. Error events and status use the separate safe metadata policy described above.

## Client construction order

`Firecrawl` and `AsyncFirecrawl` copy the v2 client's methods when they are constructed. Call `instrument()` before you construct them:

- A client constructed before `instrument()` is not traced.
- A client constructed while instrumented stops tracing at `uninstrument()`.

`FirecrawlClient` and `AsyncFirecrawlClient` used directly look the methods up on every call, so they follow `instrument()` and `uninstrument()` immediately.

## Watchers

`Firecrawl.watcher()` returns a `Watcher` that polls `get_crawl_status` from its own thread. Each poll is a separate root `firecrawl.get_crawl_status` span with no parent, and a poll error the watcher swallows still shows as an ERROR span. `AsyncFirecrawl.watcher()` polls in the task that iterates it, so its poll spans are children of the span current there.

## Live vendor verification (2026-10-17)

Run as documented above against the real `api.firecrawl.dev` with `firecrawl-py` 4.50.0 and `fi-instrumentation-otel` 1.1.0 on Python 3.12, exporting to Future AGI. `scrape`, `search`, `map`, `start_crawl`, `get_crawl_status`, `cancel_crawl` and a blocking `crawl` each produced one span under an AGENT parent (`fi_verify` PASS, trace `fc0dbe7b-cf70-494e-a5ed-d3b06b91e4df`). A wrong key ended the span ERROR `UnauthorizedError` (HTTP 401), and a free-plan HTTP 429 ended `start_crawl` ERROR `RateLimitError` (trace `0e189c43-63d1-4420-b7c1-42bdb92dbca5`). A scan of every stored column found no page content, URL path or query string, or API key; `FI_HIDE_INPUTS=true` removed the search query. The async client, `watcher()` and failed or cancelled job statuses were not run live and are covered by the test suite only.
30 changes: 30 additions & 0 deletions python/frameworks/firecrawl/examples/scrape.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""Run one traced Firecrawl scrape against the configured endpoint."""

import os

from firecrawl import Firecrawl
from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_firecrawl import FirecrawlInstrumentor


def main() -> None:
tracer_provider = register(
project_name="firecrawl-example",
project_type=ProjectType.OBSERVE,
batch=False,
verbose=False,
)
FirecrawlInstrumentor().instrument(tracer_provider=tracer_provider)

# firecrawl-py 4.46.2 takes api_url, confirmed from Firecrawl.__init__.
client = Firecrawl(
api_key="firecrawl-dummy-key",
api_url=os.getenv("FIRECRAWL_BASE_URL", "https://api.firecrawl.dev"),
)
client.scrape("https://example.com")
tracer_provider.force_flush()


if __name__ == "__main__":
main()
34 changes: 34 additions & 0 deletions python/frameworks/firecrawl/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
[tool.poetry]
name = "traceAI-firecrawl"
version = "0.1.0"
description = "OpenTelemetry instrumentation for the Firecrawl v2 client"
authors = ["Future AGI <no-reply@futureagi.com>"]
readme = "README.md"
packages = [
{ include = "traceai_firecrawl" }
]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: Software Development :: Libraries :: Python Modules",
]

[tool.poetry.dependencies]
python = ">=3.10,<3.14"
fi-instrumentation-otel = ">=1.1.0"
firecrawl-py = ">=4.46.2,<5"
wrapt = ">=1.0.0"

[tool.poetry.group.dev.dependencies]
pytest = ">=7.0.0"
pytest-asyncio = ">=0.21.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
125 changes: 125 additions & 0 deletions python/frameworks/firecrawl/tests/_firecrawl_fake.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
"""Loopback fake of the Firecrawl v2 HTTP API for traceAI-firecrawl tests.

The real ``firecrawl-py`` clients (sync ``requests`` and async ``httpx``) talk to
this server through ``api_url``. Nothing reaches api.firecrawl.dev, and every
key used against it is a placeholder.
"""

from __future__ import annotations

import json
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from typing import Any, Callable, Dict, List, Tuple, Union

JOB_ID = "crawl-job-1"
PAGE_BODY = "PAGE-BODY-MUST-NOT-BE-EXPORTED"
RESULT_TITLE = "RESULT-TITLE-MUST-NOT-BE-EXPORTED"

PAGE = {"markdown": PAGE_BODY, "metadata": {"sourceURL": "https://example.com/1", "statusCode": 200}}

Payload = Union[bytes, Dict[str, Any], Callable[[Dict[str, Any]], Dict[str, Any]]]
Route = Tuple[int, Payload]


def crawl_status(status: str, pages: int = 3) -> Dict[str, Any]:
"""Body of ``GET /v2/crawl/{id}`` for a job in ``status``."""
return {
"success": True,
"status": status,
"total": pages,
"completed": pages,
"creditsUsed": pages,
"expiresAt": "2030-01-01T00:00:00Z",
"data": [PAGE] * pages,
}


def default_routes() -> Dict[Tuple[str, str], Route]:
return {
("POST", "/v2/scrape"): (200, {"success": True, "data": PAGE}),
("POST", "/v2/search"): (
200,
{
"success": True,
"data": {
"web": [
{"url": "https://example.com/a", "title": RESULT_TITLE, "description": PAGE_BODY},
{"url": "https://example.com/b", "title": RESULT_TITLE, "description": PAGE_BODY},
]
},
},
),
("POST", "/v2/map"): (
200,
{
"success": True,
"links": [
{"url": "https://example.com/private/a", "title": RESULT_TITLE},
{"url": "https://example.com/private/b", "title": RESULT_TITLE},
],
},
),
("POST", "/v2/crawl"): (200, {"success": True, "id": JOB_ID, "url": "unused"}),
("GET", "/v2/crawl/" + JOB_ID): (200, crawl_status("completed")),
("DELETE", "/v2/crawl/" + JOB_ID): (200, {"success": True, "status": "cancelled"}),
}


class FakeFirecrawl:
"""Threaded loopback HTTP server that answers like the Firecrawl v2 API.

``routes`` maps ``(method, path)`` to ``(status, payload)``. A payload may be
a callable that receives the decoded request body. Unknown routes get 404.
"""

def __init__(self) -> None:
self.routes: Dict[Tuple[str, str], Route] = default_routes()
self.calls: List[Tuple[str, str]] = []
self._lock = threading.Lock()
owner = self

class Handler(BaseHTTPRequestHandler):
def _respond(self, method: str) -> None:
length = int(self.headers.get("Content-Length", "0") or 0)
body = json.loads(self.rfile.read(length) or b"{}") if length else {}
path = self.path.split("?")[0]
with owner._lock:
owner.calls.append((method, path))
status, payload = owner.routes.get(
(method, path), (404, {"success": False, "error": "not found"})
)
if callable(payload):
payload = payload(body)
data = payload if isinstance(payload, bytes) else json.dumps(payload).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)

def do_POST(self) -> None: # noqa: N802
self._respond("POST")

def do_GET(self) -> None: # noqa: N802
self._respond("GET")

def do_DELETE(self) -> None: # noqa: N802
self._respond("DELETE")

def log_message(self, format: str, *args: Any) -> None: # noqa: A002
return

self._server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
self.origin = "http://127.0.0.1:{0}".format(self._server.server_port)
self._thread = threading.Thread(target=self._server.serve_forever, args=(0.05,), daemon=True)
self._thread.start()

def calls_to(self, method: str, path: str) -> int:
with self._lock:
return self.calls.count((method, path))

def close(self) -> None:
self._server.shutdown()
self._server.server_close()
self._thread.join()
Loading
Loading