Skip to content

feat(webfetch): add Kimi, Z.AI and remote Claude fetch adapters - #292

Draft
Maisiqi wants to merge 12 commits into
mainfrom
codex/kimi-zai-claude-webfetch
Draft

Maisiqi wants to merge 12 commits into
mainfrom
codex/kimi-zai-claude-webfetch

Conversation

@Maisiqi

@Maisiqi Maisiqi commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add opt-in Kimi international Fetch, Z.AI international Web Reader, and remote-only Claude Web Fetch adapters to the maintained provider pipeline. Claude calls run through the declared SSH host with a shared deadline and no local HTTP fallback. The worker verifies its address and source hashes before reading the remote repository credential, uses the shared dotenv parser and HTTP client, and refuses redirects. Malformed responses remain inspectable failure evidence rather than aborting artifact persistence. Only native tool text supplies Claude's scored answer.

Pin the current Hugging Face input commit and restore published scope annotations only after verifying source-task hashes, bound reference text and exact public acceptance contracts. Restored inputs are explicitly derived; they do not claim the private export's byte hash. Scoring and frozen benchmark editions remain unchanged.

Extend the existing saved-matrix report owner with explicit post-score public evidence views. Preserve original source bytes and scores, bind every public view to its original, require unchanged task acceptance after answer redaction, and expose native responses and genuine diagnostic logs. Handle nested JSON and URL keys without dropping evidence. Add data-driven failure distributions, native-error diagnoses, remote account requirements and sourced product pricing. No parallel renderer, measured output or report-instance profile is added to Git.

Validation

  • Affected adapter, security, account, UTF-8, saved evidence, privacy, pricing, registry, interpretation and input-binding checks: 679 passed, 203 subtests.
  • Additional native selection, route, historical policy, report packaging and evidence-display checks: 525 passed, one skipped; the source-hash snapshot was then updated and its focused regression passed.
  • Focused route, privacy and interpretation checks: 77 passed, 56 subtests.
  • Further saved-evidence, privacy and language checks: 384 passed, 56 subtests; the two affected synthetic report snapshots and bilingual pricing regression also pass.
  • Targeted Ruff and whitespace checks passed.
  • The remote frozen runtime imports and source hashes were verified without an API call; the remote credential file remains mode 0600.

All three complete 530-task collections are finished and their 2,900 physical calls are retained outside Git. Post-collection fixes add no provider calls and do not rewrite historical collection-source bindings. The local Chinese report is built and its top-level pages have been inspected. The complete source check verifies 11,631 original source files, 37,479 delivered files and 2,134 canonical HTML routes; all 1,590 logical results are present and their scores are unchanged. The report remains a saved-results diagnostic rather than a benchmark certification. No independent result review or publication is claimed. No version bump or merge is requested.

@lexmount-ocr lexmount-ocr Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Review complete · 22a4ab1

This review found 3 issue(s), including 3 blocking issue(s).

payload = json.loads(response.body)
except ValueError:
payload = {}
blocks = payload.get('content', []) if isinstance(payload, dict) else []

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
P2: Validate decoded response shapes before accessing nested fields and retain malformed responses as classified failures. A JSON null or list reaches payload.get('model') later, while {"content":null}, non-object content entries, or a null tool input fail earlier in block processing. These exceptions escape before _raw_response_b64 is returned, so the caller records an exception without the native response needed for diagnosis. Normalize/check the envelope, content list, blocks and inputs in both this adapter and the Claude projection, and return an empty failed result with the original raw bytes for invalid shapes.

Comment on lines +155 to +157
blocks = payload.get('content', []) if isinstance(payload, dict) else []
calls = [b for b in blocks if b.get('type') == 'server_tool_use']
wrong_url = any(b.get('name') != 'web_fetch' or b.get('input', {}).get('url') != case.url for b in calls)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
P2: Validate decoded response shapes before accessing nested fields and retain malformed responses as classified failures. A JSON null or list reaches payload.get('model') later, while {"content":null}, non-object content entries, or a null tool input fail earlier in block processing. These exceptions escape before _raw_response_b64 is returned, so the caller records an exception without the native response needed for diagnosis. Normalize/check the envelope, content list, blocks and inputs in both this adapter and the Claude projection, and return an empty failed result with the original raw bytes for invalid shapes.

data=json.dumps(envelope['body']).encode(), headers={
'x-api-key': key, 'anthropic-version': '2023-06-01', 'Content-Type': 'application/json'})
try:
response = urllib.request.urlopen(request, timeout=budget)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security · high
P1: Disable redirects for this credential-bearing request. Python 3.11's default urllib redirect handler follows POST 301/302/303 responses and copies custom headers, including x-api-key, to the redirected request—even across origins or to HTTP. Thus an API redirect can send the remote repository credential outside the fixed HTTPS endpoint despite the preceding endpoint check. Use an opener that rejects redirects (or explicitly restricts every redirect to the approved HTTPS origin) and add a redirect test asserting that a second origin never receives the key.

@lexmount-ocr lexmount-ocr Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Re-review complete · 87cb08c

This review found 1 issue(s), including 1 blocking issue(s).

data=json.dumps(envelope['body']).encode(), headers={
'x-api-key': key, 'anthropic-version': '2023-06-01', 'Content-Type': 'application/json'})
try:
response = urllib.request.urlopen(request, timeout=budget)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security · high
P1 — Disable redirects for this credential-bearing request. Python 3.11's default urllib redirect handler follows a POST's 301/302/303 response and copies custom headers, including x-api-key, to the redirected request, even for another host or an HTTP destination. Thus an API redirect can send the remote repository credential outside the fixed HTTPS endpoint; the initial URL/base-URL checks and subsequent response-body scan do not prevent that transfer. Use an opener that rejects redirects and retain the 3xx response as a failure. Add a redirect regression test asserting that no second request carrying the key is made. This blocks on the credential-boundary risk.

@lexmount-ocr lexmount-ocr Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Re-review complete · d376038

This review found 1 issue(s), including 1 blocking issue(s).

data=json.dumps(envelope['body']).encode(), headers={
'x-api-key': key, 'anthropic-version': '2023-06-01', 'Content-Type': 'application/json'})
try:
response = urllib.request.urlopen(request, timeout=budget)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security · high
P1 (blocking): Reject redirects before sending another request. The standalone worker uses urllib's default redirect handler, which follows a POST's 301/302/303 response and copies custom headers, including x-api-key, to the redirected request—even across origins or to HTTP. Thus an API response with an external Location can send the remote repository credential to that destination despite the fixed initial endpoint check. No redirect restriction is installed by this worker or its launcher, so the previously identified credential-boundary issue remains. Use an explicit no-redirect opener and retain the 3xx as failure evidence; add a redirect regression test asserting that no second request carries the key.

@lexmount-ocr lexmount-ocr Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Review complete · b2dd3a8

This review found 2 issue(s), including 2 blocking issue(s).

if hashlib.sha256(raw).hexdigest() != binding['sha256']:
raise ValueError('Public view original changed')
identifier = hashlib.sha256(binding['path'].encode()).hexdigest()
extension = '.jsonl' if role == 'diagnostic_log' else '.txt'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
P2: Public-view generation assumes formats that the evidence producers do not guarantee, so enabling it can abort an otherwise valid saved-matrix build. CallDiagnostics.finish() appends verbatim OpenClaw/Moli/native stderr after JSONL events (including non-JSON separator lines), but every .stderr.log here becomes .jsonl and is rejected by project_evidence_view. Also, artifact_bindings() includes delivered -file-*.bin files and raw stdout, while this function projects every artifacts/ binding as UTF-8 scored text; non-UTF-8 binary evidence fails at decoding. Select projection by the actual evidence type/format, support mixed diagnostic logs without discarding their content, and retain binary attachments under their original verified bindings rather than treating them as scored text. Add end-to-end public-view tests using a real mixed diagnostic log and a binary attachment.

Comment on lines +59 to +61
response = request('https://api.anthropic.com/v1/messages', method='POST',
json_body=envelope['body'], headers={'x-api-key': key, 'anthropic-version': '2023-06-01'},
timeout=budget, max_bytes=int(envelope['max_bytes']), allow_redirects=False)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security · medium
P2: Credential-bearing exceptions bypass the response filter. The shared dotenv parser accepts escaped \n in double-quoted values; if a pasted ANTHROPIC_API_KEY contains one, Python's HTTP client raises ValueError("Invalid header value ...") containing the complete header value. This uncaught exception is printed to remote stderr, which _remote_request() transfers and records in ssh_process_completed and its RuntimeError. Thus a configuration error can copy the remote-only credential into local evidence, despite the response-body check below. Reject invalid credential header characters with a value-free error before the request, and ensure exceptions crossing this boundary cannot include the key (including chained tracebacks). Add a regression using an escaped-newline credential and assert it never appears in transferred stderr or local diagnostics.

@lexmount-ocr lexmount-ocr Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Re-review complete · 2d9b065

This review found 2 issue(s), including 2 blocking issue(s).

⚠️ 1 warning(s) occurred during review.


⚠️ Warnings:

  • src/harness/web_fetch/publication_privacy.py (token_budget_reached): skipped round 2 of group "src/harness/web_fetch/publication_privacy.py,src/harness/web_fetch/report.py,src/harness/web_fetch/report_saved_matrix.py": used 531251 tokens exceeds budget 500000

Comment on lines +1626 to +1628
bindings = [(dict(path=name, sha256=digest), 'scored_answer')
for name, digest in artifact_bindings(record)
if Path(name).parts[0] == 'artifacts']

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
P2 — Restrict text projection to text evidence. artifact_bindings() also includes file_evaluation.files[*].artifact_path, whose downloaded bytes are written under artifacts as *-file-*.bin by the runner. This comprehension labels those binaries as scored answers and sends them through project_evidence_view()'s unconditional UTF-8 decode. A valid PDF/image download containing non-UTF-8 bytes consequently prevents any saved matrix containing that attempt from building when public views are enabled. Preserve verified binary attachments unchanged (subject to the existing publication checks), and create textual views only for artifacts with a text contract.

if hashlib.sha256(raw).hexdigest() != binding['sha256']:
raise ValueError('Public view original changed')
identifier = hashlib.sha256(binding['path'].encode()).hexdigest()
extension = '.jsonl' if role == 'diagnostic_log' else '.txt'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
P2 — Do not declare every diagnostic log to be JSONL. CallDiagnostics.finish() appends verbatim native logs after non-JSON separator lines (diagnostics.py:36–47), and the headless provider also retains plain process stderr. The runner persists these valid logs as .stderr.log, so this branch renames them to .jsonl and project_evidence_view() raises Declared JSONL diagnostic evidence is not JSONL. Enabling public evidence views therefore aborts saved-matrix generation for supported attempts with native diagnostics. Detect actual JSONL content or preserve a text/mixed-log projection rather than inferring the format from the diagnostic role.

@lexmount-ocr lexmount-ocr Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Re-review complete · 6a742a6

No findings in this incremental review.

@lexmount-ocr lexmount-ocr Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Re-review complete · 83fcb3e

This review found 1 issue(s), including 1 blocking issue(s).

if item['provider']==name and item['family']==report_family(example_case)
for setting in item['settings'] if setting.get('input_class')==input_class and setting.get('request')]
if len(verified)==1:
s={**s,'request':verified[0],'notes':'按原始调用核验的请求模板;<input-url> 表示该题网址。'+s['notes']}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintainability · medium
P2 — Move the new request-template note into the language catalog. CONTRIBUTING.md#text-language requires Chinese report text to come from language catalogs, but this branch embeds it directly in the renderer alongside a separately maintained English literal. Add a localized message to the existing report-method catalog and read it through messages / english_messages, preserving the current bilingual output.

@lexmount-ocr lexmount-ocr Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Re-review complete · b054b0f

No findings in this incremental review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant