Skip to content

feat(web-search): add Yuanbao and Bocha AI output comparisons - #275

Open
lanyue-llk wants to merge 98 commits into
mainfrom
codex/yuanbao-web-search
Open

lanyue-llk wants to merge 98 commits into
mainfrom
codex/yuanbao-web-search

Conversation

@lanyue-llk

@lanyue-llk lanyue-llk commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Add Tencent Yuanbao Standard/Lite and Bocha AI Search through the existing adapter registry. Evaluate result lists, additional source excerpts and generated answers in separate pools with their own primary metrics, participants and evidence.

Preserve native result slots and complete physical judge receipts. Collection retries default to zero. Missing Yuanbao success envelopes stop collection with unknown cost. Citation grading uses an HTML5 tree with original answer spans. It follows inline CSS validity, disclosure visibility, implied tag closures and image alternatives; deeply nested answers use an iterative traversal. Targets come from the parsed link. Explicit native citation IDs retain their own delivered source fields, including distinct excerpts for a repeated URL. Repeated copies of one ID retain the first-ranked record; URL-only links fall back to the first matching URL. Malformed capture arrays remain unavailable evidence and do not abort resumable collection or L2 admission. Reference capture retains body bindings in both modes, recovers legacy bodies from verified native responses and carries prior evidence into isolated paths when retrying.

Reference matching now verifies every candidate for each SimpleQA question from hash-bound full captures. Titles must match; when the original contains the reference answer, the mirror must contain it too. Otherwise both full articles must meet the 80% block-overlap threshold. Dataset builds and report checks replay the complete audit, and only verified mirrors enter source scores. The revised dataset retains all 831 questions; the factual quality comparison retains its 773 valid references. Search requests, timings and costs remain unchanged.

The maintained report explains the two benchmarks and three output levels before the rankings. Tables default to their primary metric and support sorting and scrolling. L1 factual results also show publisher-only scores and unverified mirror counts; the dataset page includes verification coverage for all 831 questions. Failed requests are visible beside affected methods at every output level. Release preparation avoids duplicate delivery validation and revision replay.

Scoring and dataset impact: these changes affect source-retrieval scores. The data lock selects SimpleQA revision 570dde53db18031d7cafe9806f6465e8de1ed0c1 and Web Search revision 7297b3a7a665cb7f79fd7986e78d19e72f962645. Corrections carry source hashes and paired regression controls. Software and certified-edition catalogs retain their released identities.

Validation: the control-label corrections pass 384 targeted citation, answer-judge, presentation and public scoring tests, together with 23 documentation checks and repository lint. Browser comparisons cover visibility, disclosure state, duplicate attributes, target escaping and computed display applicability and initial select-control, input and textarea labels. Paired controls run through the production scoring boundary. All 980 saved answer packets retain identical grading inputs, and the maintained report passed full replay and hosted interaction checks. CI checks cover both Python shards, installation, packaging and rendered references. Report instances, full captures and measured evidence remain outside Git.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant