Replace fictional insurance claim with public FEMA appeal

.coderabbit.yaml

@@ -0,0 +1,20 @@
+# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
+
+reviews:
+  auto_review:
+    enabled: true
+    drafts: false
+  path_filters:
+    - "examples/**"
+    - "!examples/**/data/**"
+    - "!examples/**/fixtures/**"
+    - "examples/**/fixtures/SOURCES.md"
+    - "!examples/**/verified-run/**"
+    - "examples/**/verified-run/README.md"
+    - "!examples/**/uv.lock"
+  path_instructions:
+    - path: "examples/**"
+      instructions: |
+        Review runnable source, tests, configuration, and documentation.
+        Check that commands match the implementation and that tests cover failure paths.
+        Flag client-side model-name translation and undocumented synthetic results.

examples/README.md

@@ -24,7 +24,7 @@ service keys.
 | [Vision-first document RAG](./vision-doc-rag) | Retrieving and answering questions over a multi-tenant page corpus by looking at page images (including scanned drawings) with OCR kept out of the score path | `encode`, `chat/completions`, `score` (optional) | GPU SIE deployment required: ColQwen2.5 retriever + Qwen3.5-4B answer model (runs on the generation bundle) | Runnable demo |
 | [Multi-model contract review with the OpenAI Agents SDK](./contract-review-agent) | Running an OpenAI Agents SDK agent whose every model call (triage, orchestration, vision, OCR, embeddings, rerank, entity extraction, text-to-SQL, reasoning, and a safety guardrail) is served by one SIE cluster, each step on the right catalog model, with per-model observability | `chat/completions`, `encode`, `score`, `extract` | GPU SIE deployment required; standalone `uv` project; real contracts fetched from CUAD (CC BY 4.0) | Runnable demo |
 | [Turn difficult PDFs into Markdown](./document-to-markdown) | Preserving tables, reading order, headings, and form labels across real financial, academic, and government PDFs | `extract` | SIE endpoint with `docling`; standalone `uv` project; source PDFs fetched at run time | Runnable evaluation example |
-| [Review a flood claim packet](./insurance-claims-agent) | Finding an unsigned form and a $400 evidence mismatch across a policy, estimate, inventory, and damage photograph | `extract`, `score`, `chat/completions` | GPU SIE deployment; standalone `uv` project; public FEMA documents and public-domain photograph fetched at run time | Runnable agent example |
+| [Review a published flood-insurance appeal](./insurance-claims-agent) | Separating FEMA's covered stone-removal scope from excluded barge, handling, disposal, and yard costs | `extract`, `score`, `chat/completions` | GPU SIE deployment; standalone `uv` project; bundled public FEMA appeal and policy | Runnable agent example |
 
 For docs publishing, lead with the quickest runnable demos, then use the
 benchmark and evaluation examples for deeper technical users.

examples/insurance-claims-agent/README.md

@@ -1,36 +1,41 @@
-# Review a flood claim packet through one SIE cluster
+# Review a public flood-insurance appeal through one SIE cluster
 
-This example reviews a realistic claim packet built from an official FEMA proof
-of loss, the Standard Flood Insurance Policy, a public-domain damage photograph,
-and a fictional repair estimate.
+This example reviews FEMA Flood Insurance Appeal Decision B8 and the controlling
+Standard Flood Insurance Policy. The appeal is a public, redacted record. It
+concerns a July 2019 Lake Ontario flood and a disputed request to remove stones
+from underneath an insured building.
 
-The packet contains two deliberate problems. The proof of loss is unsigned. Its
-net claim total is $81,060, while the attached estimate and inventory support
-$80,660 after deductibles.
+The record is useful because the outcome turns on scope. FEMA directed the
+insurer to cover removal of 12 to 15 cubic yards of flood-borne stones from
+underneath the building to its perimeter. Barge transport, handling, disposal,
+and debris removal from the yard remain outside that covered scope.
 
 ## What SIE does
 
 | Stage | Model | Result |
 |---|---|---|
-| Parse the form, estimate, and policy | `docling` | Markdown with form labels, tables, and policy text |
-| Read the claim identity | `fastino/gliner2-large-v1` | Typed name, policy number, loss date, and property address |
-| Retrieve controlling policy language | `BAAI/bge-reranker-v2-m3` | Ranked passages about proof of loss and supporting records |
-| Detect visible damage categories | `IDEA-Research/grounding-dino-tiny` | Labels, confidence scores, and boxes |
-| Produce the evidence review | `Qwen/Qwen3.5-4B:no-spec` | JSON with route, totals, sourced findings, and next actions |
+| Parse the appeal and policy | `docling` | Markdown with the record, amounts, rules, analysis, and conclusion |
+| Extract claim facts | `fastino/gliner2-large-v1` | Amounts, debris volume, loss date, and coverage terms |
+| Retrieve controlling policy language | `BAAI/bge-reranker-v2-m3` | Ranked passages about non-owned debris removal |
+| Produce the cited review | `Qwen/Qwen3.5-4B:no-spec` | JSON separating covered work, excluded costs, and evidence still needed |
 
-Every model call goes through SIE. The review uses the generation endpoint; the
-photograph uses Grounding DINO through the extract endpoint.
+Every model call goes through SIE.
 
-## Verified result
+## Expected result
 
-We ran the complete packet on an NVIDIA L4 on July 23, 2026. The evaluator
-passed all six checks. Qwen returned `manual_review` with two sourced findings:
+The evaluator checks facts stated in FEMA's published decision:
 
-- Blocking: `Proof of Loss lacks required signature and date`
-- High priority: `Claimed total exceeds attachment total by $400.00`
+- amended proof of loss: `$182,552.00`;
+- debris-removal estimate: `$49,500.00`;
+- barge estimate: `$181,832.94`;
+- covered physical scope: `12` to `15` cubic yards beneath the building;
+- excluded scope: barge transport, handling, disposal, and yard removal;
+- follow-up evidence: comparison estimates and proof of work from prior claims.
 
-Grounding DINO found furniture at 0.509 confidence and standing water at 0.276
-confidence. The saved boxes use the original 3072 by 2304 image coordinates.
+The model summarizes a completed public appeal. It does not decide a live claim.
+
+The [`verified-run`](verified-run/) directory records a July 26, 2026 run in
+which all ten factual checks passed.
 
 ## Run it
 
@@ -40,15 +45,10 @@ cp .env.example .env
 uv sync
 
 uv run fetch-claim-sources
-uv run prepare-claim
 uv run review-claim --run-id local
 uv run eval-claim runs/local
 ```
 
-`prepare-claim` fills FEMA Form 086-0-09 with the fictional values in
-`fixtures/claim.json`. It also creates a contractor-style repair estimate and
-copies the public-domain photograph into the packet.
-
 For SIE Cloud, set one URL and key:
 
 ```bash
@@ -63,17 +63,17 @@ separate ports:
 # Terminal 1: Docling, GLiNER2, and reranking
 sie-server serve --port 8080
 
-# Terminal 2: Grounding DINO and Qwen generation
-sie-server serve --models IDEA-Research/grounding-dino-tiny,Qwen/Qwen3.5-4B:no-spec --port 8081
+# Terminal 2: Qwen generation
+sie-server serve --models Qwen/Qwen3.5-4B:no-spec --port 8081
 
 SIE_GENERATION_URL=http://localhost:8081 uv run review-claim --run-id local
 ```
 
-On one GPU, release the default bundle before loading the generation models:
+On one GPU, release the default bundle before loading the generation model:
 
 ```bash
 uv run review-claim --run-id local --stage default
-# Stop the default server, then start the Grounding DINO + Qwen server on the same port.
+# Stop the default server, then start Qwen on the same port.
 uv run review-claim --run-id local --stage generation
 ```
 
@@ -82,17 +82,16 @@ uv run review-claim --run-id local --stage generation
 ```text
 runs/<run-id>/manifest.json           endpoints, models, and per-call latency
 runs/<run-id>/source-manifest.json    source URLs, rights, sizes, and checksums
-runs/<run-id>/packet-manifest.json    packet files and expected reconciliation
-runs/<run-id>/markdown/*.md           parsed form, estimate, and policy
+runs/<run-id>/markdown/*.md           parsed appeal decision and policy
+runs/<run-id>/claim-facts.json        extracted amounts, dates, and scope phrases
 runs/<run-id>/policy-evidence.json    reranked policy passages
-runs/<run-id>/photo-analysis.md       vision-model observations
-runs/<run-id>/review.json             structured claim review
-runs/<run-id>/evaluation.json         deterministic result checks
+runs/<run-id>/review.json             structured appeal review
+runs/<run-id>/evaluation.json         deterministic factual checks
 runs/<run-id>/raw/*.json              complete model responses
 ```
 
 ## Safety boundary
 
-The output routes evidence for a human adjuster. It does not approve or deny
-coverage, calculate a payment, label fraud, or make a legal determination. The
-claim is fictional, and the generated estimate is marked as a software fixture.
+The output summarizes a published FEMA appeal for software evaluation. It does
+not approve or deny coverage, calculate a payment, label fraud, or replace an
+adjuster.

examples/insurance-claims-agent/config.yaml

@@ -9,7 +9,6 @@ models:
   parse: "docling"
   extract: "fastino/gliner2-large-v1"
   rerank: "BAAI/bge-reranker-v2-m3"
-  vision: "IDEA-Research/grounding-dino-tiny"
   review: "Qwen/Qwen3.5-4B:no-spec"
 
 retrieval:
@@ -18,14 +17,14 @@ retrieval:
   chunk_characters: 1800
 
 sources:
-  - slug: "nfip-proof-of-loss"
-    title: "NFIP Proof of Loss, FEMA Form 086-0-09"
-    file_name: "nfip-proof-of-loss.pdf"
+  - slug: "nfip-appeal-b8"
+    title: "FEMA Flood Insurance Appeal Decision B8"
+    file_name: "nfip-appeal-b8.pdf"
     media_type: "application/pdf"
-    url: "https://www.fema.gov/sites/default/files/2020-07/FEMA-Form_086-0-09_proof-of-loss.pdf"
-    source_page: "https://www.fema.gov/flood-insurance/find-form/claims"
+    url: "https://www.floodsmart.gov/sites/default/files/media/document/2022-12/B8_Direct%20Physical%20Loss_Proof%20of%20Loss_Overturn.pdf"
+    source_page: "https://www.floodsmart.gov/flood-claim-memos/appeal-issue/direct-physical-loss/proof-loss"
     rights: "U.S. federal government work. Do not imply FEMA or DHS endorsement."
-    fixture_file: "fixtures/sources/nfip-proof-of-loss.pdf"
+    fixture_file: "fixtures/sources/nfip-appeal-b8.pdf"
 
   - slug: "sfip-dwelling-policy"
     title: "Standard Flood Insurance Policy, Dwelling Form"
@@ -35,12 +34,3 @@ sources:
     source_page: "https://www.fema.gov/flood-insurance/find-form/underwriting"
     rights: "U.S. federal government work. Do not imply FEMA or DHS endorsement."
     fixture_file: "fixtures/sources/sfip-dwelling-policy.pdf"
-
-  - slug: "flooded-house-interior"
-    title: "Flooded house interior"
-    file_name: "flooded-house-interior.jpg"
-    media_type: "image/jpeg"
-    url: "https://upload.wikimedia.org/wikipedia/commons/0/01/Flooded_house_interior.jpg"
-    source_page: "https://commons.wikimedia.org/wiki/File:Flooded_house_interior.jpg"
-    rights: "Public domain U.S. Fish and Wildlife Service photograph."
-    fixture_file: "fixtures/sources/flooded-house-interior.jpg"

examples/insurance-claims-agent/fixtures/SOURCES.md

@@ -1,37 +1,25 @@
 # Source and data notes
 
-The example combines public government documents, a public-domain photograph,
-and an explicitly fictional claim.
+The example uses two public FEMA documents. Both are U.S. federal government
+works. Their inclusion does not imply FEMA or DHS endorsement.
 
-## FEMA NFIP Proof of Loss
+## FEMA Flood Insurance Appeal Decision B8
 
-- File: https://www.fema.gov/sites/default/files/2020-07/FEMA-Form_086-0-09_proof-of-loss.pdf
-- Form: FEMA Form 086-0-09 (04/17)
-- Use: the example fills the official form with fictional claim data
+- File: https://www.floodsmart.gov/sites/default/files/media/document/2022-12/B8_Direct%20Physical%20Loss_Proof%20of%20Loss_Overturn.pdf
+- Source page: https://www.floodsmart.gov/flood-claim-memos/appeal-issue/direct-physical-loss/proof-loss
+- Published record: FEMA Flood Insurance Appeal Decision B8
+- Use: the claim record, amounts, policy issue, analysis, and conclusion
+- Privacy: FEMA published the decision without the policyholder's identity or
+  property address
 - Rights: U.S. federal government work under 17 U.S.C. § 105
-- Reproducibility: the exact government PDF is bundled as a fallback because
-  FEMA may reject requests from datacenter IP addresses
+- Reproducibility: the exact PDF is bundled because government sites may reject
+  requests from datacenter IP addresses
 
 ## Standard Flood Insurance Policy, Dwelling Form
 
 - File: https://www.fema.gov/sites/default/files/documents/fema_F-122-Dwelling-SFIP_2021.pdf
-- Use: policy retrieval and cited claim-review findings
+- Source page: https://www.fema.gov/flood-insurance/find-form/underwriting
+- Use: retrieval of the policy terms governing non-owned debris removal
 - Rights: U.S. federal government work under 17 U.S.C. § 105
-- Reproducibility: the exact government PDF is bundled as a fallback because
-  FEMA may reject requests from datacenter IP addresses
-
-## Flooded house interior photograph
-
-- File page: https://commons.wikimedia.org/wiki/File:Flooded_house_interior.jpg
-- Author: U.S. Fish and Wildlife Service
-- Use: visual inspection of the submitted damage photograph
-- Rights: public domain U.S. federal government work
-- Reproducibility: the exact photograph is bundled as a fallback for Wikimedia
-  rate limits
-
-## Fictional claim
-
-`claim.json` and `claim-note.txt` contain no real policyholder information.
-`prepare-claim` fills the FEMA form and produces the attached estimate. The
-deliberate evidence issues are an unsigned proof of loss and a $400 difference
-between the form and the estimate.
+- Reproducibility: the exact PDF is bundled because FEMA may reject requests
+  from datacenter IP addresses

examples/insurance-claims-agent/insurance_claims/config.py

@@ -1,6 +1,5 @@
 from __future__ import annotations
 
-import json
 import os
 from dataclasses import dataclass
 from pathlib import Path
@@ -12,9 +11,7 @@
 ROOT = Path(__file__).resolve().parent.parent
 DATA_DIR = ROOT / "data"
 SOURCE_DIR = DATA_DIR / "sources"
-PACKET_DIR = DATA_DIR / "claim-packet"
 RUNS_DIR = ROOT / "runs"
-FIXTURES_DIR = ROOT / "fixtures"
 
 
 @dataclass(frozen=True)
@@ -31,7 +28,6 @@ class ModelsConfig:
     parse: str
     extract: str
     rerank: str
-    vision: str
     review: str
 
 
@@ -91,7 +87,6 @@ def load_config() -> AppConfig:
         parse=os.environ.get("SIE_PARSE_MODEL", model_raw["parse"]),
         extract=os.environ.get("SIE_EXTRACT_MODEL", model_raw["extract"]),
         rerank=os.environ.get("SIE_RERANK_MODEL", model_raw["rerank"]),
-        vision=os.environ.get("SIE_VISION_MODEL", model_raw["vision"]),
         review=os.environ.get("SIE_REVIEW_MODEL", model_raw["review"]),
     )
     retrieval_raw = raw["retrieval"]
@@ -113,10 +108,6 @@ def load_config() -> AppConfig:
     )
 
 
-def load_claim() -> dict[str, Any]:
-    return json.loads((FIXTURES_DIR / "claim.json").read_text(encoding="utf-8"))
-
-
 def source_by_slug(config: AppConfig, slug: str) -> Source:
     for source in config.sources:
         if source.slug == slug: