Skip to content

fix(extraction): graceful typed errors for scanned/corrupt PDFs (closes #8) - #13

Merged
aircode610 merged 1 commit into
mainfrom
fix/issue-8-scanned-pdf-graceful-clean
Jun 22, 2026
Merged

fix(extraction): graceful typed errors for scanned/corrupt PDFs (closes #8)#13
aircode610 merged 1 commit into
mainfrom
fix/issue-8-scanned-pdf-graceful-clean

Conversation

@aircode610

Copy link
Copy Markdown
Owner

Summary

Fixes #8 — uploading a scanned, image-only PDF (no text layer) crashed the letter-upload flow with a raw exception that surfaced as a 500.

This PR closes the gap at the source across all three extraction entry points, turning every "can't read this document" failure into a typed, user-facing error envelope instead of an unhandled crash.

Root cause

  1. extraction.extract_from_letter_file() raised a raw RuntimeError when the vision model returned no tool call — the expected outcome for a text-layer-less scan.
  2. pdf_pages.pdf_to_image_bytes() let raw pdf2image/poppler exceptions (corrupt/password-protected file, missing poppler) escape.
  3. The SSE /process path (ai/react_agent/ocr.py) base64-encoded raw %PDF bytes and shipped them to the image OCR model labelled image/jpeg, so scanned PDFs never produced text; a malformed/blank response then raised KeyError/IndexError.

Changes

  • pdf_pages.py — new PdfRenderError; normalize every render failure (incl. zero pages) into it.
  • extraction.py — new ExtractionError; replace the raw RuntimeError, add an empty-page guard, and guard malformed tool-call JSON.
  • ai/react_agent/ocr.py — new OcrError + _parse_ocr_response() that defensively extracts text (blank/malformed → typed error).
  • orchestrator.py — new _ocr_letter_file() renders PDFs → page PNGs before OCR (so scanned PDFs actually work, and fail cleanly when blank); map PdfRenderError/ExtractionError to friendly SSE error events.
  • routers (POST /letters, /api/letters) — map PdfRenderErrorPDF_RENDER_FAILED and ExtractionErrorEXTRACTION_FAILED (502) with actionable copy ("try uploading it as an image instead").

Tests

backend/tests/test_scanned_pdf_graceful.py21 regression tests covering every path (OCR parse, PDF render, structured extraction, the SSE pipeline, and the production POST /letters endpoint named in the issue). All pass; ruff clean on every touched file.

21 passed in 1.40s

Note on the other open PRs (#10, #11, #12)

This is a clean, focused alternative: it contains only the substantive logic + tests, with none of the repo-wide ruff reformatting that made #10/#11/#12 large and hard to review (the concern raised in the issue thread). Diff is ~260 lines across 6 files. Introduces zero new lint errors vs. main baseline. Recommend closing the reformat-heavy PRs in favour of this one.

Closes #8

A scanned, image-only PDF (no text layer) crashed the letter-upload flow
with a raw exception that surfaced as a 500. This closes the gap at the
source across all three extraction entry points:

- pdf_pages.pdf_to_image_bytes: wrap pdf2image, normalize every render
  failure (corrupt/password-protected file, missing poppler, zero pages)
  into a typed PdfRenderError instead of letting a raw exception escape.
- extraction.extract_from_letter_file: replace the raw RuntimeError on an
  empty tool_call (the scanned-PDF case) with a typed ExtractionError, add
  an empty-page guard, and guard malformed tool-call JSON.
- ai/react_agent/ocr.py: parse the OCR response defensively (_parse_ocr_response)
  so a blank/unreadable scan or malformed provider payload raises a typed
  OcrError rather than a KeyError/IndexError 500.
- orchestrator: new _ocr_letter_file renders PDFs to page images before OCR
  (the SSE /process path previously shipped raw %PDF bytes to the image OCR
  model, so scanned PDFs never produced text); map PdfRenderError /
  ExtractionError to friendly SSE error events.
- routers (POST /letters, /api/letters): map PdfRenderError -> PDF_RENDER_FAILED
  and ExtractionError -> EXTRACTION_FAILED (502) with actionable copy.

Adds backend/tests/test_scanned_pdf_graceful.py (21 regression tests, all
paths) plus pytest.ini + conftest. No unrelated reformatting.

Closes #8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

500 error on /api/letters upload when PDF has no text layer

1 participant