fix(extraction): graceful typed errors for scanned/corrupt PDFs (closes #8) - #13
Merged
Merged
Conversation
A scanned, image-only PDF (no text layer) crashed the letter-upload flow with a raw exception that surfaced as a 500. This closes the gap at the source across all three extraction entry points: - pdf_pages.pdf_to_image_bytes: wrap pdf2image, normalize every render failure (corrupt/password-protected file, missing poppler, zero pages) into a typed PdfRenderError instead of letting a raw exception escape. - extraction.extract_from_letter_file: replace the raw RuntimeError on an empty tool_call (the scanned-PDF case) with a typed ExtractionError, add an empty-page guard, and guard malformed tool-call JSON. - ai/react_agent/ocr.py: parse the OCR response defensively (_parse_ocr_response) so a blank/unreadable scan or malformed provider payload raises a typed OcrError rather than a KeyError/IndexError 500. - orchestrator: new _ocr_letter_file renders PDFs to page images before OCR (the SSE /process path previously shipped raw %PDF bytes to the image OCR model, so scanned PDFs never produced text); map PdfRenderError / ExtractionError to friendly SSE error events. - routers (POST /letters, /api/letters): map PdfRenderError -> PDF_RENDER_FAILED and ExtractionError -> EXTRACTION_FAILED (502) with actionable copy. Adds backend/tests/test_scanned_pdf_graceful.py (21 regression tests, all paths) plus pytest.ini + conftest. No unrelated reformatting. Closes #8
This was referenced Jun 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #8 — uploading a scanned, image-only PDF (no text layer) crashed the letter-upload flow with a raw exception that surfaced as a 500.
This PR closes the gap at the source across all three extraction entry points, turning every "can't read this document" failure into a typed, user-facing error envelope instead of an unhandled crash.
Root cause
extraction.extract_from_letter_file()raised a rawRuntimeErrorwhen the vision model returned no tool call — the expected outcome for a text-layer-less scan.pdf_pages.pdf_to_image_bytes()let rawpdf2image/poppler exceptions (corrupt/password-protected file, missing poppler) escape./processpath (ai/react_agent/ocr.py) base64-encoded raw%PDFbytes and shipped them to the image OCR model labelledimage/jpeg, so scanned PDFs never produced text; a malformed/blank response then raisedKeyError/IndexError.Changes
pdf_pages.py— newPdfRenderError; normalize every render failure (incl. zero pages) into it.extraction.py— newExtractionError; replace the rawRuntimeError, add an empty-page guard, and guard malformed tool-call JSON.ai/react_agent/ocr.py— newOcrError+_parse_ocr_response()that defensively extracts text (blank/malformed → typed error).orchestrator.py— new_ocr_letter_file()renders PDFs → page PNGs before OCR (so scanned PDFs actually work, and fail cleanly when blank); mapPdfRenderError/ExtractionErrorto friendly SSEerrorevents.POST /letters,/api/letters) — mapPdfRenderError→PDF_RENDER_FAILEDandExtractionError→EXTRACTION_FAILED(502) with actionable copy ("try uploading it as an image instead").Tests
backend/tests/test_scanned_pdf_graceful.py— 21 regression tests covering every path (OCR parse, PDF render, structured extraction, the SSE pipeline, and the productionPOST /lettersendpoint named in the issue). All pass;ruffclean on every touched file.Note on the other open PRs (#10, #11, #12)
This is a clean, focused alternative: it contains only the substantive logic + tests, with none of the repo-wide
ruffreformatting that made #10/#11/#12 large and hard to review (the concern raised in the issue thread). Diff is ~260 lines across 6 files. Introduces zero new lint errors vs.mainbaseline. Recommend closing the reformat-heavy PRs in favour of this one.Closes #8