OpenHuman-shaped simulation, hardening, benchmark & test suite for tinydocs #34

Why

tinydocs is how OpenHuman writes and reads office documents. It has two jobs for the host, and both are user-visible:

• Synthesis. The generate_document and generate_presentation tools turn a model-authored JSON spec into a .docx / .pptx artifact the user opens in Word, Keynote, PowerPoint or LibreOffice. If the file does not open, or opens with text missing, the turn failed even though the tool returned success.
• Intake. When a user attaches a PDF, DOCX, PPTX or XLSX and the chat model cannot take the file natively, AttachmentModel::fallback extracts the text (with page/slide/sheet provenance), renders up to four scanned PDF pages to PNG for a vision readout, and pastes the result into the prompt. [FILE:…] markers take a second, older path through ExtractText (PDF only). Whatever the extractor misses, the model never sees.

The host surface, at the pinned gitlink vendor/tinydocs = 1c9daad (v0.1.22, identical to upstream tinyhumansai/tinydocs@main per gh api …/compare):

• TinyBus module only, no library linkage. crates/openhuman-core depends on tinydocs-bus alone (Cargo.toml:484, behind feature documents, which is in scripts/ci/product-features.txt:50). All parsing and writing runs in the downloaded tinydocs-module cdylib, in the core process. Registry record crates/openhuman-core/src/modules/registry/records_docs_wallet.rs:9-74: id tinydocs, bus ai.tinyhumans.tinydocs.Documents, object /ai/tinyhumans/tinydocs/Documents, version 0.1.22, 11 platform assets, LoadPolicy::Lazy.
• Host client crates/openhuman-core/src/modules/documents.rs calls every member through tinydocs_bus::names::methods::*: GENERATE_DOCX, GENERATE_PPTX (images as one concatenated stream), EXTRACT_TEXT, EXTRACT_DOCUMENT, RENDER_PDF, then READ_OUTPUT in READ_CHUNK = 1 MiB pieces (:44), sha256 check, RELEASE_OUTPUT; MAX_DOCUMENT_BYTES = 64 MiB (:54); a drop guard releases held outputs with a 5 s per-call timeout (:238-263).
• Callers and their budgets: generate_document 30 s (tools/impl/document/mod.rs:65), generate_presentation 30 s (tools/impl/presentation/mod.rs:61), both PermissionLevel::Write, both load the module before the deadline starts (document/engine.rs:77, presentation/engine.rs:84). DocumentsTextExtractor (agent/multimodal.rs:102-145) uses ExtractText under PDF_EXTRACTION_TIMEOUT = 60 s (:40). AttachmentModel::fallback (agent/attachments/provider_fallback.rs:62-202): ExtractDocument 60 s with max_text_bytes = min(max_extracted_text_chars, 200_000) (:81); scanned pages → RenderPdf 60 s with up to 4 pages, max_dimension 2048, 16,000,000 px, 32 MiB (:100-105, all at the module's hard ceilings); each PNG → vision readout 60 s (:243); on Unavailable for a PDF, falls back to ExtractText 60 s (:158-189).
• File config defaults (config/schema/tools/multimodal.rs): 4 files per turn, 16 MB per file, 50,000 extracted chars; clamped to 1–16 files, 1–50 MB, 1,000–200,000 chars. Untrusted channels set max_files = 0.
• Bus defaults that silently apply: every call, including call_with_stream, uses tinybus::connection::DEFAULT_TIMEOUT = 30 s (vendor/tinybus/crates/tinybus/src/connection.rs:126, :869-876); streams cap at 256 MiB, 4 live streams per peer, 60 s idle (stream/mod.rs:107-113); frames cap at 16 MiB.
• Module limits: intake 64 MiB input, 2,048 ZIP members, 32 MiB expansion, 8 MiB per part, 128 XML depth, 256 sections, 200,000 text bytes, 4,096 PDF pages (src/intake/mod.rs:15-20, :34-43); render 8 pages, 2,048 px, 16 M px, 32 MiB PNG (src/pdf_render/mod.rs:118-141); output store 4 MiB chunk, 64 MiB per output, 128 MiB total, 32 live, 300 s idle TTL (crates/tinydocs-module/src/outputs/mod.rs:41-56); 2 module worker threads, CPU work on Tokio's blocking pool (service/mod.rs:313-325, :384); synthesis caps as listed in the README (128 sections / 2,000,000 chars; 64 slides / 8 images / 5 MiB each).

What has never run end-to-end in OpenHuman's shape: (a) the host client against the real cdylib (no host test loads it, and there is no TINYDOCS_TEST_MODULE hook in the host); (b) any real-world document: every module fixture is synthesized in test code, so extraction quality on born-digital PDFs, scans, Word files with headers and footnotes, or real spreadsheets has never been measured; (c) whether a generated .docx/.pptx opens in an office suite; (d) the attachment path's timeout, cold-start, concurrency and hostile-input behaviour.

Goals:

1. Replay OpenHuman's real document workloads (agent-authored specs, user attachments, scanned PDFs) through the exact bus surface and budgets, against the built cdylib, on a checked-in fixture corpus with ground truth.
2. Score what matters to the user: synthesis validity (opens, round-trips, keeps every word) and intake fidelity (text recall, reading order, provenance, structure, scanned-page detection), plus latency against OpenHuman's 30 s / 60 s budgets, memory, and concurrency, with noise bands.
3. Turn every confirmed gap into a tracked finding and a regression test in the repo that owns it.

What already exists (build on it, don't duplicate)

• Library tests (all synthetic fixtures). src/intake/mod_tests.rs (24 tests: DOCX/PPTX/XLSX text and provenance, manifest ordering, shared/inline strings, UTF-16 parts, phonetic exclusion, breaks/tabs, budgets, DTD/entity rejection, ZIP expansion), zip_admission_tests.rs (7: ZIP64 lies, split archives, duplicate raw names, footer signatures), office_order_tests.rs (3), pdf/mod_tests.rs (6), pdf_render/mod_tests.rs (3: page selection, PNG dims, budgets, encryption), docx/mod_tests.rs (4), pptx/mod_tests.rs (15: title slide, notes part, image layout), tests/public_api.rs (5). Spec validation in crates/tinydocs-bus/src/spec/*/mod_tests.rs.
• Module tests. service/mod_tests.rs (13: manifest/dispatch parity, error wire names, stream/spec mismatch, partial raster batch release), outputs/mod_tests.rs (17: chunking, budgets, TTL, unguessable ids). tests/module_e2e.rs loads the real cdylib over a MemoryBus broker and exercises every member once (one tiny DOCX, one one-line PDF, one 2-image deck); CI runs it on Linux only (.github/workflows/ci.yml:83-84).
• No fuzzing, no property tests, no real-world corpus, no office-suite validity check, no benchmark.
• Host tests. modules/documents_tests.rs covers error classification, digest vector, a disabled host, registry/contract parity and the intake version gate, never a loaded module. tools/impl/{document,presentation}/*_tests.rs cover schema, validation and engine timeouts against a module that is not there. agent/attachments/provider_routing_tests.rs:175 covers fallback dispatch with an unavailable module. tests/in_process/orchestrator_presentation_wiring.rs checks only agent/tool wiring.
• Benchmark repo. The only document touchpoint is life-scenarios trip-itinerary (scenarios.mjs:392-445): one generated hotel-booking PDF whose facts are graded by read_the_pdf. The agent reads it with workspace tools, not through the attachment path, so tinydocs is not exercised. No synthesis or intake baseline exists. Section C defines the first.

A. OpenHuman host simulator — one-way dependency

The arrow points OpenHuman → tinydocs and must stay there. tinydocs must never depend on openhuman-core, not as a dependency, a dev-dependency, or from an example or bench. Today it depends on tinybus/tinybus-module only; keep it that way.

The simulator lives in tinydocs (crates/tinydocs-module/examples/openhuman_profile/, or a small unpublished tinydocs-sim crate) and mirrors modules/documents.rs plus its two intake callers, with OpenHuman's defaults as named constants and a parity table so drift shows up in review:

The unit of simulation is one mocked OpenHuman call, in one of three shapes:

• tool call: a generate_document / generate_presentation JSON spec (as a model would emit it) → spec validation → GenerateDocx/GeneratePptx (+ image stream) → ReadOutput loop → sha256 → ReleaseOutput, under 30 s.
• attachment turn: bytes + MIME → ExtractDocument → for scanned_candidate pages, RenderPdf → collect + release each PNG → a stubbed vision readout (records the PNG, returns fixed text) → the assembled prompt text with OpenHuman's char limit.
• [FILE:] marker: bytes → ExtractText → collect → UTF-8 → char truncation.

Scenarios are JSON data files (scenarios/*.json: shape, fixture id, MIME, budgets, expected outcome class: ok / invalid_input / extraction_failed / timeout / refused, plus the scoring hooks) so OpenHuman can load the same files later from a host test.

Fixture corpus. tests/corpus/ with a manifest.json per file: source, license, sha256, format, and ground truth (per-section expected text, reading-order line list, expected section sources/indices, expected scanned_candidate pages, table cells for XLSX). Two halves: (1) generated fixtures with exact ground truth, scripted and reproducible (LibreOffice-, Word- and Google-exported variants of the same source text, so producer quirks are covered); (2) a small set of permissively licensed real-world files (for example the pdf.js test corpus, Apache-2.0) for layout cases a generator will not produce. Large files are fetched by digest in live mode, not committed.

Two modes:

• mock (CI, deterministic): calls the library functions in-process (tinydocs::intake::extract, pdf_render::render, docx::generate, pptx::generate) on the committed corpus; scoring and validity checks are pure Rust (XML well-formedness of every part, relationship and content-type integrity, round-trip through intake). No office suite, no bus.
• live: loads the built tinydocs-module cdylib over a real broker exactly like module_e2e.rs, so the stream, output store, 30 s bus timeout, 4-stream limit and blocking pool are real; adds LibreOffice headless (soffice --headless --convert-to pdf) as the "does it open" oracle for synthesized files and pdftoppm as the reference rasterizer for render scoring.

B. OpenHuman-shaped scenarios

Each scenario carries named probes. A seeded generator scales any scenario S→L (1 document → a batch of N; 1 page → 4,096 pages; 1 section → 128).

1. Weekly report, agent-authored. A 6-section generate_document spec the way models actually write it: **bold** and - bullets inside paragraph strings, embedded \n, emoji, CJK, RTL, a 20,000-char paragraph. Probes: opens_in_office, roundtrip_recall (every spec word found by intake), newline_semantics (an embedded \n becomes a space today, see D9), heading_outline (Heading1 appears as an outline/navigation entry, not just bold text), markdown_literal (documents that asterisks are written verbatim).
2. Hostile spec text. Titles and bullets containing U+0001, U+000B, U+001F, U+FFFE, and 2,000-char unbroken tokens. Probes: either validate() refuses naming the field, or the file still opens. Today neither holds (D2).
3. Full deck. 64 slides, 32 bullets each, speaker notes, 8 images × 5 MiB across slides (40 MiB stream). Probes: completes inside 30 s, ReadOutput round-trip count, output size, slide_count == slides + 1, notes part present, image aspect ratios preserved; then force a 1 s deadline and check the store afterwards (orphaned_outputs, D4).
4. Born-digital PDF report. Two-column layout, tables, ligatures (fi, fl), footnotes, a CID font without ToUnicode, rotated pages. Probes: CER and token F1 per page, reading-order score, source == page:N, footnote text present, garbage-glyph rate.
5. Scanned and mixed PDF. Fully scanned pages, scanned pages with a printed page-number footer, pages with an invisible OCR text layer, and blank pages. Probes: scanned_candidate precision/recall against ground truth (a scan with a footer is missed today, D9), rendered page == requested page (perceptual hash vs pdftoppm, D10), at most 4 renders, the "limited to four pages" note when more exist.
6. Real Word document. Headers, footers, footnotes, endnotes, comments, tracked insertions and deletions, text boxes (mc:AlternateContent), tables, fields. Probes: coverage per part type (only word/document.xml is read today), duplicate_text_rate (text boxes appear in both mc:Choice and mc:Fallback), deleted text absent, inserted text present, table cell order.
7. Budget workbook. Three named sheets, dates, currency, percentages, booleans, formulas with cached values, merged cells, a sparse sheet with a far-away cell. Probes: dates rendered as serial numbers (no styles.xml read), booleans as 0/1, sheet names absent from provenance, cell_adjacency (label and value on the same logical row), shared-string budgets.
8. Real deck intake. Speaker notes, tables, grouped shapes, a hidden slide, a chart. Probes: notes absent (so our own generated deck loses its notes on round-trip), hidden slide behaviour, table text order, manifest order.
9. Size and budget edges. A 4,096-page PDF, a 50 MB attachment at the host's clamp, a 64 MiB + 1 byte input, budgets at 1 byte and 200,000 bytes, a CJK document where OpenHuman's char limit becomes a byte budget. Probes: correct truncated/section_count, no split code points, latency vs 30 s (bus) and 60 s (tool), peak RSS.
10. Hostile corpus. ZIP bomb, overlapping/duplicate members, DTD and billion-laughs XML, ZIP64 lies, encrypted ZIP and PDF, a self-referencing Form XObject (D1), deeply nested PDF arrays, a 10^6-point page box, a JPEG2000/JBIG2 image bomb for render. Probes: a named error for each, host_survived (the process is still alive), peak RSS, wall time.
11. Concurrency. 4 attachments per turn × N parallel sessions, plus a deck generation, all streaming at once. Probes: the 5th live stream (TooManyStreams, D8), output-store refusals, blocking-thread count, p95 under load.
12. Cold start. First document request with downloads enabled through a throttled local mirror, and one with the mirror down. Probes: generation tools succeed (load is outside their deadline); the attachment and [FILE:] paths spend their 60 s on the download (D5); after one failed download, every later call in the process stays Unavailable (D5).
13. Differential against the host's other parser. The same corpus through TinyMemory's OfficeConverter (vendor/tinymemory/crates/tinymemory-integrations/src/documents/office/, linked into the host by documents → tinymemory-integrations/documents-office) and through tinydocs intake. Probe: per-file disagreement in recall and order, so chat intake and memory ingestion do not quietly see different documents (context for tinydocs#31).

C. KPIs and harness upgrades (REPEAT ≥ 3 noise bands)

• Synthesis validity: opens_in_office rate (LibreOffice live; well-formedness + rel/content-type integrity in mock), round-trip recall (spec words recovered by intake / spec words), structural checks (title slide, heading styles, bullet numbering, notes part, embedded image count and aspect ratio).
• Intake fidelity, per format and per corpus half: character error rate and token F1 against ground truth per section; reading-order score (Kendall tau over ground-truth lines); provenance accuracy (source/index correct); duplicate-text rate; XLSX cell adjacency; scanned_candidate precision/recall; render page-match rate and dimension correctness. Report one fidelity score per format as the mean of recall, order and provenance, with each component shown.
• Latency: p50/p95/p99 per member and per scenario shape, end-to-end as OpenHuman sees it (stream + work + ReadOutput loop + release); timeout rate against 30 s (tools, bus) and 60 s (intake); cold-load time reported separately from warm.
• Transfer cost: stream bytes in, ReadOutput calls and base64 bytes out per document; time spent in transfer vs in work.
• Resources: peak RSS of the process per scenario (input buffered by the stream, parser, double parse in render), blocking threads alive after a caller timeout, output-store occupancy over time.
• Concurrency: throughput and p95 at 1/4/8/16 parallel calls; refusal counts by wire name.
• Hostile corpus: fraction of hostile inputs that produce a named error with the process alive; gate is 100%.

D. Risks to simulate and confirm

Each → live simulation → regression test → fix issue in the owning repo if confirmed.

1. A crafted PDF can crash the whole core. pdf-extract 0.12.1 handles Do by recursing into the Form XObject's content stream with no depth limit or visited set (~/.cargo/registry/…/pdf-extract-0.12.1/src/lib.rs:1860-1868). A Form XObject that names itself recurses until the stack overflows. spawn_blocking catches panics, not stack overflow, and the module runs in the core process, so one attached PDF aborts OpenHuman. ExtractText, ExtractDocument (src/intake/mod.rs:405-410) and RenderPdf's preflight parse (src/pdf_render/mod.rs:47) all reach pdf-extract; check Hayro the same way. The same unguarded parser is also linked directly into the host by tinymemory-integrations/documents-office. Owner: tinydocs (pre-scan the XObject graph for cycles and depth before handing the document to pdf-extract, or fix upstream; process isolation is the long-term answer); host finding for the TinyMemory copy.
2. Control characters produce files Word will not open. docx-rs 0.4.22 escapes only & < > " ', \n and drops \r (docx-rs-0.4.22/src/escape/mod.rs:1-17); ppt-rs 0.2.27 escapes five characters (src/core/xml_utils.rs:8-23). U+0000–U+001F (other than tab/LF/CR) and U+FFFE/U+FFFF are not legal XML 1.0, and DocumentSpec::validate / PresentationSpec::validate (crates/tinydocs-bus/src/spec/document/mod.rs:156, presentation/mod.rs:205) do not reject them. The tool reports success and the user gets a file Word calls corrupt. Owner: tinydocs: reject in validate() with the field name so the model can fix it.
3. The effective deadline is the bus's 30 s, and it is reported as a failure. call_with_stream and Proxy::call use DEFAULT_TIMEOUT = 30 s (tinybus connection.rs:126, :869-876), so the 60 s intake budgets (multimodal.rs:40, provider_fallback.rs:83,107,163) never apply to the call. A slow extraction comes back as ai.tinyhumans.tinybus.Error.Timeout, which classify maps to Failed (modules/documents.rs:353-361), so the prompt says "Document extraction unavailable" rather than "timed out". The module header (documents.rs:20-25, "Nothing here imposes one") is wrong. Owner: OpenHuman (use call_with_stream_timeout with the caller's budget, classify Timeout).
4. A caller timeout leaves orphaned outputs and busy threads. The module's blocking work continues after the caller drops the future (service/mod.rs:313-325) and inserts its output into the store when done. The host's release guard only knows ids it has already received (documents.rs:238-263), so that output sits for IDLE_TTL = 300 s. A few timed-out 40 MiB decks fill MAX_TOTAL_BYTES = 128 MiB and every document call returns OutputRefused for five minutes. Owner: tinydocs (cancel or discard work whose caller is gone; reserve budget before work starts), OpenHuman (retry/backoff shape on OutputRefused).
5. First download inside the intake deadline, and a failed download is permanent. The generation tools load outside their deadline, but DocumentsTextExtractor (multimodal.rs:127-131) and AttachmentModel::fallback (provider_fallback.rs:82-84) call ready() inside the 60 s timeout. The resolution slot keeps Failed for the process lifetime (tinybus module/resolution.rs "remembered for the process lifetime"; the host never calls forget), so a user who is offline the first time they attach a PDF loses documents until restart. Owner: OpenHuman.
6. The intake version gate is a dead string compare, and its fallback is unreachable. intake_available refuses only record.version == "0.1.20" (documents.rs:440-451); the registry is compiled at 0.1.22, so this never fires, and a pre-intake 0.1.19 would pass it. A module without ExtractDocument answers UnknownMethod, which classify maps to Failed, so the Unavailable → ExtractText branch (provider_fallback.rs:158) cannot trigger on that path. Owner: OpenHuman (check the admitted manifest's method list, or delete the gate).
7. The module buffers up to 256 MiB per stream before its 64 MiB check. read_stream (service/mod.rs:197-206) reads to the bus cap of 256 MiB (tinybus stream/mod.rs:107), and only then does intake::extract/pdf::extract_text reject anything over 64 MiB. With 4 streams per peer that is 1 GiB resident from one caller. render then copies the input again (pdf_render/mod.rs:51, bytes.to_vec()) after a full pdf-extract parse (:47). The host's public functions (documents.rs extract_text, extract_document, render_pdf) apply no cap; today's callers cap at ≤50 MB. Owner: tinydocs (bound the read at the library limit), OpenHuman (cap in documents.rs).
8. Four streams per peer. The host is one peer, so a fifth concurrent streamed call (4 attachments × parallel sessions, or attachments plus a deck) should get TooManyStreams, classified Failed. Confirm the behaviour and decide whether the host queues or the module raises its limit. Owner: OpenHuman (queue/semaphore) after confirmation.
9. Intake drops or garbles content users expect. DOCX reads only word/document.xml (src/intake/mod.rs:126-130): headers, footers, footnotes, endnotes and comments are lost. xml_text does not skip mc:Fallback, so text boxes are expected to appear twice (to confirm). XLSX emits cached raw values with no styles.xml (dates as serials), no sheet names, and a newline after every cell and row (:291-293). PPTX notes are not read, so our own generated decks lose their notes on round-trip. scanned_candidate requires an empty text layer (:413), so a scanned page with a printed page number is never rendered. On the synthesis side, \n inside a paragraph is written as 
 in w:t and renders as a space. Owner: tinydocs; each confirmed gap becomes a fidelity-gated test.
10. Extract and render disagree about page numbers. Extraction numbers pages from lopdf's page tree (intake/mod.rs:387-398); render indexes Hayro's page list (pdf_render/mod.rs:59, :90). On a damaged cross-reference table the two can disagree, and the vision readout of one page is labelled as another. Owner: tinydocs (one page model for both).
11. Office text reaches the prompt without the untrusted label. The text-file branch and the PDF fallback label content "(untrusted)" (provider_fallback.rs:56, :169); the ExtractDocument branch pastes section.source and section.text bare (:97), and source is a part path from the attacker's ZIP. Owner: OpenHuman.
12. Worst-case attachment latency is minutes. 60 s extract + 60 s render + 4 × 60 s vision readout = 360 s for one PDF, sequential across up to 4 files per turn. Owner: OpenHuman (one budget per attachment turn).
13. Nothing in host CI loads the module. local_override (modules/ops.rs:443-466) has env hooks for TINYSEARCH_TEST_MODULE and TINYCONNECTORS_TEST_MODULE but not tinydocs, and documents_tests.rs never loads the cdylib, so a contract drift between the host client and the published artifact is caught only by users. Owner: OpenHuman.

E. Feature & e2e test extension

Feature matrix (tinydocs repo). ✅ = covered (cite), ❌ = missing.

e2e to add against the real built artifact (libtinydocs_module.{so,dylib} / tinydocs_module.dll over a real broker; extend module_e2e.rs, whose cross-OS run is tinydocs#32):

• corpus-driven ExtractDocument/RenderPdf runs with fidelity assertions for each format, and synthesized-file validity for both generators;
• a 40 MiB-image deck through GeneratePptx and back through the ReadOutput loop, timed;
• a caller that drops its future mid-call, then asserts the store's live count and bytes return to zero within a bounded time (D4);
• five concurrent streamed calls from one peer (D8);
• a 257 MiB stream and a 65 MiB stream, asserting peak RSS stays near the library limit (D7);
• regression tests for D1 (self-referencing XObject: named error, process alive), D2 (control characters refused by name), D9 (each confirmed fidelity gap), D10 (page mapping on a damaged xref).

OpenHuman-side tests (filed and tracked in the master issue, not implemented in tinydocs):

• Add a TINYDOCS_TEST_MODULE arm to local_override (modules/ops.rs), mirroring tinysearch.
• tests/documents_module_e2e.rs (new openhuman-cli [[test]] in crates/openhuman-cli/Cargo.toml, path ../../tests/documents_module_e2e.rs): drive modules::documents::{generate_docx, generate_pptx, extract_text, extract_document, render_pdf} against the built cdylib; assert digests, release (store empty afterwards), the 30 s bus timeout surfacing as a timeout (D3), TooManyStreams handling (D8). It must print SKIPPED (not run, not asserted) and is vacuous when TINYDOCS_TEST_MODULE is unset, like the tinysearch suite in tests/raw_coverage/worker_b_raw_coverage_e2e.rs:437; CI must set it.
• agent/attachments/provider_routing_tests.rs: DOCX/XLSX/PDF attachments with a loaded module (labels include "(untrusted)", D11; scanned PDF triggers ≤4 renders; UnknownMethod from an old module reaches the ExtractText fallback, D6).
• agent/multimodal tests: [FILE:] PDF with the module loading slowly, asserting the load is not charged to the 60 s budget (D5).
• modules/documents_tests.rs: replace pinned_release_exposes_intake_without_loading with a manifest-method check (D6); a failed first load can be retried (D5).
• tools/impl/document/document_tests.rs / presentation_tests.rs: with the module loaded, a successful tool call produces a file that passes the same validity check as the module suite; a control-character spec returns a field-named InvalidInput (D2).
• tests/json_rpc_e2e.rs: one openhuman.inference_agent_chat turn where the mock LLM (scripts/mock-api/routes/llm.mjs tool-call scripting) calls generate_document, and one turn with a PDF attachment on a model without file input; assert the artifact and the extracted text in the captured prompt.
• app/test/e2e/specs/: extend file-drop-guard.spec.ts into a drop-a-PDF → answer-cites-its-content flow, and a document-artifact card flow, both on the mock backend.

Boundary: cross-OS module builds, per-platform scripts and release/CI gating are tinydocs#32 (and PR tinydocs#33); link, don't restate.

F. Fuzzing and property tests

None exist. Add proptest (dev-dependency) and cargo-fuzz targets:

• Fuzz targets: intake::extract for each DocumentFormat on arbitrary bytes, seeded with the corpus; pdf_render::render with fixed small budgets; zip_admission::zip_preflight; office_order manifest parsing; normalize_xml (UTF-16/BOM handling). Run each target under a reduced thread stack so recursion bugs (D1) surface as crashes in the fuzzer, not in production.
• Properties: for any valid DocumentSpec/PresentationSpec (generated, including arbitrary Unicode), generate either returns InvalidInput naming a field or produces a package in which every XML part is well-formed and intake::extract recovers every non-blank spec string; TextSink never splits a code point and never exceeds its budget; truncated is true iff something was omitted; section_count ≥ sections.len(); RenderPdfSpec budgets are monotone (narrowing a budget never yields more output).
• Wire properties: WirePresentationSpec + concatenated stream: any declared byte_len sequence either resolves exactly or is refused, never sliced wrong.
• Chaos (live): a stream that stalls past the 60 s idle timeout; a caller that disconnects mid-ReadOutput; an output read after TTL; the output store at exactly its 32-entry and 128 MiB edges under parallel inserts.

G. Mock fidelity + CI smoke

• Mock mode scores the committed corpus with the same scorer as live mode, so CI measures extraction quality, not just "returned Ok". The well-formedness/relationship checker for synthesized packages runs in CI on every generated fixture.
• CI smoke job (Linux, per PR): build the cdylib, run module_e2e plus the corpus subset (≤ 2 min), fail on any fidelity score below its gate or any hostile-corpus crash. A fuzz smoke runs each target for 60 s. LibreOffice validity, the full corpus, and the 4,096-page and 40 MiB cases run nightly.
• Guard against vacuous passes: the smoke job asserts the module was loaded and the corpus count is non-zero; a run that skipped fails.

H. Running it locally

cargo build --release -p tinydocs-module
export TINYDOCS_TEST_MODULE="$PWD/target/release/libtinydocs_module.so"
# mock: library in-process, committed corpus, scores to stdout + JSON
cargo run -p tinydocs-module --example openhuman_profile -- --mode mock --scenarios scenarios/*.json
# live: real cdylib over a broker, LibreOffice + pdftoppm oracles, REPEAT=3
cargo run --release -p tinydocs-module --example openhuman_profile -- --mode live --repeat 3 --scenarios scenarios/*.json
# module e2e (extended)
cargo test -p tinydocs-module --test module_e2e -- --ignored
# fuzz smoke
cargo +nightly fuzz run intake_docx -- -max_total_time=60
# host side (once the hook lands)
TINYDOCS_TEST_MODULE=... cargo test -p openhuman-cli --test documents_module_e2e

I. Docs

Add docs/evals/documents.md: how the corpus and ground truth are laid out, how to add a fixture (license, digest, ground truth), what each fidelity component means, and how to benchmark an alternative parser or writer (for example Hayro 0.6 from PR #27, a different PDF text extractor, or TinyMemory's OfficeConverter from tinydocs#31) against the recorded baseline with --repeat 3. Update src/intake/README.md with the measured gaps from D9 until they are fixed, and fix the stale README "Layout" block (it still lists test.rs files that are now mod_tests.rs).

Proposed gates (initial)

Suggested order (sub-issues)

1. D1 and D2 confirmation fixtures first (crash and corrupt-file risks), with fix issues routed to their owners.
2. A: the simulator, the parity table, the corpus layout and the scorer; mock mode in CI.
3. B scenarios 1–3 (synthesis) and 4–8 (intake fidelity) with baselines recorded.
4. D3–D13 confirmation runs, one fix issue per confirmed risk.
5. B scenarios 9–13 (edges, hostile, concurrency, cold start, differential) and the S→L generator.
6. E: extended module_e2e and the host-side suite; G's smoke job.
7. F: fuzz and property tests.
8. I: docs.

Related

• tinydocs#32: per-platform scripts and cross-OS e2e/feature tests (owns the cross-OS run of module_e2e that E extends); PR tinydocs#33 implements it.
• PR tinydocs#31: ConvertMarkdown, contract version 3, moving TinyMemory's document conversion into the module (scenario 13 and D1 apply to it too); part of oh#7292.
• PR tinydocs#27 (Hayro 0.6) and deps: bump quick-xml from 0.41.0 to 0.42.0 #29 (quick-xml 0.42): run the render and intake benchmarks before and after each bump.
• Tracked by the OpenHuman master issue (link added on filing); cross-OS master oh#7335, uniformity oh#7336.

Out of scope

• New formats, OCR, template/theme support in synthesis, or a replacement PDF engine (the suite measures candidates; choosing one is separate work).
• Moving document parsing out of the core process (D1 records why it matters; the isolation design is its own issue).
• OpenHuman changes beyond the listed host tests, and the host-owned findings below, which go to the master issue.