Repository navigation
Open
Labels
featureFeatureFeaturestatus: blockedWaits for the issues in its Blocked by sectionWaits for the issues in its Blocked by sectiontrack: pipelineDocuments, extraction, AI and failure handlingDocuments, extraction, AI and failure handlingv3PromiseTracker v3PromiseTracker v3
Milestone
Description
Type: AFK. Can be built and merged without a decision.
Owner: koechkevin. Order: 6 of 12 for this owner, after #652 and before #654. Track: pipeline. Build order: #672
Part of PromiseTracker v3, version 1. Specification: docs/spec/promisetracker-v3-system-specification.md. Section numbers below refer to it.
What to build
The tika content extractor, so documents in complex formats reach the extracted state.
Start here
- Read first: sections 6.2, 9.3 and 16.3 of the specification.
- Code goes in: a new
plugins/tikapackage, shaped likeplugins/site. A new plugin is also added tofirstPartyPluginsinapps/cms/src/kernel.ts, totranspilePackagesinapps/site/next.config.ts, and to theCOPYlines forpackage.jsonfiles indocker/Dockerfile. - Already there: the
ContentExtractorport inpackages/kernel/src/ports.ts, the Tika container indocker/compose.yamlonPT_TIKA_PORT, and the health report inapps/cms/src/health.ts. - From v2:
src/tasks/extractDocuments.ts, for how v2 called its extractor. Do not copy itscleanText, which strips accents. Read a v2 file withgit show v2-final:<path>. - Migration: Only if the plugin declares settings, which adds the
settingsglobal. - How to work:
AGENTS.md, section "Working on an issue".
Acceptance criteria
-
plugins/tikaprovides thetikaextractor for PDF, Word, Excel, PowerPoint, OpenDocument,
RTF and EPUB, chosen by file type - The Tika server address lives in the plugin's encrypted settings
- Requests time out after two minutes and pass through the circuit breaker
- Tika appears in
/api/healthonly when the plugin is enabled, and never in
/api/health/ready - A spreadsheet uploaded as a document, not as records, is read as text
- Integration tests run against the Tika container in Compose and CI, with a sample PDF and
Word file containing French text
Blocked by
- V1-14: Documents, connectors and the processing pipeline #652, owned by koechkevin
Unblocks
- V1-16: plugin-ai: candidate claims and review #654, owned by koechkevin
Metadata
Metadata
Assignees
Labels
featureFeatureFeaturestatus: blockedWaits for the issues in its Blocked by sectionWaits for the issues in its Blocked by sectiontrack: pipelineDocuments, extraction, AI and failure handlingDocuments, extraction, AI and failure handlingv3PromiseTracker v3PromiseTracker v3
Activity