Skip to content

feat: expose the supported-format tables and a per-file-type capability query - #634

Merged
andiwand merged 3 commits into
mainfrom
feat/format-tables
Jul 30, 2026
Merged

feat: expose the supported-format tables and a per-file-type capability query#634
andiwand merged 3 commits into
mainfrom
feat/format-tables

Conversation

@andiwand

@andiwand andiwand commented Jul 30, 2026

Copy link
Copy Markdown
Member

🤖 Generated with Claude Code

Closes #632.

Why

Consumers have to rebuild odrcore's format knowledge by hand — OpenDocument.droid keeps three separate format tables plus an instrumented test to hold them in sync. The lookups in odr.hpp are one-directional and incomplete, and there is no way at all to ask what the library can do with a format before holding a file, which is exactly when an app has to decide (intent filters, picker MIME lists, "open with" entries).

All of it lived as six independent if-chains over FileType in odr.cpp, which had already drifted apart:

  • markdown had a MIME type but no extension.
  • The wpd MIME round trip was broken — libmagic reports application/vnd.wordperfect, file_type_by_mimetype could not map it back, and odr_test::types_wpd documented the gap.
  • rtf, svm, cfb and office_open_xml_encrypted had no MIME type at all.
  • There was no extension_by_file_type; file_type_to_string accidentally doubled as one.
  • Whole alias families were missing (docm, dotx, xlsb, ppsx, the macro-enabled and template MIME types, application/x-vnd.oasis.*).

What

One central table. src/odr/internal/file_type_table.{hpp,cpp} — a constexpr array, one row per FileType, holding the name, every extension, every MIME type (canonical first), the category, the document type and the capabilities. Every public lookup is now a thin forward into it; odr.cpp shrinks from 441 to 164 lines.

Enumeration. all_file_types, file_extensions_by_file_type, file_extension_by_file_type, mimetypes_by_file_type — so a consumer never needs its own copy of these tables.

Capabilities. FileTypeCapabilities + capabilities_by_file_type answer, per format, whether we can detect / open / decrypt / render / edit / save / encrypt it:

struct FileTypeCapabilities final {
  bool detect_by_content{}; ///< recognised from its bytes alone
  bool open{};              ///< a decoder exists
  bool decrypt{};           ///< encrypted instances can be decrypted
  bool translate_html{};    ///< `html::translate` produces output
  bool edit{};              ///< `Document::is_editable` can be `true`
  bool save{};              ///< `Document::save` is supported
  bool encrypt{};           ///< `Document::save` with a password is supported
};

It is explicitly a declared upper bound — the question you have to answer before you hold a file. DecodedFile::capabilities() refines it for the file in hand, and Document::is_editable/is_savable stay the precise answer.

Aliases. docm/dotx/dotm, pptm/potx/potm/ppsx/ppsm, xlsm/xltx/xltm, dot/pot/pps/xlt/xlm, md/markdown, otm, text, ttc, xlsb; the ODF x-vnd/-template/-master/-flat-xml families, the OOXML .template/.slideshow/macroEnabled.12 families, plus text/rtf, application/x-pdf, image/x-ms-bmp and friends.

Keeping it honest

A hand-maintained table can lie, so the invariant is a test rather than discipline.

FileTypeCapabilities.declaration_matches_the_engines walks the corpus and asserts that DecodedFile::capabilities() is a subset of what the table declares, and that Document::is_editable() / is_savable(false|true) equal the declared edit/save/encrypt. Over-declaring edit support for a format that cannot edit now fails CI.

Alongside it: every FileType has exactly one row (derived from the enum, not the table, so a missing row is caught), no extension or MIME type is claimed twice, every alias round-trips, the canonical alias is front(), and only unknown lacks a MIME type. html_output_test now derives its "cannot open" skip from capabilities_by_file_type(...).open instead of a hardcoded list.

Behaviour changes

  • mimetype_by_file_type throws only for FileType::unknown now. wpd, rtf, cfb, ooxml-encrypted and svm gained canonical types; svm's is taken from svm_file.cpp so the static answer matches what the file object reports.
  • file_category_by_file_type(pdf) reported unknown while abstract::PdfFile::file_category() has always said document. It is document now, with document_type text — matching what pdf_file.cpp already puts in file_meta. office_open_xml_encrypted likewise becomes document, and is named ooxml_encrypted instead of unnamed.
  • internal::ImageFile::mimetype() returned ""; it answers from the table.

xlsb gets its own FileType::excel_binary_workbook rather than riding along with xlsx: the package is OOXML but the workbook parts are binary, so there is no decoder. Since capabilities are keyed on FileType, sharing the xlsx row would have made it inherit open/translate_html and led exactly the pickers this PR is meant to serve into offering files that cannot be opened. Its own type carries an all-false capability row.

Bindings

Both layers expose the new API: JNI gets FileTypeCapabilities.java, five natives on Odr and DecodedFile.capabilities(); pyodr gets the matching free functions plus FileTypeCapabilities and DecodedFile.capabilities.

Reference output

Regenerated for the two changed metadata fields (fileCategory on every PDF and the two encrypted OOXML packages; fileType on those two). Every HTML output is byte-identical, so no html-tidy pass was involved.

Testing

  • Full odr_test suite green. The category change surfaced five PDFs whose document_type is unset because the meta parse is all-or-nothing (see pdf/AGENTS.md); the comparison is guarded for that.
  • pytest python/tests: 43/43.
  • JNI compiles on both sides (javac --release 17 -Xlint:all over all of jni/java, and the touched jni_*.cpp against JDK headers). The JUnit suite itself was not run locally — CI covers it.

…ty query

Consumers had to rebuild odrcore's format knowledge by hand — OpenDocument.droid
keeps three separate format tables plus an instrumented test to hold them in
sync. The lookups in `odr.hpp` were one-directional and incomplete, and there
was no way at all to ask what the library can do with a format *before* holding
a file, which is exactly when an app has to decide (intent filters, picker MIME
lists).

Everything lived as six independent if-chains over `FileType` in `odr.cpp`,
which had already drifted apart: `markdown` had a MIME type but no extension,
the `wpd` MIME round trip was broken (`odr_test` documented it), `rtf`/`svm`/
`cfb`/`ooxml_encrypted` had no MIME type at all, there was no
`extension_by_file_type`, and whole alias families were missing.

Collapse all of it into one `constexpr` table in the new
`internal/file_type_table.{hpp,cpp}` — one row per `FileType` holding the name,
every extension, every MIME type, the category, the document type and the
capabilities. Every public lookup is now a thin forward into it; `odr.cpp`
shrinks from 441 to 164 lines.

On top of that:

- `all_file_types`, `file_extensions_by_file_type`, `file_extension_by_file_type`
  and `mimetypes_by_file_type` enumerate what a file type accepts, so a consumer
  never needs its own copy of these tables.
- `FileTypeCapabilities` + `capabilities_by_file_type` answer, per format,
  whether we can detect, open, decrypt, render, edit, save or encrypt it. It is
  explicitly a *declared upper bound*; `DecodedFile::capabilities()` refines it
  for the file in hand, and `Document::is_editable`/`is_savable` remain the
  precise answer.
- The missing aliases: `docm`/`dotx`/`dotm`, `pptm`/`potx`/`ppsx`/`ppsm`,
  `xlsm`/`xltx`/`xlsb`, `dot`/`pot`/`pps`/`xlt`, `md`, `otm`, `ttc`, the ODF
  `x-vnd`/`-template`/`-master`/`-flat-xml` families and the OOXML
  `.template`/`.slideshow`/`macroEnabled.12` families.

A hand-maintained table can lie, so the invariant is a test rather than
discipline: `FileTypeCapabilities.declaration_matches_the_engines` walks the
corpus and asserts that `DecodedFile::capabilities()` is a subset of what the
table declares, and that `Document::is_editable()`/`is_savable(false|true)`
*equal* the declared `edit`/`save`/`encrypt`. Further tests assert that every
`FileType` has exactly one row (derived from the enum, not the table), that no
extension or MIME type is claimed twice, and that every alias round-trips.
`html_output_test` now derives its "cannot open" skip from the table instead of
a hardcoded list.

Behaviour changes:

- `mimetype_by_file_type` throws only for `FileType::unknown` now. wpd, rtf,
  cfb, ooxml-encrypted and svm gained canonical types; svm's is taken from
  `svm_file.cpp` so the static answer matches what the file object reports.
- `file_category_by_file_type(pdf)` reported `unknown` while
  `abstract::PdfFile::file_category()` has always said `document` — it is
  `document` now, with `document_type` `text`, matching what `pdf_file.cpp`
  already puts in `file_meta`. `office_open_xml_encrypted` likewise becomes
  `document` and is named `ooxml_encrypted` instead of `unnamed`.
- `internal::ImageFile::mimetype()` returned `""`; it answers from the table.

`xlsb` is deliberately classified as a workbook (that is its MIME family) even
though there is no decoder for the binary package — a `detect != open` case,
which is what the capability struct exists to express.

Reference outputs are regenerated for the two changed metadata fields; every
HTML output is byte-identical.

Closes #632.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012c8P8ueTw7pfNtCAzzVA8Z

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 46f9d4ad05

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/odr/internal/file_type_table.cpp Outdated
Comment thread src/odr/odr.hpp Outdated
andiwand and others added 2 commits July 30, 2026 21:44
Two review findings on #634.

`xlsb` rode along in the `office_open_xml_workbook` row, so
`file_type_by_file_extension("xlsb")` inherited that row's `open` /
`translate_html` — a caller configuring a file picker from the new tables would
offer xlsb files that then fail to open. The claim that "the capability struct
expresses this" was wrong: capabilities are keyed on `FileType`, so a type
shared with xlsx cannot say anything different. Give it `FileType::
excel_binary_workbook` instead, still classified as a spreadsheet document but
with an all-false capability row.

`odr.hpp` only forward-declared `FileTypeCapabilities`, which made
`odr::capabilities_by_file_type(type)` fail with an incomplete return type
unless the caller also knew to include `<odr/file.hpp>`. Include `file.hpp`
from `odr.hpp` — every lookup there is *about* those types, and two of them
(`FileTypeCapabilities`, `DecodedFile`) are returned by value — and drop the
now-redundant forward declarations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012c8P8ueTw7pfNtCAzzVA8Z
@andiwand
andiwand merged commit ba7d5bd into main Jul 30, 2026
18 checks passed
@andiwand
andiwand deleted the feat/format-tables branch July 30, 2026 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose the supported-format tables: all mime types and extensions per FileType, and what can be rendered

1 participant