fix(docx): read the text inside a DrawingML text box - #2514
Open
Lukas (L4XB) wants to merge 1 commit into
Open
Lukas (L4XB) wants to merge 1 commit into
Lukas (L4XB) wants to merge 1 commit into
Conversation
A text box keeps its own paragraphs in a `w:txbxContent`. Mammoth reads that
element, but only reaches it through the legacy VML path
(`w:pict` -> `v:shape` -> `v:textbox`). A modern text box is a DrawingML shape
instead -- `w:drawing` -> `wp:inline` -> `wps:wsp` -> `wps:txbx` -- and
`wp:inline` is read as a picture, so the shape's text is dropped with no warning
and nothing in the messages. Callouts, pull quotes, sidebars and diagram labels
simply are not in the output.
Measured on a document whose only text box is a bare `w:drawing`:
before: 'PARAGRAPH TEXT\n\nAFTER'
after: 'PARAGRAPH TEXT\n\nCALLOUT\n\nAFTER'
This is the same kind of repair the file already performs for Mammoth: a copy of
the `w:txbxContent` is inserted after the `w:drawing` that holds it, wrapped in
`w:pict`/`v:shape`/`v:textbox`, and the original is removed so the text cannot be
read twice. Mammoth treats a `w:pict` as extra content following the paragraph it
sits in, which is where a reader of the page sees the text box anyway.
Text boxes inside `mc:AlternateContent` are left alone. Word writes the same text
twice there -- a DrawingML shape under `mc:Choice` and a VML shape under
`mc:Fallback` -- and Mammoth reads the fallback, so promoting the choice as well
would duplicate it. The second test pins that it appears exactly once.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A text box keeps its own paragraphs in a
w:txbxContent. Mammoth reads thatelement — it is in its handler table — but only reaches it through the legacy VML
path,
w:pict→v:shape→v:textbox. A modern text box is a DrawingMLshape instead:
and
wp:inlineis handled as a picture: it looks for ana:blip, finds none,and returns nothing. The shape's text is dropped with no exception, no warning,
and nothing in
result.messages. Callouts, pull quotes, sidebars and diagramlabels are simply absent from the Markdown.
Measured on a document whose only text box is a bare
w:drawing:PARAGRAPH TEXT\n\nAFTERPARAGRAPH TEXT\n\nCALLOUT\n\nAFTERWhat changed
One more step in
pre_process_docx, which is exactly what that module is for —it already renames
w:dstriketow:strikeand repairsw:styleelementsbecause "downstream converters do not recognize" them.
A copy of the
w:txbxContentis inserted after thew:drawingthat holds it,wrapped in
w:pict/v:shape/v:textbox, and the original is removed so the textcannot be read twice. Mammoth treats a
w:pictas extra content that follows theparagraph containing it, so the text lands right after the paragraph the box is
anchored in — where a reader of the page sees it — rather than appended at the
end of the document.
Text boxes inside
mc:AlternateContentare deliberately left alone. Wordwrites the same text twice there: a DrawingML shape under
mc:Choiceand a VMLshape under
mc:Fallback. Mammoth reads the fallback branch, so promoting thechoice as well would emit the text twice. That is also why this has gone
unnoticed — a file saved by Word usually carries the fallback, while a text box
written as a bare
w:drawing(which is valid OOXML and what several generatorsand exporters produce) has nothing for Mammoth to fall back to.
No new dependency: the step uses the same BeautifulSoup parse the neighbouring
steps use, and returns the input untouched when the document contains no
txbxContentat all — the same fast path_pre_process_striketakes.Tests
packages/markitdown/tests/test_docx_text_boxes.pybuilds its fixtures byinjecting a text box into the committed
test.docx, the waytest_docx_styles.pydoes:w:drawingtext box appears once in the outputmc:AlternateContenttext box appears exactly once, not twiceThe two that pass on both sides are the guards: no duplication, and no change to
documents without a text box.
black(23.7.0, the pinned pre-commit version) leaves both files unchanged.