fix(html): keep the rows under a rowspan in their own columns - #2522
Open
Lukas (L4XB) wants to merge 2 commits into
Open
Lukas (L4XB) wants to merge 2 commits into
Lukas (L4XB) wants to merge 2 commits into
Conversation
A Markdown table has no way to merge cells down, so a cell carrying a rowspan
is written once and every row it reaches into comes out one cell short:
| Region | Product | Units |
| --- | --- | --- |
| EU | Cable | 12 |
| Hub | 7 |
Read back, `Hub` sits under Region and `7` under Product, so the value is
attributed to the wrong column and Units is empty. A Word table with a
vertically merged cell produces exactly this: mammoth writes the merge as
rowspan="2".
Fill the rows a span reaches into with the empty cells it stands for, before
markdownify lays the table out.
Two costs in the rowspan padding, both reachable from any HTML input: - A span asked for its placeholders unbounded. One cell with rowspan="1000" colspan="1000" over 999 more rows requested a million of them: 19 KB of HTML became 3 MB of Markdown in 20 s. A rowspan now reaches no further than the table's last row, and a table gets placeholders only while they stay within 64 + 8 per real cell; past that it is converted as markdownify converts it, without the padding. The same input now converts in 0.05 s. - Each placeholder was inserted with insert_before, which finds its anchor by scanning the siblings, so many placeholders in one row cost O(n^2): 30,000 in front of one cell took 12 s. The row is now rebuilt in one pass, taking its children out front to back (every extract() finds its node at index 0) and putting them back with the placeholders in place: 1 s. Tables within the budget, including every existing case, come out the same.
Author
|
Pushed 1538d62 after reviewers of the same padding logic elsewhere (QwenLM/qwen-code#12039, LearningCircuit/local-deep-research#6520) measured two costs that apply here too. Both are reachable from any HTML input:
Two tests added: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
A Markdown table has no way to merge cells downward, so a cell carrying a
rowspanis written once and every row it reaches into comes out one cell short. The rows below it then read one column out of step.Measured on
main(eb31b5c):Read back,
Hubsits under Region and7under Product.Unitsis empty for that row, and the count7is silently attributed to the wrong column — which is worse than losing it, because nothing about the output looks wrong.A Word table with a vertically merged cell produces exactly this. Measured end to end on a
.docxbuilt withpython-docx(table.cell(1, 0).merge(table.cell(2, 0))), mammoth writes the merge asrowspan="2":markdownify pads a
colspanbut has nothing forrowspan, so this reaches.docx,.html,.epuband RSS alike. A region spanning several product rows, a category spanning its items, a date spanning a day's entries — merged first columns are ordinary in real documents.The fix
convert_soupfills the rows a span reaches into with the empty cells it stands for, before markdownify lays the table out:_fill_row_spanswalks each table once and builds the grid the way HTML defines it — a cell takes the next column not already taken by a span from above, then claimscolspancolumns across androwspanrows down. Each row that ends up with claimed columns gets an empty<td>at each of them.Two details worth calling out:
_spanclamps to1..1000, the same bounds markdownify already applies tocolspan, so a hostilerowspan="999999"cannot blow up the row count.A table with no
rowspanis untouched, andcolspankeeps being handled by markdownify exactly as before.Tests
packages/markitdown/tests/test_table_rowspan.py, 7 cases, all reading the produced table back the way a reader does and asserting on the cells:rowspan="3"filling both rows below;colspan-only table;.docxwith a real vertically merged Word cell.Measured:
Full suite:
849 passed, 14 skipped, no failures.black --checkclean.