Skip to content

fix(html): keep a pipe in a table cell from adding a column - #2520

Open
Lukas (L4XB) wants to merge 1 commit into
microsoft:mainfrom
L4XB:fix/table-cell-pipe
Open

Lukas (L4XB) wants to merge 1 commit into
microsoft:mainfrom
L4XB:fix/table-cell-pipe

Conversation

@L4XB

Copy link
Copy Markdown

The bug

A Markdown table row is split on every unescaped pipe. _CustomMarkdownify inherits markdownify's convert_td/convert_th, which write the cell's text through unchanged, so a cell that holds a pipe silently adds a column.

On main (eb31b5c), an .xlsx with a part number in it:

| Product | Spec |
| --- | --- |
| Cable | USB-A|USB-C |
| Hub | 4 ports |

The header has 2 columns and the Cable row now has 3. Read back, the row is Cable / USB-A / USB-C4 ports is still in column 2 while USB-C has moved to a column that does not exist, so the table no longer lines up from that row on.

It is not an exotic cell value. A pipe shows up in part numbers, shell commands (grep -E 'a|b'), regex alternations, "either/or" notes, and anything pasted out of a terminal.

This reaches Markdown through the shared HTML converter, so every format that routes through it is affected:

format path
.html directly
.docx mammoth → HTML → _CustomMarkdownify
.xlsx / .xls sheet_to_html_CustomMarkdownify
.epub, .ipynb HTML output, RSS same converter

The fix

convert_td/convert_th escape the pipes in the cell's text before markdownify lays the row out:

_TABLE_CELL_PIPE = re.compile(r"(?<!\\)(\\*)\|")

def _escape_table_cell(text: str) -> str:
    return _TABLE_CELL_PIPE.sub(lambda match: match.group(1) * 2 + r"\|", text)

The regex matches the run of backslashes in front of the pipe rather than the pipe alone. That is the part that matters: a cell already containing a\|b would otherwise come out as a\\|b, where the cell's own backslash escapes the one we added and the pipe splits the row anyway. Doubling the run first makes a\|b into a\\\|b, which reads back as the literal a\|b.

A pipe outside a table is untouched — only the two cell converters call it.

Tests

packages/markitdown/tests/test_table_cell_pipe.py. The helper reads the produced table back the way a Markdown reader does (split on unescaped pipes, then resolve backslash escapes) and asserts on the cells, not on a fixed string, so it tests the property rather than the formatting.

9 cases: a part number, a shell command, a doubled ||, a cell with its own a\|b, a header cell, an .xlsx end to end, plus 3 guards — C:\path, a plain cell, and a pipe in a paragraph outside any table.

Measured:

result
with the change 9 passed
source change stashed, tests kept 6 failed, 3 passed

Full suite: 851 passed, 14 skipped (unchanged). black clean.

Note: this touches the same file as #2519, which adds a URL helper near the top. They are independent changes; I will rebase whichever one lands second.

A Markdown table row is split on every unescaped pipe, and markdownify writes
a cell's text through unchanged. A part number, a shell command or a regex
alternation in a cell therefore pushes the rest of the row into columns the
header does not have:

    | Product | Spec |
    | --- | --- |
    | Cable | USB-A|USB-C |

Escape the pipes in a cell before markdownify lays the row out. A backslash
already in the cell is doubled first, so it cannot consume the escape.

This runs for every format that reaches Markdown through the HTML converter,
including .html, .docx (mammoth) and .xlsx (sheet_to_html).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant