Skip to content

fix(html): keep caption, colgroup and tfoot from breaking table headers - #2537

Open
om singhal (Om-singhaI) wants to merge 1 commit into
microsoft:mainfrom
Om-singhaI:fix/html-table-caption-colgroup-tfoot
Open

om singhal (Om-singhaI) wants to merge 1 commit into
microsoft:mainfrom
Om-singhaI:fix/html-table-caption-colgroup-tfoot

Conversation

@Om-singhaI

Copy link
Copy Markdown

Fixes #2536

markdownify only treats a <tr> as the first row when it has no previous sibling. html.parser doesn't add a <tbody>, so when rows sit directly under <table>, a <caption>, <colgroup> or <col> before them drops the header's | --- | line. A <tfoot> row has no previous sibling either, so it's rendered as a header: a <td> footer gets an empty header and delimiter, and a <th> footer gets a delimiter under it.

The check lives in markdownify's convert_tr, so I fixed it in _CustomMarkdownify. A new _normalize_table_sections helper runs first in convert_soup:

  • A direct <caption> moves in front of its table, and a convert_caption override adds a blank line before it so it doesn't glue onto text before it.
  • <colgroup> is unwrapped and <col> dropped, since html.parser can leave an unclosed <colgroup> holding every row.
  • <tfoot> rows move to the end of the last <tbody> (or the table), which also puts HTML4 style footers last.

Testing:

  • 10 new cases in tests/test_html_converter.py fail on main and pass here.
  • The related test files pass (test_html_converter, test_module_vectors, test_module_misc, RSS, EPUB, Office images). Skips need network, an API key or exiftool, and 2 test_docintel_html failures happen on main too.
  • All of tests/test_files converts to identical output before and after.
  • black 23.7.0 is clean.

markdownify only treats a <tr> as the first row when it has no previous
sibling. When the rows sit directly under <table>, a <caption>, <colgroup>
or <col> before them drops the delimiter row. JDK 8 and 11 Javadoc summary
tables are built this way with a <caption>. The first <tfoot> row also
gets a second header block. Reshape those tables before converting, and
keep a blank line before the moved caption so it doesn't join the text in
front of the table.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTML tables lose the header delimiter row after <caption> or <colgroup>, and <tfoot> adds a fake header

1 participant