Skip to content

Fix regression parsing out-of-order tables split by another table - #625

Open
faresrafat3 wants to merge 2 commits into
python-poetry:masterfrom
faresrafat3:tomlkit-571-out-of-order-proxy
Open

faresrafat3 wants to merge 2 commits into
python-poetry:masterfrom
faresrafat3:tomlkit-571-out-of-order-proxy

Conversation

@faresrafat3

Copy link
Copy Markdown

Fixes #571.

tomlkit 0.15.1 rejects a valid document that tomllib and tomlkit
0.13.3/0.14.0/0.15.0 all accept:

import tomlkit
tomlkit.loads("""\
[tool.ruff]
[tool.ruff.lint.a]
[tool.ruff.lint]
[[tool.poetry.source]]
[tool.ruff.lint.b]
""")
# tomlkit.exceptions.ParseError: Key "lint" already exists. at line 5 col 0

All three ingredients are needed: an out-of-order child, its concrete parent
declared afterwards, and a later sibling child separated by an unrelated table.

Root cause

When a key is defined out of order, Container._raw_append records several
body positions for it and Container.item() then returns an
OutOfOrderTableProxy instead of the Table:

if isinstance(idx, tuple):
    # The item we are getting is an out of order table
    return OutOfOrderTableProxy(self, idx)

_validate_table_candidate compared the candidate against that proxy using

if isinstance(existing, (Table, AoT)) != isinstance(v, (Table, AoT)):
    raise KeyAlreadyPresent(k)

isinstance(proxy, (Table, AoT)) is False — the proxy is a separate class —
so a valid table-vs-table extension looked like a table-vs-scalar conflict and
raised KeyAlreadyPresent. Traced at the failure point:

[1] current super=False body=[('lint', 'Table'), ('lint', 'Table')]
    candidate super=True body=[('lint', 'Table')]
    CONFLICT key=lint existing=OutOfOrderTableProxy v=Table

Change

Skip the type comparison when the existing entry is an out-of-order proxy; its
fragments are already validated by the proxy. One condition, three lines of
comment.

Verification

The reported document now parses and round-trips byte-identically, and its
unwrap() matches tomllib exactly:

>>> tomlkit.loads(src).unwrap() == tomllib.loads(src)
True
>>> tomlkit.loads(src).as_string() == src
True

Genuine duplicates are still rejected — checked explicitly, each raising:

input result
[a]\n[a]\n ParseError
[a]\nb=1\n[a]\n ParseError
[a]\n[a.b]\n[a.b]\n KeyAlreadyPresent
a=1\n[a]\n ParseError
[a]\n[[a]]\n ParseError
[a]\nb.c=1\n[a.b]\n TOMLKitError (dotted redefinition)

Two regression tests added to tests/test_toml_document.py (the issue's
document, and a deeper variant without the array-of-tables). Full suite:
390 passed (388 before).

faresrafat3 added 2 commits October 1, 2026 15:37
Building the docs with `sphinx-build -n` reports many
"py:class reference target not found" warnings for annotations such as
collections.abc.Mapping, datetime.datetime and collections.abc.Sequence,
because no intersphinx mapping is configured (see python-poetry#192).

Add the sphinx.ext.intersphinx extension with the Python inventory.

Warnings under -n drop from 55 to 31. The remainder are project-internal
references (tomlkit.items.ItemT, tomlkit.container.Container, E, Encoder)
that are not part of this change.

Fixes python-poetry#192 (partially).
A document where a table has an out-of-order child, its concrete parent is
declared afterwards, and a later sibling child is separated by an unrelated
table was rejected with 'Key already exists', although tomllib and tomlkit
0.13.3/0.14.0/0.15.0 all accept it:

    [tool.ruff]
    [tool.ruff.lint.a]
    [tool.ruff.lint]
    [[tool.poetry.source]]
    [tool.ruff.lint.b]

When a key is defined out of order, Container.item() returns an
OutOfOrderTableProxy rather than the Table itself, because the key maps to
several body positions. _validate_table_candidate compared the candidate
against that proxy with isinstance(existing, (Table, AoT)), which is False
for a proxy, so a valid table-vs-table extension was reported as a
table-vs-scalar type conflict and raised KeyAlreadyPresent.

Skip the type comparison when the existing entry is an out-of-order proxy;
its fragments are validated by the proxy itself.

Fixes python-poetry#571.
Copilot AI balanced review requested due to automatic review settings October 1, 2026 13:36

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@dimbleby

dimbleby commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Explain why not #572

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regression in 0.15.1: valid out-of-order child table raises KeyAlreadyPresent

3 participants