[ryanchou1994]

Summary

Since #567, an emptied array of tables is rendered as key = [] at the top of the nearest header scope. If a table on its path is declared by a header in a later fragment ([[a.b.c]] ... [a], or [a.b]), the hoisted dotted key a.b.c = [] declares that table before its header does, which is invalid TOML (both tomlkit and tomllib reject the output).

This change only affects rendering in tomlkit/container.py: before writing a fallback, _declaring_table looks for the deepest table on its path that emits a header in the current scope. If there is one, the fallback is handed to that table and written at the top of its own scope with a relative key, e.g.

[a]
b.c = []
y = 1

When the out-of-order table is split into fragments, the scope holding all of them hands the fallback over before anything is rendered, so the declaring header can also be in another fragment (an earlier one, or a deeper header in a later one).

Fallbacks with no declaring header are written exactly as before, and parsing is untouched.

Tests: three parametrized groups (13 shapes, also checked with tomllib on 3.11+) and three exact-output tests, covering a declaring header in a later fragment, one in an earlier fragment, and a deeper header in another fragment (the last two are the cases reported in the comments below). All 16 fail with master's container.py and pass with the change. Full suite including the toml-test submodule on Python 3.14 (macOS): 1450 passed. ruff check and ruff format --check are clean.

Fixes #690

Duplicate Check

• I searched the open PRs and found none that already fixes this issue.
• Related open PRs (if any, and why this PR is still needed): none touch the emptied-AoT fallback added in Render an emptied array of tables as an empty array #567; Render the header of emptied super tables created from dotted keys #590 and Render each run of a split array of tables where it was written #601 also render out-of-order tables but address different cases.

Agent Drafting Metadata

• Agent: Claude Code
• Model: Claude Opus 5.5
• Notes: The fix and tests were written with Claude Code, and I decided to submit it. Before opening the PR it was re-run in a clean Linux container (Python 3.14, 1223 passed at the time), and two extra test shapes were added after a review pass. The follow-up commits for the cases reported in the comments were also written and tested with Claude Code.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

[Ekko82]

Checked 0c0b3867 on Windows with Python 3.13.9. All 98 tests in tests/test_toml_document.py pass. The six added regression cases fail against base d7def7d and pass on this head; the five existing emptied-AoT cases pass on both.

Twelve additional parse/pop/dump cases also pass, covering equivalent quoted keys, deeper declaring headers, multiple emptied siblings, nested array elements, and controls without a declaring header or with a nonempty AoT. Both tomllib and tomlkit recover the mutated document's data, and rendering is unchanged after a second call or a deepcopy. Nine of these cases produce invalid TOML on the base.

[lexingtonhibiki]

One header-order case still produces invalid TOML on 0c0b3867:

import tomlkit
import tomllib

source = "[a.b]\nz=1\n[a]\ny=1\n[[a.b.c]]\nx=1\n"
tomllib.loads(source)  # valid input

doc = tomlkit.parse(source)
doc["a"]["b"]["c"].pop()
rendered = doc.as_string()
print(rendered)
tomllib.loads(rendered)

The output is:

[a.b]
z=1
[a]
b.c = []
y=1

tomllib rejects this with Cannot redefine namespace ('a', 'b'); tomlkit also rejects it. The c = [] fallback needs to go under the earlier [a.b] header. In this shape that header lives in a different a fragment from the emptied AoT, while the _scope_aots() call for the [a] fragment only searches that fragment's body.

This is an uncovered case, not a regression introduced by the PR: it also fails on base d7def7d. I checked five standalone parse/pop/dump cases on Python 3.12.14: the original PR case and two controls pass on this head; this case and a deeper variant with a separate declaring-header fragment still fail. Repeated rendering and deepcopy retain the same output. I have not run the full upstream suite. This was AI-assisted verification, not human review.

[ryanchou1994]

Thanks for the repro. I can reproduce it on 0c0b386 and on the base too, so it's the same fallback with the declaring [a.b] header sitting in an earlier fragment of a. I'll extend the lookup to cover that, add this case and the deeper variant to the tests, and push here.

[ryanchou1994]

Pushed 4d0be60. The scope that holds all the fragments of a now hands the fallback to the deepest header declaring its parent before anything is rendered, so your example comes out as

[a.b]
c = []
z = 1
[a]
y = 1

Added your case, the deeper [a.b.d] variant, and one with [a.b] between [a] and [[a.b.d.e]] to the tests. The full suite and toml-test pass on Python 3.14 (Linux). A randomized run over ~40k header orders (among inputs tomlkit parses) also turned up no remaining output that tomllib rejects; 0c0b386 had 45 such cases.

[Ekko82]

Rechecked 4d0be60 on Windows with Python 3.13.9. The reported earlier-[a.b] case now emits c = [] under that header, and the deeper variants round-trip correctly through tomllib and tomlkit.

All 102 tests in tests/test_toml_document.py pass. The four cases added in this update fail against 0c0b3867; the other 11 emptied-AoT cases pass there.

Twenty standalone parse/pop/dump probes also pass, including eight earlier-fragment shapes that fail on the old head. These cover quoted keys, multiple emptied siblings, separate outer-AoT elements and retained nonempty siblings. Both parsers recover the mutated data; repeated rendering and deepcopy preserve the output.

[manav4u]

One deeper-header case still emits invalid TOML on b359a20:

import tomlkit
import tomllib

source = "[a.b]\n[[a.b.c.d]]\n[a]\n[a.b.c]\n"
tomllib.loads(source)  # valid input

doc = tomlkit.parse(source)
doc["a"]["b"]["c"]["d"].pop()
rendered = doc.as_string()
print(rendered)
tomllib.loads(rendered)

The output is:

[a.b]
c.d = []
[a]
[a.b.c]

Both tomllib and tomlkit reject this: the fallback under [a.b] declares a.b.c before the later [a.b.c] header. The d = [] fallback needs to be under that deeper header instead. Removing either the initial [a.b] or the intervening [a] makes this case pass on the current head.

This also fails on base d41d237 and current master 57638ed, so it is an uncovered existing case, not a regression introduced by this PR. Quoted-key and one-level-deeper variants reproduce it; both pop() and clear() behave the same. Rendering twice and after deepcopy produces the same invalid output.

Checked on Linux / CPython 3.12.15. The current head passes all 1,444 upstream tests, including the toml-test submodule. This was AI-assisted verification, not independent human review.

[ryanchou1994]

@manav4u Thanks, that's a real gap. The cross-fragment lookup only ran when the repeated key had a header of its own, so the first fragment of a here (a super table holding [a.b]) was never searched, and my earlier note about ~40k random header orders coming up clean was too broad.

Pushed b131faf, which searches every fragment of a repeated key. Your example now renders as

[a.b]
[a]
[a.b.c]
d = []

It's in the tests now along with the quoted-key and one-level-deeper variants, one with [x] in place of [a] (so no fragment of a has a header), and one with [a.b.c] coming first. The full suite with toml-test passes on Python 3.14 (macOS). A new randomized run over about 73k emptied arrays found no more output that tomllib rejects, which is still only a sample; it did find a few outputs that tomllib accepts but tomlkit's own parser can't read back, two of which fail even with the = [] line removed and all of which fail the same way on b359a20, so I've left those out of this PR.

[human-attestation]

✅ Human Attestation: @ryanchou1994 attested this PR at commit b131faf (2026-11-11T13:47:31.348Z).

I confirm the code in this PR was written by a human. / 我确认此 PR 中的代码均由人类编写。

Later pushes to this PR stay attested. Attestation page

[Ekko82]

Checked b131faf on Windows with CPython 3.13.9 and pytest 8.4.2. The deeper-header example now places d = [] under [a.b.c]. All six saved pop()/clear() report and control cases preserve the expected data through both tomllib and tomlkit, with stable repeated rendering and deepcopy.

All 163 tests in tests/test_toml_document.py and the 20 previously checked public-API probes pass on this head.