Skip to content

Flush the real streams before each B0.2 test captures output - #596

Merged
MaxGhenis merged 1 commit into
masterfrom
coord-b02-flush-real-streams-20261102
Nov 2, 2026
Merged

MaxGhenis merged 1 commit into
masterfrom
coord-b02-flush-real-streams-20261102

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

What fails

On #574, one Tests shard fails at
tests/test_track_b_b0_2_counts.py::test_a_failure_from_outside_this_script_prints_no_value.
The test asserts that capfd holds ("", "") after script.main stops, and
it finds pytest-split's header instead:

At index 0 diff: '\n\n[pytest-split] Splitting tests with algorithm: duration_based_chunks\n[pytest-split] Running group 3/4 (estimated duration: 3249.47s)\n\n' != ''

Why

  • pytest-split 0.11.0 writes its header with self.writer.line(...), a
    terminal writer over sys.stdout (pytest_split/plugin.py:124 and
    :171–179). When stdout is a pipe, as in CI, that text waits unflushed in the
    buffer of sys.__stdout__.
  • The B0.2 script's _flush_streams (scripts/track_b_b0_2_counts.py:2340)
    flushes sys.__stdout__ and sys.__stderr__ before _silenced redirects
    fds 1 and 2.
  • With -q, as CI runs pytest, nothing flushes that buffer before the first
    test's call report (_pytest/terminal.py:699). At default verbosity the
    test passes at master with pytest-split.
  • So when this test runs first in its shard, the script's flush pushes the
    header into fd 1 while capfd holds it, and the test captures text it never
    wrote. In Aggregate-table captures and readers, part 1 (D4) #574's run (job 125184609985) it ran first in shard 3 of 4: that
    shard's progress line opens with F. Which test runs first depends on the
    shard split, so another change to the test set can move this one back to the
    head of a group.

Fix

An autouse fixture in the module flushes sys.__stdout__ and sys.__stderr__
before each test starts capturing. The script and its assertions don't change.
The fixture adds no tests, so the tier files don't change.

Checks run

  • Real pytest-split 0.11.0 with the test first in group 2 of 2 fails at
    master's test file with the header diff above, and passes with the fixture.
  • A minimal plugin that writes to sys.stdout at collection without flushing
    fails the test at master and passes with the fixture.
  • Without any plugin the test passes at both.
  • Full module run, serial: 247 of 248 passed under taskpolicy -b. The one failure, test_the_host_reads_see_a_background_policy_and_a_qos_clamp[policy1-qos], comes from that wrapper: it fails the same way at master under taskpolicy -b, and it passes at both master and this branch in a plain run.

Review

An independent Claude Opus 5.5 review returned APPROVE. It reproduced the
failure and the fix under real pytest-split, found the same latent failure in
test_relabeling_a_death_and_a_move_changes_no_written_byte[ages1-before1-0-7-released]
when that test runs first, and showed that the fixture runs before capfd
starts: moved after it, the target fails again. Five injected leaks from the
script still fail the target with the fixture in place. This PR's own shard
split doesn't put the target first, so its CI doesn't exercise the fix;
#574 does once it merges this.

🤖 Generated with Claude Code

pytest-split writes its header to sys.stdout through the terminal
writer and doesn't flush it when stdout is a pipe, as it is in CI. The
B0.2 script's _flush_streams flushes sys.__stdout__, so the first test
in a shard that runs it under capfd captured the header and failed its
empty-output assertion. That test is
test_a_failure_from_outside_this_script_prints_no_value. An autouse
fixture now flushes sys.__stdout__ and sys.__stderr__ before each test
in the module, so no earlier writer's text reaches a test's capture.

The fixture adds no tests, so the tier files don't change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
social-security-model Ready Ready Preview Nov 2, 2026 2:52pm UTC

Request Review

@MaxGhenis
MaxGhenis marked this pull request as ready for review November 2, 2026 15:35
@MaxGhenis
MaxGhenis merged commit dd9c7be into master Nov 2, 2026
12 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

The Microcosm Dynamics coordinator merged this PR at 562d0d7087e43e07c5af4de4348261de82cabfad with --merge --match-head-commit, as dd9c7be24b7228c17123a44d1fd1150d4335312f (16:52Z, 2026-11-02).

@MaxGhenis
MaxGhenis deleted the coord-b02-flush-real-streams-20261102 branch November 2, 2026 16:53
MaxGhenis added a commit that referenced this pull request Nov 2, 2026
Master adds the B0.2 stream flush (#596) and no test, so the recount
on 89d064b stands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Nov 2, 2026
Master brings the B0.2 stream flush (#596), which touches one B0.2 test file
and no Track B payment file. The tier files take master's counts plus this
branch's new tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Nov 2, 2026
Master gained #563's last commits (dbdceac, f5a18c8) and #596; the merge is clean, and neither side's tier files conflict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 562d0d70 Deployed Nov 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant