Skip to content

Stage 5: use the alternate screen for the built-in pager in reserved rendering - #1815

Draft
tleonhardt wants to merge 20 commits into
reserved_row_toolbarfrom
stage5-pager-investigation
Draft

tleonhardt wants to merge 20 commits into
reserved_row_toolbarfrom
stage5-pager-investigation

Conversation

@tleonhardt

@tleonhardt tleonhardt commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Stage 5: the built-in pager, investigated and repaired

Stage 5 asked whether the built-in pager is still worth keeping now that reserved rendering exists. The spike compared it with an external pager, and the disposition, accepted by the maintainer on 2026-11-15, is keep it and repair it.

This PR is the reserved-rendering part of that repair, plus what the Windows check of it found. The rest went where it belongs:

What Where
Print ppaged() output when the pager program cannot be run #1813, to main
The pager as its own PR, split out of #1745 #1814, stacked on #1745
Parse paged output a line at a time (first page for 100,000 lines: 1.9 s to 0.03 s) #1814; already merged into this branch through reserved_row_toolbar
Enter the alternate screen in reserved rendering this PR (9993b724)
Five defects found while checking it, none specific to the pager this PR (dc84f6c6, 3f349136, fb041f6b, 8e37a0b1, 79c44604 with 6eeecab6)
Keep the pager's pages out of iTerm2's scrollback #1814 (242d9d96); merged into this branch, with a test of it on a reservation (d181692c)

The pager defect

In reserved rendering the pager was drawn on the main screen. One pager session sent no switch to the alternate screen and one switch back; legacy rendering sends one of each. The pager painted over earlier output and left a page of paged rows, its own key summary and a second toolbar row in the scrollback. This has been so since Stage 3.

Cause. A renderer enters the alternate screen as the first thing in a full-screen frame, and marks itself as there before asking the output to switch. Under reserved rendering a frame is recorded and emitted afterwards, and the recorder refused to record the switch. The frame failed to prepare, the renderer went on believing it had switched, and the next frame was drawn without it.

The fix (9993b724)

  • RecordingOutput.enter_alternate_screen() records the switch, so it is emitted in the same batch as the frame drawn after it: both reach the terminal or neither does.
  • A frame which is not emitted has not switched screens. PromptToolkitBridge puts the renderer back to believing what it did before preparing that frame, and does not keep the screen it laid out as a baseline (_forget_screen_switch()).
  • Leaving the alternate screen stays unrecordable: a renderer does that when it is reset, never inside a frame.
  • commit()'s four "is this frame still current" checks moved into _still_current() so the rollback is written once. No change in what they check.

This also covers the pager coming back from Ctrl-Z, where the renderer leaves the alternate screen to lend the terminal and has to enter it again.

Found while checking it, and fixed here

None of these is specific to the pager. Each has a test which fails without its change.

Commit Defect Fix
dc84f6c6 Windows: command output wider than the terminal lost everything past the last column in reserved rendering. Legacy rendering wrapped it. prompt-toolkit's Windows output sets a console mode without end-of-row wrap around each flush, and its own stdout proxy turns autowrap on before each write for that reason. SerializedTerminalWriter takes that proxy's place and now does the same, for that backend only. A VT backend is sent exactly what it was before.
3f349136 A managed write landing while a frame is being prepared left the renderer's baseline at the dropped frame. The next frame was compared with a screen the terminal never received and could draw nothing. prepare() puts back the screen and cursor the renderer had before it prepared the frame.
fb041f6b A managed write landing after a frame is prepared and before it is emitted put the screen back but not the cursor. The next frame moved from where the dropped frame would have left the cursor, and was drawn that many rows off. A prepared frame carries where the cursor was before it, and the write which drops the frame puts it back.
8e37a0b1 The reserved toolbar was plain text; in legacy rendering it is a reverse-video bar. So since Stage 3, and on every platform. The painter resolves styles as an application does, with the theme laid over prompt-toolkit's default style, and gives the toolbar's text the classes prompt-toolkit's own toolbar window gives it. Theme rules for bottom-toolbar and bottom-toolbar.text now reach the text as well as the padding.
79c44604, 6eeecab6 Resizing the window while a command is printing damaged its output in reserved rendering. Seen in iTerm2: a line was printed over the old toolbar and the rest of the toolbar stayed in the output, and growing the window erased a line of output. Legacy rendering has neither. Three causes. A terminal drops the scroll region when it is resized, and the display, off the main thread, learned of a resize only from a poll or its next frame: the main thread now passes the window-change signal on. reconfigure() cleared the old toolbar at the rows it had been painted on, which iTerm2 has moved: it now clears everything below the cursor's row. And when output has just moved on to the old toolbar's row, that row is cleared too.

What to review

  1. Recording the switch at all. The recorder's rule was that screen-buffer transitions are ownership boundaries taken before a frame is prepared. Entering is now the exception, on the argument that the renderer has already laid the frame out for the whole terminal and that the reservation was released before the frame was prepared (page() suspends it first). Is that argument sound, or should the switch be taken explicitly before the first frame instead? That alternative would blank the screen for the time it takes to prepare the first frame. On Windows, mintty's log of what it was sent shows one enter and one leave for each pager session.
  2. A frame which fails part-way through being emitted keeps the renderer believing it is on the alternate screen. The switch is the first operation in a frame, so it has probably happened, and the wrong belief in that direction only costs one extra switch back.
  3. The resize change clears more than it did. After a resize during a command, every row below the cursor's is erased, and the cursor's own row too when the last command output ended a line. That is right for output which goes on from the cursor. A command which had drawn below the cursor and moved back up loses those rows until it draws them again, and one whose child process left a line unfinished without cmd2 seeing it loses that line.
  4. A signal handler during commands. While a display holding a reservation runs, the main thread has a SIGWINCH handler which asks the application for a frame and calls the handler which was there before. Is anything a cmd2 application does with that signal at odds with it?
  5. The toolbar's look changes. Anyone who has been running a reserved toolbar has seen plain text; it is now a bar, as in legacy rendering. A theme can still say otherwise.

Evidence

At the current head, 6eeecab6, on macOS:

  • make check, make test (3,896 passed, 7 skipped) and make docs-test.
  • Functional harness, reserved rendering: 126 passed with 4 known limitations (two-line toolbars, which are Stage 7, and an emulator artifact).
  • Continuity and dynamic gates at 24 rows: PASS.
  • The toolbar row in a pseudo-terminal, at 8e37a0b1: 80 of 80 cells in reverse video at the prompt, during a command and at the next prompt, in both renderings. Before that commit it was 0 of 80 in reserved rendering.

On Windows, at 9993b724 plus dc84f6c6 and 3f349136: see the results comment. examples/stage5_pager_check.py was run in Windows Terminal, conhost and mintty, in reserved and in legacy rendering. The built-in pager passes the whole checklist in all six, and its first page for 100,000 lines takes 0.06 to 0.08 s.

In iTerm2 3.7.7 on macOS, in a real window driven by AppleScript, with its screen and scrollback read back and screenshots looked at:

  • The pager checklist of examples/stage5_pager_check.py, in reserved and in legacy rendering: paging, search, colors, chopped lines and sideways scrolling, wide characters, 100,000 rows (first page after 0.02 s), a resize while open, Ctrl-C, and Ctrl-Z with fg. All pass in both, apart from what the first iTerm2 finding below leaves behind. The two renderings behave the same.
  • The reserved toolbar away from the pager: it stays on the bottom row at every sample while output scrolls, no toolbar row reaches the scrollback, and output is complete and in order. Resizing at the prompt, unfinished lines, a Rich progress bar, read_input(), select(), less run from the shell, Ctrl-Z during a command, typeahead and quitting all pass. The whole window is the shell's again while the application is stopped and after it quits.
  • The toolbar is a reverse-video bar in both renderings.
  • Two things did not pass at 8e37a0b1, and both are changed since. iTerm2 kept a page of the pager in its scrollback each time the pager closed, was resized or was stopped with Ctrl-Z, in both renderings: one session with three resizes and a Ctrl-Z left 226 paged rows before 242d9d96, measured on Add a built-in pager which keeps the bottom toolbar visible #1814's branch, and none after it there or here in either rendering. And resizing while a command printed damaged the output in reserved rendering; see the table above and the last item under "Found, not fixed here".

In four more terminals, at 6eeecab6, each driven for real (typed into, resized, and its screen and scrollback read back): Terminal.app and tmux 3.8 on macOS, and tmux 3.3a and xterm 379 on Linux in a container, xterm on a virtual X display. "Before" is 8e37a0b1, the commit before the pager and resize changes.

Runs in which resizing the window while a command printed damaged its output:

Terminal 4 lines a second, before after legacy 200 lines a second, before after legacy
iTerm2 3.7.7 5 of 8 0 of 4 0 of 4 4 of 4 2 of 8 0 of 4
Terminal.app 3 of 4 0 of 4 0 of 4 4 of 4 0 of 8 0 of 4
tmux, macOS 2 of 4 0 of 4 0 of 4 4 of 4 0 of 8 0 of 4
tmux, Linux 2 of 4 0 of 4 0 of 4 4 of 4 0 of 8 0 of 4
xterm, Linux 3 of 4 0 of 4 0 of 4 4 of 4 0 of 8 0 of 4
  • So the damage was never iTerm2's alone: every terminal tried drops the scroll region when it is resized. Where it grew, the erased line of output was seen in iTerm2, tmux on Linux and xterm.
  • The whole reserved toolbar scenario (output scrolling past the toolbar, resizes during a command and at the prompt, unfinished lines, a Rich progress bar, read_input(), select(), less from the shell, Ctrl-Z during a command, typeahead, quitting) passes in all four, as it does in iTerm2.
  • The pager: none of the four ever kept its pages in the scrollback, before or after 242d9d96, so that was iTerm2's alone, and the change makes no difference to them. Paging, three resizes while open, Ctrl-Z and fg, and quitting behave the same before and after.
  • One new thing does not pass, on Linux; see "Found, not fixed here".

At 9993b724, before the four later commits, and not repeated since:

  • In tmux, which has a real alternate screen and scrollback: reserved rendering matches legacy. The pager is on the alternate screen, earlier output is intact, and no paged rows, key summary or toolbar row are left behind.
  • Continuity and dynamic gates at 12 and 40 rows as well as 24: all PASS.
  • Functional harness, legacy rendering: 130 passed.
  • Real-pty Ctrl-Z/fg in the pager passes in both modes.
  • In a Linux container: the suite passes, the tmux checks give the same result as on macOS, and the functional harness gives reserved 126 passed with the same 4 known limitations and legacy 129 passed with 1 known (the legacy flicker).

New tests for the alternate screen: the recorder records and replays the switch; a full-screen frame through the bridge emits it once, before its content; a frame retired by a managed write or one whose preparation raised leaves the renderer not switched, and the next frame switches; the pager on a reservation sends one enter before its first page and one leave after; and after the terminal is lent during paging, two of each.

Still open for the stage

The Windows check this section used to ask for is done, and the maintainer has since tested the built-in pager by hand in Windows Terminal, with examples/stage5_pager_check.py and its rows command, and it looked good.

The later commits have since been run on Windows as well, at 6eeecab6, in Windows Terminal, conhost and mintty: see the resize results comment. The resize checks from Stage 4 were repeated there in reserved rendering before and after 79c44604, and in legacy rendering. The pager checklist was run again and gives the same results as it did before fb041f6b, 8e37a0b1 and 242d9d96, apart from how far more had paged in mintty's legacy run. The clearing rule makes nothing worse on Windows. Resizing at a prompt, with text typed, in read_input(), read_secret() and select(), around a handoff and in the pager passes in all three terminals, apart from what is listed below. Resizing while a command prints 4 or 20 lines a second damaged its output in 13 of 18 runs before these changes and in 0 of 18 after them.

The three things this section then listed as not looked at on Windows have been run as well, at 49c18d0c: see the remaining checks comment. A real pointer, moved and pressed by program, dragged the edges and the corner of each window, turned the wheel and selected text. The application was started in windows of 3 and of 2 rows in Windows Terminal, Windows Terminal Preview 1.26, conhost and mintty, in both renderings, and passes in all sixteen. Preview gives the results of stable, step for step, and its pager checklist has no value which differs. In the legacy console, where prompt-toolkit uses Win32Output, auto selects legacy rendering, every resize check passes with no damage to output at any rate, and the built-in pager passes its checklist; asking for reserved there raises the ValueError meant for it. Nothing new is wrong with the pager, with the reserved toolbar at a prompt, or with small windows. What that run found is in the list below. What is left:

Not yet looked at

  • By a person at the terminal, on Windows. The pointer was real but moved by program, and the windows were judged from pictures taken five times a second, looked at afterwards. Still for a hand and an eye: how a drag looks while it happens, whether Windows Terminal's view comes back after a selection, and a paste by right-click, which was left alone because it reads the clipboard. The maintainer's test by hand was of the pager, in Windows Terminal.
  • A mouse in the legacy console, and mintty while an edge is moving. Windows scales the legacy console's window, and mintty follows its window only when the button is let go. Selection in mintty was left alone as well: it copies what is selected.

Found, not fixed here

  • In iTerm2, resizing while a command prints very fast can still leave one line printed over the old toolbar. With a command printing 200 lines a second, that happened at every resize before 79c44604 and 6eeecab6 (8 of 8) and at 2 of 16 after them; legacy rendering, 0 of 8. It did not happen in Terminal.app, tmux or xterm (0 of 16 resizes each). What is left is output iTerm2 had already taken when it was resized, before the application could know: reinstalling the region inside the signal handler was tried and measured no better.
  • On Windows, making the window shorter while a command prints fast still damages its output, as often as before. Windows has no window-change signal, so 6eeecab6 does nothing there: the display learns of a resize from prompt-toolkit's poll or from its own next frame. With a command printing 200 lines a second, a line was printed over the old toolbar in 4 of 4 runs in each of Windows Terminal, conhost and mintty, before these changes and after them; legacy rendering, 0 of 4. It was the shrink every time and never the other three directions. Dragging the window shorter a row at a time at 20 lines a second did the same at 6 of 15 steps in Windows Terminal and in mintty. An experiment outside the product, which looks at the terminal's size before every write of command output, brought 200 lines a second to 2 of 4 runs in Windows Terminal and 0 of 4 in conhost and mintty, and the drag to none. It has not been through the test suite. A drag by mouse does it at 4 lines a second as well: 5 of 41 lines were printed over the toolbar in Windows Terminal, 4 of 41 in Preview, and 7 and 6 of 201 at 20 lines a second; in mintty, which resizes when the button is let go, none at 4 and 1 of 201 at 20; in conhost and in legacy rendering, none.
  • conhost loses output when the window grows while a command prints. Dragging it taller a row at a time at 20 lines a second lost 44 of 201 lines after these changes and 54 before them; legacy rendering, none. Not seen in Windows Terminal or mintty, nor when the window grew ten rows in one step. A drag by mouse from 30 rows to 15 and back, at 4 and at 20 lines a second, lost none.
  • The old toolbar row stays when a window grows from 2 rows to 3 during a command which prints nothing. At 2 rows the reservation is released and the old toolbar's text is left on row 2. At 3 rows the toolbar is painted on row 3 and the old one stays above it until the next resize. Before 79c44604 it stayed through later resizes as well. reconfigure() clears below the cursor only where it has a previous geometry, and coming back from below the two-row floor it has none. Seen in Windows Terminal and mintty, and not tried on other platforms.
  • Resizing just after a command line is submitted loses the line, in reserved rendering. The command runs, but the line which was typed is not left on the screen, and the first line of output is printed on the prompt's row (burst> burst 0). Under tmux on Linux, where the resize followed the keys within milliseconds: 4 of 4, before these changes and after them; legacy rendering, 0 of 4. Under tmux on macOS, where it arrived a little later: 0 of 4. Before 79c44604 a cursor report was also sometimes printed there (^[[15;8R). Not investigated. On Windows it depends on when the resize arrives. Written straight after the keys it does not happen: 0 of 8 in each of Windows Terminal, conhost and mintty, before these changes, after them and in legacy rendering, and 0 of 120 since. Between 2 and 35 ms after Enter it does: 14 of 60 in Windows Terminal, 11 of 60 in Preview and 6 of 60 in conhost, and 12 of 60 in Windows Terminal at d181692c, before the resize commits; at 50 ms, none. Legacy rendering kept the line in 60 of 60, twice with empty rows before its output.
  • conhost, a resize while the pager is open. In both renderings, quitting after shrinking the window puts the prompt a few lines too high. In reserved rendering the toolbar's old row is also left behind, below the window. The cause of the second is traced: release_region() erases the old row only if a fresh measurement equals the geometry it was given, and that geometry's buffer_id includes the window's position in conhost's buffer, which moves as lines scroll. Not seen in Windows Terminal or mintty. Not fixed, and how much conhost matters is undecided.
  • conhost, a burst of resizes at a prompt near the top of the screen leaves a second toolbar row and the prompt two or three times over: 4 of 4 tries after these changes, and before them 1 of 2 with the second toolbar and 2 of 2 with the repeated prompt. It did not happen from a full screen, or with the same resizes 1.5 s apart, or in Windows Terminal or mintty. Dragging the corner in and back with a mouse does the same from a full screen, and in legacy rendering too: the prompt two or three times over and the toolbar below the window, 2 of 2 in legacy rendering and 1 of 2 in reserved. Dragging one edge did not.
  • In mintty the toolbar's bar shrinks to its text once output has scrolled, until the next resize repaints it, in reserved and in legacy rendering alike. The Windows pseudo console which mintty is given does not pass the scroll region on, and repaints the toolbar's row without its padding. Not from this PR.
  • Output from another thread while the pager is open is written straight onto the pager's screen in reserved rendering, where legacy rendering puts it on the main screen. A design question, not measured.
  • more under the UTF-8 code page prints nothing, or garbage, for long lines of non-ASCII text. This is from main and affects every branch, so it belongs in an issue of its own rather than here. The built-in pager shows the same lines correctly.
  • One conhost run died at the external pager's wide-character step and did not do so again, nor in two more runs of the checklist in conhost at 6eeecab6. Not explained.
  • A toolbar style set on the session rather than in the theme is not seen by the reserved toolbar, which reads cmd2's theme. Legacy rendering uses the session's style. Noticed while fixing the bar; not changed.
  • A traceback, once, on Windows. One resize just after a line was submitted, in Windows Terminal and reserved rendering, printed a traceback ending RuntimeError: Executor shutdown has been called, from asyncio's _check_default_executor, before the command's output. The command ran and the next prompt was right. Its upper part was not kept, and it did not come back in some 470 further tries nor under 40 s of resizes 12 ms apart, with cmd2 in either rendering or with prompt-toolkit alone. Not explained. prompt-toolkit's Win32 input reader starts its next wait on the loop's executor after every input event, and a resize is one, so an event which arrives while an application is being torn down would do this; that is a reading of its code, not something measured.
  • In the legacy console the external pager prints escape sequences. more is handed them and that console does not interpret them (←[32mrow 0←[0m). From main. The built-in pager shows the colors.
  • read_secret() keeps the newline of a paste. In the legacy console, an answer and Enter written in one go are taken for a paste by prompt-toolkit's Win32 input: the newline went into the secret, and one more Enter returned 3 characters for pw. The main prompt and read_input() accepted the same write; read_secret()'s session has none of cmd2's key bindings. From main. Not tried in a terminal with bracketed paste.
  • Two things which are the terminal's. Windows Terminal Preview 1.26 shows the new size in the middle of the window for about 0.8 s after a resize, which in a window of 2 to 4 rows covers the middle of the toolbar's bar. And selecting text in Windows Terminal while a command prints stops its view from following the output, so the toolbar is below what is shown until the view is scrolled down; Escape did not bring it back. Both are the same in legacy rendering.

Before main

  • examples/stage5_pager_check.py is to be removed, like stage4_manual.py.

Stage 5 asks whether the built-in pager is still worth keeping now that
reserved rendering exists. The comparison with an external pager has been
measured on macOS and Linux in a pseudo-terminal, which cannot be done on
Windows.

examples/stage5_pager_check.py is the Windows half: an application which
pages generated text through either pager, says which toolbar rendering and
terminal classes were selected, and reports how long Rich took to render the
text, how long the built-in pager took to draw its first page, and how long
the pager was open. Its docstring lists what to record in Windows Terminal,
conhost and mintty.

Like examples/stage4_manual.py, it is for the integration branch only and
is to be removed before the work reaches main.
In reserved rendering the pager was drawn on the main screen. One pager
session sent no switch to the alternate screen and one switch back; legacy
rendering sends one of each. The pager painted over the command's earlier
output and left a page of paged rows, its key summary and a second toolbar
row in the scrollback.

A renderer enters the alternate screen as the first thing in a full-screen
frame, and marks itself as there before it asks the output to switch. Under
reserved rendering a frame is recorded first and emitted afterwards, and the
recorder refused to record the switch. The frame failed to prepare, the
renderer went on believing it had switched, and the frame prepared next was
drawn without the switch.

Record the switch with the frame which is drawn after it, so both reach the
terminal in one piece or not at all. A frame which is not emitted has not
switched screens, so the renderer is put back to believing what it did
before preparing it, and the screen it laid out is not kept as a baseline.

This also covers the pager coming back from Ctrl-Z, where the renderer has
left the alternate screen to lend the terminal and enters it again.

Leaving the alternate screen stays unrecordable: a renderer does that when
it is reset, never inside a frame.
@codecov

codecov Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.82%. Comparing base (712f353) to head (f0475d2).
⚠️ Report is 54 commits behind head on reserved_row_toolbar.

Additional details and impacted files
@@                  Coverage Diff                   @@
##           reserved_row_toolbar    #1815    +/-   ##
======================================================
  Coverage                 99.82%   99.82%            
======================================================
  Files                        35       35            
  Lines                      9059     9215   +156     
======================================================
+ Hits                       9043     9199   +156     
  Misses                       16       16            
Flag Coverage Δ
unittests 99.82% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@tleonhardt tleonhardt self-assigned this Nov 15, 2026
…Windows

prompt-toolkit's Windows10_Output sets the console mode around every flush,
and the mode it sets does not wrap at the end of a row. Its own stdout proxy
turns autowrap on before each write for that reason. SerializedTerminalWriter
takes that proxy's place in reserved rendering and did not, so everything
past the last column of a line of command output was written over the last
column and lost. Legacy rendering wrapped the same line.

Autowrap is now turned on before each write to that backend. A VT backend is
sent exactly what it was before.

Found in Windows Terminal while running examples/stage5_pager_check.py: with
a window narrower than the timing line, the line ended "its first pags".
…prepared

A managed write which lands while a frame is being prepared retires the
frame. The write puts the renderer's last screen back to the committed one,
but the renderer is still preparing and takes the frame it is preparing in
its place when it finishes. The frame prepared next was then diffed against
one the terminal never received, drew nothing where the two agreed, and left
the terminal showing the frame before.

prepare() now notes the renderer's last screen and cursor before the frame is
prepared, and puts both back when the frame is dropped for this reason.

Not changed: output written by another thread while the built-in pager is
open still goes straight onto the pager's screen in reserved rendering.
@tleonhardt

Copy link
Copy Markdown
Member Author

Stage 5 pager check on Windows, 2026-11-15

Results of examples/stage5_pager_check.py, run in Windows Terminal, conhost and mintty, once as auto and once as legacy. The built-in pager passes the whole checklist in all six. Two defects were found and fixed along the way, and three things are reported without a fix.

Environment

  • Windows 11 Pro 10.0.26300.9974, ARM64 virtual machine.
  • Windows Terminal 1.25.3131.0; conhost from the same Windows build; mintty 3.8.3 from Git for Windows, started as Git Bash starts it (login bash, MSYSTEM=MINGW64). Python gets a pseudo console there without winpty.
  • Python 3.15.2, the x86-64 build from the uv venv, so emulated on this machine. prompt-toolkit 3.0.53, Rich 15.0.0.
  • Tree: 9993b724 plus the two fixes below, since committed as dc84f6c6 and 3f349136. Windows Terminal auto was run again after merging reserved_row_toolbar (31d46236, which carries main's 0047276a) with the same results.
  • Launch: .venv\Scripts\python.exe examples\stage5_pager_check.py auto|legacy, with NO_COLOR unset.

Method

Keys were injected into the application's console with WriteConsoleInput, not typed. The screen was read with the console API; the scrollback through UI Automation in Windows Terminal, from the whole screen buffer in conhost, and from mintty's log of what it was sent. Window screenshots were taken at each step and a sample of them looked at. Scripts, results, logs and screenshots are kept outside the repository, in toolbar-flicker-harness/runs/20261115-stage5-windows-pager/.

1. info

Terminal Mode Rendering Output / input Size TERM
Windows Terminal auto reserved ReservedOutput / Win32Input 119x29 None
Windows Terminal legacy legacy Windows10_Output / Win32Input 119x30 None
conhost auto reserved ReservedOutput / Win32Input 119x29 None
conhost legacy legacy Windows10_Output / Win32Input 119x30 None
mintty auto reserved ReservedOutput / Win32Input 119x29 xterm-256color
mintty legacy legacy Windows10_Output / Win32Input 119x30 xterm-256color

The window is 120 columns in each: prompt-toolkit's Windows outputs report one fewer. The built-in pager is the default in all six, and the external pagers are 'more' / chop 'more'.

2 to 6. Built-in pager

The same in all six runs unless a row says otherwise.

Check Result
rows 2000 color brings up the built-in pager yes
Toolbar clock visible and ticking while paging yes
Space, b, G, g work
/row 150 Enter, then n work; n goes to row 1500
row N green, bold red red yes
After q: hist 0 to hist 39 intact yes
After q: paged rows, key summary or second toolbar left behind none
Alternate screen: one enter and one leave for each pager session yes (mintty's log)
rows 2000 long chop: right arrow reaches tail0, left returns yes, 4 presses each way
rows 2000 wide: wide characters wrap without breaking yes
Resize while open, page on, quit re-laid out, one toolbar; see finding 4 for conhost
Ctrl-C while open closes the pager; prompt and toolbar return

Timing lines, in seconds. "First page" is from handing the text to the pager to its first frame.

Run 2000 color: Rich / first page 100000: Rich / first page
Windows Terminal auto 0.10 / 0.02 0.52 / 0.06
Windows Terminal legacy 0.08 / 0.02 0.53 / 0.06
conhost auto 0.08 / 0.02 0.58 / 0.08
conhost legacy 0.09 / 0.02 0.52 / 0.06
mintty auto 0.08 / 0.02 0.59 / 0.06
mintty legacy 0.09 / 0.02 0.55 / 0.06

7. External pager (more)

The same in all six runs.

  • Colors are shown, not printed as escape sequences.
  • Space pages and Enter moves one line. It cannot go back or search: b and / do nothing.
  • The toolbar is hidden while it runs and returns, ticking, when it exits.
  • It pages on the main screen, so every row it showed stays in the scrollback.
  • chop has no effect: long lines wrap.
  • A resize is not followed; it goes on paging at the old height.
  • Ctrl-C ends it and the prompt returns.
  • rows 100000: Rich takes 0.52 to 0.60 s. The check measures no first-page time for it.
  • rows 2000 wide shows nothing. more exits after 0.04 s having printed a blank line or nothing. See finding 3.

Findings

1. Fixed (dc84f6c6): command output wider than the terminal was cut off in reserved rendering

In reserved rendering on Windows, everything past the last column of a line of command output was lost: each character there was written over the one before it. Legacy rendering wrapped the line. Seen first as a timing line ending its first pags, then with rows 2 long, which lost both of its tails.

prompt-toolkit's Windows10_Output sets the console mode around each flush, and the mode it sets has no wrap at the end of a row. Its own stdout proxy turns autowrap on before each write for that reason. SerializedTerminalWriter, which takes the proxy's place in reserved rendering, did not. It now does, for that backend only; a VT backend is sent exactly what it was before. Confirmed in Windows Terminal before and after. Not specific to the pager: it affects any command output in reserved rendering on Windows.

2. Fixed (3f349136): the defect this PR lists as "found, not fixed here"

A managed write which lands while a frame is being prepared left the renderer's baseline at the dropped frame. PromptToolkitBridge.prepare() now puts back the screen and cursor the renderer had before it prepared the frame. The new test fails without the change: the frame after the dropped one emitted no content.

Not changed, and a design question rather than a defect of this kind: output which another thread writes while the pager is open is written straight onto the pager's screen in reserved rendering, where legacy rendering puts it on the main screen. That was so before and is so for a write between frames as well. Not measured here.

3. more cannot show long lines of non-ASCII text under the UTF-8 code page

From main, not from this PR: cmd2 sets the console to code page 65001 while an external pager runs. Under it, piped to more in a plain cmd.exe:

  • a line of 2 wide characters is shown;
  • a line of 50 or 100 wide characters prints nothing;
  • a line of 200 é prints part of it and then garbage from memory;
  • a line of 200 ASCII characters is shown.

Under code page 437 the same lines are shown complete, as mojibake. The built-in pager shows all of them correctly.

4. conhost: a resize while the pager is open

conhost keeps a 9001-line buffer and a window onto it. Two things follow, neither seen in Windows Terminal or mintty.

  • Both modes. After shrinking the window while the pager is open, quitting puts the prompt a few lines too high. Two or three earlier lines of the main screen are overwritten, and the lines which were below them are left under the bottom of the window.
  • Reserved only. The toolbar's old row is left there as well, as a second toolbar row below the window. The old row is in fact left on the main screen whenever the pager opens, resize or not: release_region() erases it only if a fresh measurement equals the geometry it was given, and the geometry's buffer_id holds the window's position in the buffer, which conhost moves with every line scrolled. Traced: given (0, 0, 120, 9001), measured (0, 14, 120, 9001), not erased. Without a resize the toolbar is repainted on the same row when the pager closes, so nothing shows. A shell command and quitting the application leave no stale row.

This is the "backing buffer taller than the viewport" case which the Stage 1 Windows probes never exercised.

5. The reserved toolbar is drawn without its bar

At the prompt, a reserved toolbar is plain text. In legacy rendering, and in the pager in either mode, it is a reverse-video bar. The painter is given DynamicStyle(get_pt_theme), cmd2's theme alone, which has no rule for bottom-toolbar; prompt-toolkit merges its default UI style into an application's, and that is where reverse comes from. Not specific to Windows. Not changed: it may be intended.

6. One run died

The first conhost auto run ended at the external pager's rows 2000 wide step: the application was gone and the script driving it with it. A second complete run, and separate attempts at that step, did not repeat it. Not explained.

Limits

  • Keys were injected, not typed, and nothing was judged by a person at the terminal; the screenshots were looked at afterwards.
  • Timings are from an emulated x86-64 Python in a virtual machine.
  • mintty's scrollback was judged from what mintty was sent and from the visible screen.
  • The legacy Win32 console without VT processing (Win32Output) was not run.
  • iTerm2 is still to be checked by eye on macOS.

…e it is emitted

A managed write which lands after a frame is prepared, and before it is
emitted, drops the frame. The write puts the renderer's last screen back to
the committed one, but the renderer's cursor stayed where the dropped frame
would have left it. The frame prepared next moved from there, and was drawn
that many rows from where it belonged: one row too high where the dropped
frame had left the cursor a row lower.

A prepared frame now carries where the renderer's cursor was before it was
prepared, and the write which drops the frame puts it back. This is the
other half of 3f34913, which did the same for a write landing while the
frame is still being prepared.
The reserved toolbar was plain text. In legacy rendering it is a
reverse-video bar across the whole row.

Two things differed. The painter resolved styles against cmd2's theme alone,
which has no rule for bottom-toolbar; an application lays its style over
prompt-toolkit's default one, and "reverse" comes from there. And the
toolbar's text had no class at all: prompt-toolkit's window gives every cell
class:bottom-toolbar and its text class:bottom-toolbar.text besides, so a
rule for either reached only the padding after the text.

The painter is now given the default style with the theme over it, and a
style to put under every fragment's own. A theme's rules still win.

Found on Windows while running examples/stage5_pager_check.py, and not
specific to it. In a pseudo-terminal on macOS the bottom row went from 0 to
80 of 80 cells in reverse video, at the prompt and during a command, as it
is in legacy rendering.
… corner

There the erase reaches the terminal through the reserved adapter, or
replayed from a recorded frame, and the adapter is what the application
holds. Without looking through it to the backend, no erase was changed.
…oved it

After a resize the old band's text is still on the screen, and where depends
on the terminal. reconfigure() cleared the rows it had been painted on, and
only when the terminal had grown, which is right for a terminal that adds
rows at the bottom. iTerm2 brings rows back from its scrollback instead, so
the old band moves down with everything else and the rows it was painted on
hold output: a line of output was erased each time the window grew. Made
narrower, iTerm2 rewraps the band's row into two, and one was left.

The rows of the old band are below the cursor's row wherever the terminal
has put them, and nothing else is: output goes on from the cursor, and a
prompt redraws itself after a resize. reconfigure() now makes room for the
new band and then clears everything below the cursor's row, after any
resize.

One case is not below it. A terminal sets its scroll region back to the
whole screen when it is resized, so the next line of output to end moves the
cursor on to the old band's row instead of scrolling. The line after it was
then written over the old toolbar and the rest of the toolbar stayed in the
output. The bridge now notes when the last thing to move the cursor was
command output ending a line, which leaves it at the start of a row holding
none of that output, and that row is then cleared as well. It is erased from
its second column, with the first cleared separately, because iTerm2 saves a
screen erased from its top-left corner to the scrollback.

In iTerm2 3.7.7, resizing while a command printed 200 lines a second left a
line printed over the toolbar at every resize before this and the next
commit (8 of 8), and at 2 of 16 after them. Legacy rendering: 0 of 8.
A terminal sets its scroll region back to the whole screen when it is
resized. Until the display has installed the region again, command output
runs on to the toolbar's row. The display runs off the main thread, where
prompt-toolkit cannot attach the window-change signal and looks at the
terminal's size twice a second instead; the bridge also looks whenever a
frame is drawn, which is as often as the toolbar refreshes. A command
printing four lines a second had its output damaged at most resizes.

The signal comes to the main thread, where the command is. While a display
holding a reservation runs, a handler there asks the application for a
frame, which is where the bridge looks at the size. It calls whatever
handler was installed before it, puts that one back when the display stops,
and is not installed off the main thread or where there is no such signal.

Reinstalling the region inside the handler instead, in the thread doing the
writing, was tried and measured no better: what is left is output the
terminal had already taken when it was resized.

In iTerm2 3.7.7, with a command printing four lines a second, resizes in
four directions damaged the output in 5 runs of 8 before this and the
previous commit and in 0 of 4 after them.
@tleonhardt

Copy link
Copy Markdown
Member Author

Resize checks on Windows after 79c44604 and 6eeecab6, 2026-11-15

The Stage 4 resize checks, repeated on Windows at 6eeecab6, in Windows Terminal, conhost and mintty. Each was run in reserved rendering at the head, in reserved rendering at the tree from before the two resize commits, and in legacy rendering.

The clearing rule makes nothing worse on Windows and several things better. The resize checks at a prompt pass in all three terminals, apart from one conhost case which was there before (finding 4). Resizing while a command prints is clean at 4 and at 20 lines a second, where it was not before. What is left is damage when the window is made shorter during fast output, which is as common after the change as before it: Windows has no window-change signal, so 6eeecab6, which closed that gap elsewhere, does nothing here.

Environment

  • Windows 11 Pro 10.0.26300.9974, ARM64 virtual machine.
  • Windows Terminal 1.25.3131.0; conhost from the same Windows build; mintty 3.8.3 from Git for Windows, started as Git Bash starts it.
  • Python 3.15.2, the x86-64 build from the uv venv, so emulated. prompt-toolkit 3.0.53.
  • "After" is the branch head, 6eeecab6. "Before" is d181692c, the commit before 79c44604. It differs from the 8e37a0b1 the PR's table uses only by the pager's scrollback change and tests.
  • The test suite on this machine at the head (python -Xutf8 -m pytest tests): 3,704 passed, 199 skipped. PR CI: 30 of 30.

Method

A check application outside the repository (resize_app.py: prompt check> , toolbar TOOLBAR HH:MM:SS, refresh interval 0.5 s) was run in a real window of each terminal. A driver set the window to exact rows and columns by moving its edges, injected keys with WriteConsoleInput, and judged each step three ways:

  • the console's screen buffer, characters and attributes, read in one call so that rows agree;
  • a screenshot of the window, searched for bright full-width bands, which is what the toolbar's bar looks like;
  • in Windows Terminal, the terminal's own text through UI Automation, scrollback included. In conhost the whole 9001-line buffer. mintty offers neither, so there the screen was sampled about 30 times a second while output ran, which sees every line before it scrolls away.

Nothing was dragged with a mouse or watched by a person. Scripts, logs, samples and screenshots are kept outside the repository, in toolbar-flicker-harness/runs/20261115-stage5-windows-resize/.

Resizing at a prompt and around handoffs: all pass

The same in Windows Terminal, conhost and mintty at the head, unless a finding below says otherwise.

Check Result
Idle prompt, 30 rows to 2, 3, 4, 30, 3, 2, 3, 4, 30; from an empty screen and from a full one one toolbar on the bottom row at every size, one prompt, no key needed; output above intact
The same with text typed and not submitted, then 90 and 100 columns text survives every resize and the command then runs
Sixteen resizes one straight after another, from a full screen settles with one toolbar, one prompt, output intact
A drag: a row at a time down to 12 rows and back, 0.1 s apart the same
Resize at the instant a line is submitted, 8 times the typed line stays above its output, 8 of 8 (also before, and in legacy)
A quiet command and a line left unfinished, through 4, 3, 2, 3, 4, 30 rows and 90 columns PARTIALEND whole on one row; toolbar right except at one step, finding 3
read_input(), read_secret(), select(): shorter, taller, narrower, wider toolbar on the bottom row each time; the answer is accepted
suspend_bottom_toolbar() twice nested, resized while the guest has the terminal no toolbar while the guest runs; back on the bottom row of the new size afterwards
3, 2, 3, 30 rows after a shell command returned right at each size
Built-in pager open: shorter, taller, narrower, wider, page on, quit toolbar on the bottom row and key summary above it at each size; no paged rows or key summary left
Toolbar lines in everything the terminal holds, after all of it one (Windows Terminal, conhost)
Quit, then 60 lines from the shell the shell has the whole window; no toolbar row, no bar

At 2 rows an idle prompt shows the fallback toolbar on row 2, and at 3 the reserved toolbar is back on row 3, as on 2026-11-14.

Resizing while a command prints

Runs in which the output was damaged. Each run resizes four ways: shorter (30 to 20 rows), taller, narrower (120 to 90 columns), wider.

Terminal 4 lines a second, before after legacy 200 lines a second, before after legacy
Windows Terminal 1.25 1 of 4 0 of 4 0 of 4 4 of 4 4 of 4 0 of 4
conhost 3 of 4 0 of 4 0 of 4 4 of 4 4 of 4 0 of 4
mintty 3.8.3 3 of 4 0 of 4 0 of 4 4 of 4 4 of 4 0 of 4

At 20 lines a second: 2 of 2 before, 0 of 2 after and 0 of 2 in legacy, in each terminal.

  • The damage is the one the PR describes: a line printed over the old toolbar, with the rest of the toolbar left in the output (out 030418:35:06).
  • At 200 lines a second it is the shrink every time, and only the shrink: 12 of 12 shrinks damaged before and 12 of 12 after, against 0 of 36 of the other three directions in both.
  • A drag at 20 lines a second, 15 steps shorter and 15 taller a row at a time, 0.2 s apart:
Terminal Before After Legacy
Windows Terminal 6 lines printed over the toolbar, all while shrinking 6, the same none
mintty 6, all while shrinking 6, the same none
conhost 5 printed over the toolbar and 54 of 201 lines lost, while growing 1 and 44 lost none

Findings

1. Not fixed on Windows: shrinking the window during fast output

On POSIX the PR measures 0 of 8 at 200 lines a second after the change in every terminal but iTerm2. On Windows it is 4 of 4 in all three terminals, before and after. The application learns of a resize from prompt-toolkit's poll or from its own next frame, and by then more than one line has been written; clearing the row output has just moved on to covers only the first.

An experiment, not product code: the check application was run with PromptToolkitBridge.before_managed_write patched to call _reconfigure_if_resized() first, so that the size is looked at before every write of command output.

Terminal 200 lines a second, after with the experiment Drag at 20 lines a second, after with the experiment
Windows Terminal 4 of 4 runs 2 of 4 6 lines over the toolbar none
conhost 4 of 4 0 of 4 1 over the toolbar, 44 lost none
mintty 4 of 4 0 of 4 6 over the toolbar none

The other steps it was run through (resize at submit, quiet, unfinished line, 1, 4 and 20 lines a second) gave the same results as without it, with one exception: in Windows Terminal, with a line left unfinished, the toolbar was not repainted at 3 rows after 2, where without the experiment it was. One try each. It costs one console call for each write and was not run through the test suite.

2. conhost loses lines of output when the window grows during output

In the drag, each step taller lost the lines written until the application caught up: out 0094 is followed by out 0103. 54 of 201 lines before the change and 44 after; none in legacy. Not seen in Windows Terminal or mintty, and not when the window grew by ten rows in one step. With the experiment of finding 1: none.

Also conhost only, and the same before and after: at one line a second through 2, 3, 4 and 30 rows, one line was printed over the toolbar (out 000418:38:28).

3. The old toolbar row stays when a quiet command's window grows from 2 rows to 3

In Windows Terminal and mintty, during a command which prints nothing: at 2 rows the reservation is released and the old toolbar's text stays on row 2, no longer updated. At 3 rows the toolbar is painted on row 3 and the old one is still on row 2, for as long as the size stays at 3 rows.

  • Before the change it stayed longer: still there at 4 rows, and with an unfinished line at 30 rows too. After it, the next resize clears it. So the clearing rule helps here.
  • TerminalDisplay.reconfigure() clears below the cursor only where it has a previous geometry. Coming back from below the two-row floor it has none, and only makes room.
  • Not seen in conhost, where nothing is left on screen at 2 rows.

4. conhost: a burst of resizes at a prompt near the top of the screen

Fourteen or more resizes with no pause, starting from a prompt near the top of a mostly empty screen, left a second toolbar row and the prompt two or three times over. At the head 4 of 4 tries; before the change 1 of 2 with the second toolbar and 2 of 2 with the repeated prompt. The same sizes 1.5 s apart were right in both, and from a full screen, which is what the checklist's burst step uses, it did not happen in either. Not seen in Windows Terminal or mintty.

5. mintty: the bar shrinks to its text once output scrolls

8e37a0b1 on Windows: the reserved toolbar is a reverse-video bar across the window in all three terminals. In mintty, after any command output has scrolled, only the toolbar's text is in reverse video and the rest of the row is black, until the next resize repaints it. That was so in 9 of the 10 looks taken after a command's output; the tenth followed a resize with no scrolling after it. Legacy rendering is the same, 9 of 10, so the reserved toolbar now matches legacy there too.

mintty's log of what it was sent shows why. The Windows pseudo console does not pass the scroll region on: for each line of output it scrolls the whole screen, blanks the row the toolbar moved to, and repaints the toolbar as its text followed by a cursor move over the padding. Not from this PR, and not seen in Windows Terminal, whose pseudo console is its own.

The pager checklist at the head

examples/stage5_pager_check.py was run again at 6eeecab6 with the driver of the earlier run, in the same six combinations, for fb041f6b, 8e37a0b1 and 242d9d96, which that run predates.

  • Windows Terminal, conhost and mintty in reserved rendering, and Windows Terminal and conhost in legacy: every recorded value is the same as in the earlier run, clock times and timings aside. That covers paging, search, colors, chopped lines, wide characters, 100,000 rows, a resize while open, Ctrl-C, and what is left behind after quitting. The one error is the one known from that run: more shows nothing for rows 2000 wide.
  • mintty in legacy rendering differs in the external pager only: more was one page further back at each step (row 30 at the top after Space, where it was row 60). The built-in pager's values are the same. Not looked into; neither legacy rendering nor more is touched by these commits.
  • First page of 100,000 rows: 0.06 to 0.07 s, as before.

Limits

  • Keys were injected and windows resized by program. Nothing was dragged with a mouse or judged by a person at the terminal; a sample of the screenshots was looked at afterwards.
  • Small numbers: four runs at 4 and at 200 lines a second, two at 20, and one drag, for each terminal and tree. The experiment of finding 1 is one pass.
  • mintty's scrollback cannot be read, so "output above intact" and the count of toolbar lines held are from Windows Terminal and conhost only. In mintty the damage counts come from sampling the pseudo console's screen, which showed the same damaged rows that Windows Terminal's own text did.
  • Legacy runs were judged by the terminal's own text where there is one: the sampling reads the legacy toolbar being erased and redrawn as damage.
  • Starting the application in a window of 2 or 3 rows was not repeated, nor Windows Terminal Preview, nor the Win32 console without VT processing (Win32Output).
  • Python is an emulated x86-64 build in a virtual machine, which stretches every race measured here.

@tleonhardt

Copy link
Copy Markdown
Member Author

The remaining Windows checks for Stage 5, 2026-11-16 and 17

The three things this PR listed as "not yet looked at" on Windows, run at the branch head
49c18d0c: a real mouse, starting in a window of 2 or 3 rows together with Windows Terminal
Preview, and the legacy console without VT processing (Win32Output).

Nothing new is wrong with the pager, the reserved toolbar at a prompt, or small windows. Two
of the PR's "found, not fixed" items need rewording from what was measured here, and three smaller
things were seen for the first time. All are listed under Findings.

Environment

  • Windows 11 Pro 10.0.26300, ARM64 virtual machine. Python 3.15.2, the x86-64 build from the uv
    venv, so emulated. prompt-toolkit 3.0.53.
  • Windows Terminal 1.25.3131.0, Windows Terminal Preview 1.26.3133.0, conhost from the Windows
    build, mintty 3.8.3 as Git Bash starts it, and the legacy console (conhost v1,
    ConhostV1.dll).
  • Tree: 49c18d0c. In cmd2/ it differs from the 6eeecab6 of the last Windows run only by what
    the merges from main brought to cmd2.py, completion.py and utils.py.
  • The virtual machine stalled three times during the runs: the wall clock jumped by 72 s, by 475 s
    and once ran at a crawl for about two minutes. Each run it touched was repeated, and is named
    below.

Method

The drivers of 2026-11-15 were extended and are kept outside the repository, in
toolbar-flicker-harness/runs/20261116-stage5-windows-remaining/.

  • A real mouse. The pointer is moved, pressed and released with SendInput, so a window goes
    through its modal sizing loop as it does under a hand, where the earlier runs set the size with
    SetWindowPos. The wheel and a text selection are made the same way. While an edge is dragged
    the window is photographed five times a second and each picture is searched for the toolbar's
    bar. About forty pictures were looked at.
  • Starting small. The check application is held back until its window has been made 120
    columns by 3 or by 2 rows, so its first prompt, toolbar and reservation are made at that size.
  • The legacy console is what Windows gives cmd.exe while HKCU\Console\ForceV2 is 0. The
    value was set for the two seconds a launch takes and put back each time. conhost.exe with a
    command line of its own refuses to start in that mode.
  • Judged as before: the console's screen buffer, screenshots, and Windows Terminal's own text
    through UI Automation or conhost's whole buffer.

1. Windows Terminal Preview 1.26

Check Result
Resize checklist, reserved as stable 1.25 at 6eeecab6, step for step, apart from the two rows below
Resize checklist, legacy as stable
Pager checklist, reserved and legacy no recorded value differs from stable; first page of 100,000 rows after 0.06 s
Started in 3 and in 2 rows passes, as stable
Real mouse as stable; see section 4
  • Preview shows the new size in the middle of the window for about 0.8 s after every resize
    (120 × 30). In a window of 2 to 4 rows that covers the middle of the toolbar's bar for as
    long. It is the terminal's, not the application's: stable draws the bar 0.03 s after the console
    has it, and so does Preview at 20 rows and more.
  • The resize-at-submit step lost the typed line once in 8 tries; see finding 1.

2. Starting in a window of 3 or 2 rows

Sixteen runs: Windows Terminal, Preview, conhost and mintty, each in reserved and legacy
rendering, each started at 3 and at 2 rows.

Step Result in all sixteen
As started one prompt; the toolbar on the bottom row as a bar; its clock ticks
say, hist 10, text typed and then submitted, at the starting size run; every line of output is held by the terminal; toolbar stays on the bottom row
rows 50 at 3 rows the built-in pager comes up with one row of text, its key summary and the toolbar; Space pages; q returns
rows 50 at 2 rows prompt-toolkit's Window too small...; q returns to the prompt
Grown through 3, 4, 30, 3, 2, 3, 4 and 30 rows toolbar on the bottom row and one prompt at each
At full size: 4 lines a second with two resizes, then the pager with four output clean; nothing left behind
Toolbar lines in everything the terminal holds one (Windows Terminal, Preview, conhost)
Quit, then 60 lines from the shell the shell has the whole window
  • mintty passes by the console's screen buffer. In the window its bar is as wide as its text from
    the start at 2 rows and after any output at 3, which is the mintty item already listed.
  • The legacy console's window does not go below 3 rows and was not started small.
  • One run, Windows Terminal legacy at 2 rows, fell into the 72 s clock jump and was repeated clean.

3. The legacy console, without VT processing

info there: Win32Output / Win32Input, console output mode 3, a 120 by 9001 buffer.

Check Result
auto legacy rendering, as designed for this backend
legacy legacy rendering
reserved cmdloop() raises ValueError: reserved bottom toolbar mode is not available here: legacy Win32 console has no VT scroll margins
Toolbar a reverse-video bar on the bottom row; its clock ticks
Idle prompt and typed text through 3 to 47 rows, 90 and 100 columns, a burst, a drag toolbar on the bottom row and one prompt at every size reached; the text survives and the command runs
A quiet command and an unfinished line, resized right at each size; PARTIALEND whole
1, 4, 20 and 200 lines a second with resizes; the 30-step drag at 20 no damage in any of 17 runs; one more at 200 was lost to the 475 s clock jump
read_input(), select(), a nested handoff, after a shell command pass; select() has no toolbar, as in legacy rendering elsewhere
read_secret() passes when Enter is a key of its own; see the note below
Quit the shell has the whole window

The built-in pager, judged from screenshots, since Win32Output makes its alternate screen a
buffer which nothing else may open. Fifteen were looked at:

Check Result
rows 2000 color built-in pager; row N green and bold red red; toolbar under the key summary, ticking
Space, G, g, /row 150 Enter, n work; G and g repaint within 0.1 s; n goes to row 1500 and the match is highlighted
long chop, right arrow reaches tail0 after four presses
wide laid out without breaking; every wide character is drawn as ?, which is the console's font and code page
rows 100000 first page after 0.13 s; 2000 rows after 0.08 to 0.20 s
Shorter, narrower, back, paging on the pager follows the window each time, one toolbar
Ctrl-C closes it; prompt and toolbar return
After q hist 0 to hist 39 intact; no paged rows, key summary or second toolbar in the buffer
  • The external pager is worse there than anywhere else: more is handed the escape sequences
    and the legacy console prints them (←[32mrow 0←[0m plain ←[1;31mbold red←[0m). From main, not
    from this PR.
  • read_secret() and a paste. With the answer and Enter written to the console in one go,
    prompt-toolkit's Win32 input takes them for a paste: the newline went into the secret instead of
    submitting it, and one more Enter returned 3 characters for pw. The main prompt and
    read_input() accepted the same write. read_secret()'s session has none of cmd2's key
    bindings, among them the one for a paste. Also from main. Seen in the legacy console only; a
    terminal with bracketed paste was not tried.

4. A real mouse

Reserved rendering in Windows Terminal, Preview, conhost and mintty; legacy rendering in Windows
Terminal, conhost and mintty as a control.

Check Windows Terminal and Preview conhost mintty
Idle prompt: drag the bottom edge to 11 rows and back, the right edge to 81 columns and back right; the bar is in every picture taken during the drag but one right afterwards; no bar in a third of the pictures while the bottom edge moves right after each release
The same with text typed right; the text survives and the command runs right for the edges not tried
Drag the corner to 90 by 15 and back right; no bar in 4 to 6 of 35 pictures on the way prompt drawn two or three times, toolbar below the window: finding 3 not tried
Drag shorter and taller while a command prints 4 or 20 lines a second lines printed over the old toolbar: finding 2 clean 4: clean; 20: one line over the toolbar
Built-in pager open: drag the bottom and the right edge toolbar on the bottom row, key summary above it; nothing left after q the same not tried with a release
Wheel over the pager scrolls a line for each notch nothing scrolls three lines for each notch
Wheel back at a prompt and during output, and forward again the whole view scrolls, toolbar with it; output complete; right once back at the bottom the same right
Select text while a command prints, then Escape output complete; see below output and clock stop while selected and catch up; complete not tried: mintty copies what is selected
Toolbar lines in everything the terminal holds, after all of it one one cannot be read
  • Legacy rendering gives the same results in each terminal, with one difference: dragging shorter
    while a command prints damages nothing there.
  • Selecting text in Windows Terminal while a command prints stops its view from following the
    output, so the toolbar and the newest lines are below what is shown until the view is scrolled
    down again. Escape did not bring it back. The same in legacy rendering, and in Preview.
  • The first legacy run in Windows Terminal fell into the two-minute crawl: its command ended with
    _DisplayStillRunningError: the bottom toolbar did not stop within 10.0 seconds and the toolbar
    stayed off during commands for the rest of that session. A second complete run was clean.

Findings

1. The typed line is lost on Windows too, when the resize follows Enter by a few milliseconds

The PR says that resizing at the instant a line is submitted loses the line under tmux on Linux
and did not on Windows, 0 of 8. With the resize written straight after the keys it is still 0 of
120 here. With it 2 to 50 ms after Enter:

Terminal Reserved, head Reserved, d181692c Legacy
Windows Terminal 14 of 60 12 of 60 0 of 60
Preview 11 of 60
conhost 6 of 60
  • The line reads check> said sub7: the first line of output is on the prompt's row. It happened
    at every delay from 2 to 35 ms and at none of the tries at 50 ms.
  • Not from the resize commits: the tree from before them does the same.
  • Legacy rendering keeps every typed line. In 2 of 60 it left four or five empty rows between the
    line and its output.
  • With the window resized every 12 ms for 40 s while 648 short commands were typed ahead, reserved
    rendering in Windows Terminal showed 559 typed lines above their output, 26 outputs on the
    prompt's row and 63 with no typed line at all; conhost 605 and 8. Legacy rendering kept all 648
    typed lines, 52 of them with empty rows before their output.

2. Dragging the window shorter while a command prints, with a mouse

The PR's Windows item gives this at 200 lines a second, and at 20 for a drag a row at a time. A
drag by mouse does it at 4 lines a second as well.

Terminal 4 lines a second 20 lines a second Legacy
Windows Terminal, held and dragged back 5 of 41 lines over the toolbar 7 of 201 none
Windows Terminal, let go at 15 rows 3 of 41 4 of 201
Preview 4 of 41 6 of 201
mintty, let go at 15 rows none 1 of 201 none
conhost none none none

Three of the old toolbar's bars were still on the screen when the command ended
(out 001410:23:16). conhost lost no output while growing either, where the earlier drag by
SetWindowPos lost 44 of 201 lines.

3. conhost: a corner drag at a prompt

Dragging the bottom right corner in and back, from a full screen, left the prompt two or three
times over with empty rows between and the cursor's row and the toolbar below the bottom of the
window: 2 of 2 in legacy rendering and 1 of 2 in reserved. The edges alone did not. This is the
PR's "burst of resizes at a prompt" item, now also from a full screen and in legacy rendering.

4. A traceback, once

One try of the resize-at-submit step in Windows Terminal, reserved rendering, printed a traceback
ending in

File "...\Lib\asyncio\base_events.py", line 559, in _check_default_executor
    raise RuntimeError('Executor shutdown has been called')
RuntimeError: Executor shutdown has been called

before the command's output. The command ran and the next prompt was right. Its upper part had
scrolled away and was not kept. It did not come back in some 470 further tries, nor in 40 s of the
resize storm of finding 1 in each of reserved rendering, legacy rendering and a prompt-toolkit loop
with no cmd2 in it.

An inference, not a measurement: prompt-toolkit's Win32 input reader (input/win32.py, ready())
waits for the console's input handle on the loop's executor and starts the next wait after each
event. A resize is an input event. If one arrives while an Application.run() is being torn down,
the next wait is asked of an executor asyncio has already shut down. That would make it
prompt-toolkit's and independent of rendering, but the frames which would show it were lost.

Limits

  • Small numbers throughout: one or two runs of each mouse step for each terminal.
  • Nothing was watched by a person at the terminal: the pictures were looked at afterwards, and five
    a second misses anything shorter. A mouse was not used in the legacy console, whose window
    Windows scales.
  • mintty follows the window only when the button is let go, so nothing is known about it while an
    edge moves. Its scrollback cannot be read.
  • Python is an emulated x86-64 build in a virtual machine which stalled three times. Races measured
    here are stretched, and finding 4 may owe its one appearance to that.
  • Right-click, paste by mouse and selection in mintty were left alone, because they read or replace
    the clipboard.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant