Skip to content

Py tooling: fix Windows terminal color output and handle color flags in compare.py (fixes #642) - #2297

Open
jdymitarai wants to merge 2 commits into
google:mainfrom
jdymitarai:fix-compare-windows-color-642
Open

Py tooling: fix Windows terminal color output and handle color flags in compare.py (fixes #642)#2297
jdymitarai wants to merge 2 commits into
google:mainfrom
jdymitarai:fix-compare-windows-color-642

Conversation

@jdymitarai

Copy link
Copy Markdown

Summary

This PR fixes issue #642 where compare.py produced literal ANSI escape characters (^[[92m, ^[[0m, etc.) on Windows consoles, and ensures that color settings (--color, --no-color, --benchmark_color=...) are properly recognized regardless of argument position.

Root Cause

  1. Windows Console VT Processing: On Windows, ANSI escape sequences (\033[...]) require ENABLE_VIRTUAL_TERMINAL_PROCESSING (0x0004) to be set on the console output buffer. Without this flag, cmd.exe and standard console hosts print literal escape sequences.
  2. Auto-detection & Redirects: compare.py previously defaulted color=True unconditionally, even when stdout was not a TTY (such as redirected to a file or piped to another tool), or when the standard NO_COLOR environment variable was set.
  3. Argparse Remainder Flag Leak: Because subparsers in compare.py collect trailing options into benchmark_options (nargs=argparse.REMAINDER), placing --no-color or --benchmark_color=false after the subcommand or arguments caused the flag to be captured as benchmark runner options instead of setting args.color, while triggering a spurious warning when inputs were JSON files.
  4. Test import issue: In tools/gbench/report.py line 1492, import util caused ModuleNotFoundError when run via python -m unittest.

Changes

  • Windows VT Processing: Enable ENABLE_VIRTUAL_TERMINAL_PROCESSING on Windows console handles when available, and disable color formatting if VT mode is not supported by the console.
  • Color Auto-Detection: Auto-detect TTY output (sys.stdout.isatty()) and respect NO_COLOR standard (https://no-color.org/).
  • Flag Extraction: Allow both --color and --no-color on root parser, and extract --color, --no-color, and --benchmark_color=false|true|auto from trailing benchmark_options.
  • Suppress False Warning: Avoid printing "WARNING: passing --benchmark_color=... has no effect since both inputs are JSON" in check_inputs.
  • Test Fix & Unit Tests: Fix from gbench import util in report.py, and add unit tests in compare.py covering color flag parsing, option resolution, and NO_COLOR handling.
  • Added contributor details to AUTHORS and CONTRIBUTORS.

Verification

  • All 35 Python unit tests in tools/ pass:
    python -m unittest discover -s tools -p "*.py"
  • C++ library builds cleanly with CMake on MSVC:
    cmake --build build --config Release --target benchmark

…in compare.py (fixes google#642)

* Enable virtual terminal processing on Windows console handles when supported so ANSI escape codes are rendered natively instead of printing literal '^[[92m' sequences.
* Fall back to disabled colors when stdout is not a TTY or when the standard NO_COLOR environment variable is present.
* Support --color and --no-color on the root parser and correctly extract --color, --no-color, and --benchmark_color=false|true|auto if provided after subcommands.
* Prevent false warnings in check_inputs when --benchmark_color is provided with JSON inputs.
* Fix 'import util' in tools/gbench/report.py test to 'from gbench import util'.
* Add unit tests covering color flag parsing, option extraction, and NO_COLOR detection.
@jdymitarai
jdymitarai force-pushed the fix-compare-windows-color-642 branch from 8799b00 to 80784e6 Compare September 10, 2026 08:51
Comment thread tools/compare.py

parser.add_argument(
color_group = parser.add_mutually_exclusive_group()
color_group.add_argument(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't the opposite of --no-color just --color? why do we need both flags?

@jdymitarai

Copy link
Copy Markdown
Author

Previously, \compare.py\ only had --no-color\ (with \default=True), which meant color could only be turned off, and could never be explicitly forced on (e.g., when piping output into \less -R\ or in CI runners where stdout is not detected as an interactive TTY).

We added --color\ to allow explicitly forcing color output, while retaining --no-color\ for 100% backwards compatibility with existing user scripts and CLI habits. When neither flag is specified (\default=None), \compare.py\ auto-detects whether stdout is an interactive TTY and whether VT processing is supported (and honors the standard \NO_COLOR\ environment variable).

@dmah42

dmah42 commented Sep 10, 2026

Copy link
Copy Markdown
Member

Previously, \compare.py\ only had --no-color\ (with \default=True), which meant color could only be turned off, and could never be explicitly forced on (e.g., when piping output into \less -R\ or in CI runners where stdout is not detected as an interactive TTY).

We added --color\ to allow explicitly forcing color output, while retaining --no-color\ for 100% backwards compatibility with existing user scripts and CLI habits. When neither flag is specified (\default=None), \compare.py\ auto-detects whether stdout is an interactive TTY and whether VT processing is supported (and honors the standard \NO_COLOR\ environment variable).

that explains what it does, but not why. why do we need to force color on?

@LebedevRI

Copy link
Copy Markdown
Collaborator

(This also probably needs LLM disclosure notice.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants