Skip to content

gh-153319: Read turtledemo scripts with the source file's own encoding#153321

Open
tonghuaroot wants to merge 1 commit into
python:mainfrom
tonghuaroot:fix-turtledemo-encoding
Open

gh-153319: Read turtledemo scripts with the source file's own encoding#153321
tonghuaroot wants to merge 1 commit into
python:mainfrom
tonghuaroot:fix-turtledemo-encoding

Conversation

@tonghuaroot

@tonghuaroot tonghuaroot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

turtledemo's example viewer opened each demo script for display with
open(self.module.__file__, 'r') (no encoding), decoding with the locale
default encoding. That emits an EncodingWarning under
-X warn_default_encoding and, on a non-UTF-8 locale, can mis-decode a UTF-8
or coding-cookie'd demo, showing mojibake.

This reads the source with tokenize.open() instead, which decodes using the
encoding detected from the file (its PEP 263 coding cookie, else UTF-8). That
is the correct encoding for reading Python source, and it matches how the
module was imported one line above. It is the only unspecified-encoding
open() in Lib/turtledemo (grep-verified).

The viewer is Tkinter GUI code with no test harness (test_turtle.py covers
the turtle module, not the demo runner), so this was verified manually in
both directions: under -X warn_default_encoding the old open() raises an
EncodingWarning and the new tokenize.open() does not, while reading the
file unchanged.

…ncoding

The demo viewer read each script with open() and no encoding argument, i.e. the locale default encoding. That emits an EncodingWarning under -X warn_default_encoding and can mis-decode a script on a non-UTF-8 locale. Read the source with tokenize.open() instead, which decodes using the encoding detected from the file (its coding cookie, else UTF-8), matching how Python read the module when it was imported just above.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant