Give the Unicode code point a name of its own - #1409
Open
vmscvx wants to merge 1 commit into
Open
Conversation
Mosh moves a single code point between the UTF-8 parser, the emulator
and the display in a wchar_t. That works wherever wchar_t is 32 bits
wide, which is everywhere mosh currently builds, but it is not portable:
on Windows wchar_t is 16 bits and cannot hold anything above U+FFFF, so
the type has to become char32_t there.
This introduces mosh_wchar_t, mosh_wstring and MOSH_L() in
src/util/unicode.h, and switches src/terminal and src/frontend over. On
POSIX all three resolve to exactly what the code says today, so the
generated code does not change; the only reason to land it separately is
that it is a wide mechanical diff and reviewing it alongside actual
Windows support would be unpleasant.
Two things come along that stand on their own:
- Cell::append() and Cell::append_to_str() each carried an identical
copy of the same wcrtomb() dance. Both now call mosh_append_utf8().
- src/tests/unicode covers the UTF-8 round trip, ill-formed and
truncated input, and the width of combining, wide and control
characters. There was no test for any of that before. It skips, in
the manner of the other locale-dependent tests, when the environment
offers no UTF-8 locale.
The Windows halves of unicode.h and unicode.cc are included here rather
than held back, so the reason for the abstraction is visible in the same
patch that introduces it. They compile to nothing anywhere else.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mosh moves a single code point between the UTF-8 parser, the emulator
and the display in a wchar_t. That works wherever wchar_t is 32 bits
wide, which is everywhere mosh currently builds, but it is not portable:
on Windows wchar_t is 16 bits and cannot hold anything above U+FFFF, so
the type has to become char32_t there.
This introduces mosh_wchar_t, mosh_wstring and MOSH_L() in
src/util/unicode.h, and switches src/terminal and src/frontend over. On
POSIX all three resolve to exactly what the code says today, so the
generated code does not change; the only reason to land it separately is
that it is a wide mechanical diff and reviewing it alongside actual
Windows support would be unpleasant.
Two things come along that stand on their own:
Cell::append() and Cell::append_to_str() each carried an identical
copy of the same wcrtomb() dance. Both now call mosh_append_utf8().
src/tests/unicode covers the UTF-8 round trip, ill-formed and
truncated input, and the width of combining, wide and control
characters. There was no test for any of that before. It skips, in
the manner of the other locale-dependent tests, when the environment
offers no UTF-8 locale.
The Windows halves of unicode.h and unicode.cc are included here rather
than held back, so the reason for the abstraction is visible in the same
patch that introduces it. They compile to nothing anywhere else.
This is groundwork for a native Windows build of mosh-client, which needs
char32_t where wchar_t is 16 bits wide. The remaining pieces are separate
branches and are not proposed here; this one stands on its own as a rename
plus a test, and changes nothing on POSIX.