Repository navigation
Assignment of a signed value 'str' to a bigger unsigned variable 'ucs4' at /Objects/unicodeobject.c #161429
Description
Bug report
Potential Problem
In /Objects/unicodeobject.c in the PyUnicode_4BYTE_KIND branch https://github.com/python/cpython/blob/3.12/Objects/unicodeobject.c#L1922. A static analyzer has identified a genuine issue: *str is of type char (which is signed on x86/x64 architectures). Consequently, when Latin-1 bytes in the range 0x80–0xFF are cast to Py_UCS4, they undergo sign extension—for example, 0xE9 becomes 0xFFFFFFE9. This value exceeds MAX_UNICODE and violates the invariants of the str object. This code path can be triggered via the exported function _PyUnicodeWriter_WriteLatin1String when using a PyUnicode_4BYTE_KIND buffer (while not invoked internally by CPython itself, the issue can manifest in extensions). The fix is to introduce an intermediate cast to Py_UCS1
case PyUnicode_4BYTE_KIND: {
Py_UCS4 *start = (Py_UCS4 *)data + index;
Py_UCS4 *ucs4 = start;
for (; str < end; ++ucs4, ++str)
*ucs4 = (Py_UCS4)*str;
assert((ucs4 - start) <= PyUnicode_GET_LENGTH(unicode));
break;
}
default:
Py_UNREACHABLE();
}
}
Possible Solution
Add an intermediate cast to Py_UCS1:
*ucs4 = (Py_UCS4)(Py_UCS1)*str;
Found by Linux Verification Center (portal.linuxtesting.ru (https://portal.linuxtesting.ru/)) with SVACE.
Author D. Babushkin.
CPython versions tested on:
3.12
Operating systems tested on:
No response
Activity
divya-git03 commentedon Nov 5, 2026
Hi! I'd like to work on this issue.
I understand that the problem is caused by sign extension when converting Latin-1 bytes (
0x80–0xFF) fromchartoPy_UCS4in thePyUnicode_4BYTE_KINDbranch.The proposed intermediate cast to
Py_UCS1appears to address this by ensuring the byte is treated as unsigned before widening.I'd like to investigate the affected code path, implement the fix, and add an appropriate regression test if feasible.
Would a PR addressing this change be welcome?
gvanrossum commentedon Nov 5, 2026
@vstinner -- you touched this function last. But isn't it deprecated?
vadmium commentedon Nov 5, 2026
Doesn't the fix make the outer (Py_UCS4) explicit cast redundant?
Also, it looks to me a similar problem exists with the PyUnicode_2BYTE_KIND branch directly above. It won't exceed the Unicode range but will still store the wrong code point value.
vstinner commentedon Nov 6, 2026
@Danila-Babushkin: You analyzed outdated code. Please re-run your analyzer on the current main branch. I close the issue.
This code is outdated: the URL points to Python 3.12 source code which no longer accepts bugfixes.
In 3.14, 3.15 and main branches, _PyUnicodeWriter_WriteLatin1String() now uses:
Yeah, I noticed this bug recently, I fixed it and I added tests: commit 26d3f5d.
@gvanrossum:
Correct, it's a private function deprecated since Python 3.14. It's scheduled for removal in Python 3.18.