refactor[eve]: replace extended_typing re-exports with extra_typing - #2842
refactor[eve]: replace extended_typing re-exports with extra_typing#2842egparedes wants to merge 1 commit into
Conversation
9e9ba56 to
597ea48
Compare
597ea48 to
4abefd4
Compare
4abefd4 to
4f9a014
Compare
4f9a014 to
0e8c359
Compare
|
Following up on the compatibility-shim question: Consequently I dropped the Worth recording why a shim would not have been free: a forwarding module has to keep the |
0e8c359 to
c7bf410
Compare
c7bf410 to
5af4e4a
Compare
'gt4py.eve.extended_typing' star-imported 'typing' and 'typing_extensions', re-exported 131 names, and forwarded anything else through a module '__getattr__'. Replace it with 'gt4py.eve.extra_typing', which holds only the definitions the standard modules do not provide, and import the rest directly at the use site. The re-export layer was not just redundant, it suppressed both static checkers: - mypy cannot model a module '__getattr__', so it typed every unknown attribute as 'Any'. All 446 'xtyping.*' references in the tree were unchecked names, in the module that defines the project's type infrastructure. Enabling this surfaced two real findings: a 'type: ignore[type-abstract]' in 'next/embedded/operators.py' sitting on the closing paren instead of the argument, which had never applied because the call resolved to 'Any', and an unused ignore in this module that only survived because of its 'warn_unused_ignores' override. - ruff needed 'typing-modules = [...]' to see through the module at all; without it 'UP045' does not fire on a name imported from it. Both that setting and the mypy override are removed. The codebase had already voted: 243 of 361 source modules import 'typing' directly, only 49 touched 'extended_typing', and 29 imported from both. Destinations were resolved by the identity of each symbol's current binding, not by whether 'typing' happens to have the name: the container protocols are deliberately bound to their 'collections.abc' objects, so sending them to 'typing' would have introduced 34 PEP 585 deprecated spellings. 'Protocol', 'runtime_checkable' and 'get_type_hints' keep the 'typing_extensions' implementation they have always had -- those objects genuinely differ, 'eve' runs every annotation through 'get_type_hints', and 'next' checks '@runtime_checkable' protocols on hot paths. The per-symbol reasoning is recorded in the module docstring. 'eval_forward_ref' resolved a reference against this module's own 'globals()' when given no namespace, which worked only because the star imports happened to put every typing name there. It now builds that namespace explicitly, which is what it always meant. Behaviour is unchanged: the old and new modules were differential-tested while both existed -- 312/312 identical across 13 public functions, 27/27 on forward reference resolution. 'extra_typing' declares '__all__' of its own definitions only, and tests assert both that nothing in it is a re-export and that there is no '__getattr__' fallback -- the two properties whose absence made the old module unverifiable. Also here, because they fall out of the same work: - Close the 'xtyping.Buffer' TODO in 'storage/allocators.py': 'collections.abc.Buffer' exists on 3.12 and is the same object. - Move 'Self' from 'typing_extensions' to 'typing'. - Add './scripts/run check typing-extensions-usage', which reports the 'typing_extensions' imports a raised Python floor makes redundant. It only flags names where 'typing' provides the identical object, so the deliberate divergences above are never reported as movable.
5af4e4a to
533f720
Compare
extended_typingstar-importedtypingandtyping_extensions, re-exported 115 oftheir names, and forwarded anything else through a module
__getattr__. It is replacedby
extra_typing, holding only the definitions the standard modules do not provide;call sites now import from
typing,typing_extensionsorcollections.abcdirectly.Not just tidying — the re-export layer suppressed both static checkers. A module
__getattr__is unmodellable, so mypy typed everything reached through it asAny:249
xtyping.<name>references across 53 distinct names, in the module that definesthe project's type infrastructure.
Restoring checking surfaced two real findings, both fixed here: a
type: ignore[type-abstract]innext/embedded/operators.pythat sat on the closingparen instead of the argument and so had never applied, and an unused ignore that
survived only via this module's
warn_unused_ignoresoverride. ruff needed its ownescape hatch,
typing-modules = ['gt4py.eve.extended_typing'], without whichUP045stops firing on names imported from the module; that setting and the mypy override are
both deleted, so later work analyses plain imports natively.
Destinations were chosen by the identity of each symbol's current binding, not by
whether
typinghas the name —extended_typingrebinds the container protocols tocollections.abc, so the naive test would have introduced 7 deprecated PEP 585spellings. Of the 83 symbols in use: 39 are
extra_typing's own, 32 go totyping, 7to
collections.abc, 5 stay ontyping_extensions.Seven symbols are bound to the
typing_extensionsversion and differ from thetypingone, so each was a decision rather than a rewrite.get_type_hintsstaysbecause
everuns every annotation through it and the two differ onAnnotated/ PEP695 / PEP 649;
Protocolandruntime_checkablestay becausenext/common.pychecksfour
@runtime_checkableprotocols on hot paths, and keeping the currentimplementation is the no-behaviour-change choice.
TypeVar,TypeVarTuple,ParamSpecand
NamedTuplemove totyping, their divergences (PEP 696default=, genericnamedtuples) being unused or verified here. The reasoning is repeated in the
extra_typingdocstring.The subtle part:
eval_forward_refresolved references against this module's ownglobals()when given no namespace, which worked only because the star imports putevery typing name there.
datamodels/core.pycalls it that way inside_is_strictly_immutable_type, wrapped inexcept Exception: return False— sodeleting the star imports would not have crashed, it would have quietly started
reporting
frozen="strict"datamodels with string annotations as not immutable. Thenamespace is now built explicitly. Old and new were differential-tested while both
existed: 312/312 identical across 13 public functions, 27/27 on forward references.
extra_typingdeclares__all__of its own 61 definitions, and two tests assert whatmade the old module unverifiable: nothing in
__all__is a re-export, and there is no__getattr__fallback.Falling out of the same work: closes the
# TODO: add xtyping.Buffer once we update typing_extensionsinstorage/allocators.py; movesSelftotyping; adds./scripts/run check typing-extensions-usage, which reportstyping_extensionsimports a raised Python floor makes redundant, flagging only names where
typingprovides the identical object.
The call-site rewrite was a deterministic codemod driven by the destination map, which
aborts on any unmapped symbol rather than guessing.
Verified on 3.12, 3.13 and 3.14: 848 passed + 57 doctests each.
next/storage/cartesianunit tests: 3057 passed, 108 skipped, 13 xfailed.pre-commit run --all-filesclean, including mypy and tach. Not run locally: the backend matrix.gt4py.eve.extended_typingis removed outright, with no compatibility shim. Themodule is private: it is not in
gt4py.eve.__all__and no user documentation refersto it. A forwarding shim would in any case have to keep the module
__getattr__thatblinds the checkers, which is the hole this removes. Worth a release-note line, as the
comparable change in 1.2.1 got.