Skip to content

refactor[eve]: replace extended_typing re-exports with extra_typing - #2842

Open
egparedes wants to merge 1 commit into
mainfrom
py312-2-extra-typing
Open

refactor[eve]: replace extended_typing re-exports with extra_typing#2842
egparedes wants to merge 1 commit into
mainfrom
py312-2-extra-typing

Conversation

@egparedes

@egparedes egparedes commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

extended_typing star-imported typing and typing_extensions, re-exported 115 of
their names, and forwarded anything else through a module __getattr__. It is replaced
by extra_typing, holding only the definitions the standard modules do not provide;
call sites now import from typing, typing_extensions or collections.abc directly.

Not just tidying — the re-export layer suppressed both static checkers. A module
__getattr__ is unmodellable, so mypy typed everything reached through it as Any:
249 xtyping.<name> references across 53 distinct names, in the module that defines
the project's type infrastructure.

$ mypy -c 'from gt4py.eve import extended_typing as xtyping; reveal_type(xtyping.TotallyMadeUpSymbol)'
note: Revealed type is "Any"
Success: no issues found in 1 source file

Restoring checking surfaced two real findings, both fixed here: a
type: ignore[type-abstract] in next/embedded/operators.py that sat on the closing
paren instead of the argument and so had never applied, and an unused ignore that
survived only via this module's warn_unused_ignores override. ruff needed its own
escape hatch, typing-modules = ['gt4py.eve.extended_typing'], without which UP045
stops firing on names imported from the module; that setting and the mypy override are
both deleted, so later work analyses plain imports natively.

Destinations were chosen by the identity of each symbol's current binding, not by
whether typing has the name — extended_typing rebinds the container protocols to
collections.abc, so the naive test would have introduced 7 deprecated PEP 585
spellings. Of the 83 symbols in use: 39 are extra_typing's own, 32 go to typing, 7
to collections.abc, 5 stay on typing_extensions.

Seven symbols are bound to the typing_extensions version and differ from the
typing one, so each was a decision rather than a rewrite. get_type_hints stays
because eve runs every annotation through it and the two differ on Annotated / PEP
695 / PEP 649; Protocol and runtime_checkable stay because next/common.py checks
four @runtime_checkable protocols on hot paths, and keeping the current
implementation is the no-behaviour-change choice. TypeVar, TypeVarTuple, ParamSpec
and NamedTuple move to typing, their divergences (PEP 696 default=, generic
namedtuples) being unused or verified here. The reasoning is repeated in the
extra_typing docstring.

The subtle part: eval_forward_ref resolved references against this module's own
globals() when given no namespace, which worked only because the star imports put
every typing name there. datamodels/core.py calls it that way inside
_is_strictly_immutable_type, wrapped in except Exception: return False — so
deleting the star imports would not have crashed, it would have quietly started
reporting frozen="strict" datamodels with string annotations as not immutable. The
namespace is now built explicitly. Old and new were differential-tested while both
existed: 312/312 identical across 13 public functions, 27/27 on forward references.

extra_typing declares __all__ of its own 61 definitions, and two tests assert what
made the old module unverifiable: nothing in __all__ is a re-export, and there is no
__getattr__ fallback.

Falling out of the same work: closes the # TODO: add xtyping.Buffer once we update typing_extensions in storage/allocators.py; moves Self to typing; adds
./scripts/run check typing-extensions-usage, which reports typing_extensions
imports a raised Python floor makes redundant, flagging only names where typing
provides the identical object.

The call-site rewrite was a deterministic codemod driven by the destination map, which
aborts on any unmapped symbol rather than guessing.

Verified on 3.12, 3.13 and 3.14: 848 passed + 57 doctests each. next / storage /
cartesian unit tests: 3057 passed, 108 skipped, 13 xfailed. pre-commit run --all-files clean, including mypy and tach. Not run locally: the backend matrix.

gt4py.eve.extended_typing is removed outright, with no compatibility shim. The
module is private: it is not in gt4py.eve.__all__ and no user documentation refers
to it. A forwarding shim would in any case have to keep the module __getattr__ that
blinds the checkers, which is the hole this removes. Worth a release-note line, as the
comparable change in 1.2.1 got.

@egparedes
egparedes force-pushed the py312-2-extra-typing branch from 9e9ba56 to 597ea48 Compare August 27, 2026 18:37
@egparedes
egparedes force-pushed the py312-2-extra-typing branch from 597ea48 to 4abefd4 Compare August 28, 2026 18:45
@egparedes
egparedes force-pushed the py312-2-extra-typing branch from 4abefd4 to 4f9a014 Compare August 31, 2026 07:30
@egparedes
egparedes force-pushed the py312-2-extra-typing branch from 4f9a014 to 0e8c359 Compare August 31, 2026 09:36
@egparedes egparedes changed the title refactor[eve]!: replace extended_typing re-exports with extra_typing refactor[eve]: replace extended_typing re-exports with extra_typing Aug 31, 2026
@egparedes

Copy link
Copy Markdown
Contributor Author

Following up on the compatibility-shim question: gt4py.eve.extended_typing is being treated as private, so this lands without a shim and without a deprecation release.

Consequently I dropped the ! breaking-change marker from the title. The module is not in gt4py.eve.__all__ and nothing under docs/ refers to it, and the closest precedent — #2755 in 1.2.1, which stopped re-exporting the deprecated typing aliases from the same module — shipped as a plain fix[...] with a changelog line rather than a breaking-change marker. This still deserves that changelog line when the notes are written.

Worth recording why a shim would not have been free: a forwarding module has to keep the __getattr__ in order to forward typing names, and that __getattr__ is precisely what made mypy type 249 references across 53 names as Any. Anything still importing through the shim would keep the defect this PR removes, deprecation warning notwithstanding.

@egparedes
egparedes force-pushed the py312-2-extra-typing branch from 0e8c359 to c7bf410 Compare August 31, 2026 09:39
@egparedes
egparedes force-pushed the py312-2-extra-typing branch from c7bf410 to 5af4e4a Compare August 31, 2026 11:37
Base automatically changed from py312-1-eve-annotation-funnels to main August 31, 2026 12:05
'gt4py.eve.extended_typing' star-imported 'typing' and 'typing_extensions',
re-exported 131 names, and forwarded anything else through a module
'__getattr__'. Replace it with 'gt4py.eve.extra_typing', which holds only
the definitions the standard modules do not provide, and import the rest
directly at the use site.

The re-export layer was not just redundant, it suppressed both static
checkers:

- mypy cannot model a module '__getattr__', so it typed every unknown
  attribute as 'Any'. All 446 'xtyping.*' references in the tree were
  unchecked names, in the module that defines the project's type
  infrastructure. Enabling this surfaced two real findings: a
  'type: ignore[type-abstract]' in 'next/embedded/operators.py' sitting on
  the closing paren instead of the argument, which had never applied
  because the call resolved to 'Any', and an unused ignore in this module
  that only survived because of its 'warn_unused_ignores' override.
- ruff needed 'typing-modules = [...]' to see through the module at all;
  without it 'UP045' does not fire on a name imported from it. Both that
  setting and the mypy override are removed.

The codebase had already voted: 243 of 361 source modules import 'typing'
directly, only 49 touched 'extended_typing', and 29 imported from both.

Destinations were resolved by the identity of each symbol's current
binding, not by whether 'typing' happens to have the name: the container
protocols are deliberately bound to their 'collections.abc' objects, so
sending them to 'typing' would have introduced 34 PEP 585 deprecated
spellings. 'Protocol', 'runtime_checkable' and 'get_type_hints' keep the
'typing_extensions' implementation they have always had -- those objects
genuinely differ, 'eve' runs every annotation through 'get_type_hints',
and 'next' checks '@runtime_checkable' protocols on hot paths. The
per-symbol reasoning is recorded in the module docstring.

'eval_forward_ref' resolved a reference against this module's own
'globals()' when given no namespace, which worked only because the star
imports happened to put every typing name there. It now builds that
namespace explicitly, which is what it always meant. Behaviour is
unchanged: the old and new modules were differential-tested while both
existed -- 312/312 identical across 13 public functions, 27/27 on forward
reference resolution.

'extra_typing' declares '__all__' of its own definitions only, and tests
assert both that nothing in it is a re-export and that there is no
'__getattr__' fallback -- the two properties whose absence made the old
module unverifiable.

Also here, because they fall out of the same work:
- Close the 'xtyping.Buffer' TODO in 'storage/allocators.py':
  'collections.abc.Buffer' exists on 3.12 and is the same object.
- Move 'Self' from 'typing_extensions' to 'typing'.
- Add './scripts/run check typing-extensions-usage', which reports the
  'typing_extensions' imports a raised Python floor makes redundant. It
  only flags names where 'typing' provides the identical object, so the
  deliberate divergences above are never reported as movable.
@egparedes
egparedes force-pushed the py312-2-extra-typing branch from 5af4e4a to 533f720 Compare August 31, 2026 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant