[Lillecarl]

TL;DR

With lazy imports, a package's lazy from .sub import sub can end up bound to the submodule pkg.sub instead of the object it names. This happens when other code imports pkg.sub before anything uses the name: the import system binds each newly loaded submodule on its parent, and that setattr overwrites the lazy placeholder. Without lazy, the package's own binding wins.

The fix skips that setattr when the name on the parent is a lazy import that would itself import the submodule. It costs one dict lookup per submodule import.

Details

When pkg/__init__.py does lazy from .sub import sub and something else imports pkg.sub first, _find_and_load_unlocked sets pkg.sub to the submodule over the package's placeholder, and the binding is lost. Eagerly, the statement imports the submodule itself and keeps its own binding.

_find_and_load_unlocked now skips that setattr when the parent binds the name to a lazy import of the child, or of a module inside it: run eagerly, such a statement would already have imported the child. _imp._lazily_binds_child() makes that check in C, because Python code cannot read a placeholder's target without resolving it. A binding of any other module is still overwritten, as eagerly. The call is not wrapped in try, unlike _set_lazy_attributes() below it: it can only fail if comparing the name raises.

Two behaviours to review:

• The check sees the binding as it stands when the child is imported. If the package rebinds the name after the lazy statement (sub = "override"), the outside import still overwrites it, where eagerly "override" stays. This is unchanged.
• A placeholder that cannot resolve stays bound, and raises on use. With lazy from pkg.sub import missing as sub, pkg.sub gave the module and now raises ImportError. The eager package fails at import, so this follows PEP 810's error timing.

Alternatives I tried:

• Resolve the placeholder when the child is imported, instead of keeping it. Rejected: resolving lazy from .sub.deep import x as sub then imports pkg.sub.deep as a side effect of someone else's import pkg.sub, and a placeholder that cannot resolve makes that unrelated import raise.
• Fix it in module getattro, as Import of submodule shadowing parent module attribute fails under lazy_imports mode #151208 discusses. Not tried, because it cannot work here: the setattr has already replaced the placeholder in the dict, so a read finds the module and nothing is left to intercept. LOAD_ATTR_MODULE would also bypass it, as noted on Import of submodule shadowing parent module attribute fails under lazy_imports mode #151208.
• Drop lazy_import_replace_child (Implement PEP 810 – Explicit lazy imports #142349), since the placeholder now stays bound while it resolves. With its call disabled on top of this change, the same test suites pass, also under -X lazy_imports=all. I left it in to keep this change small, and can remove it here if you prefer one mechanism.

Tests: six in test_lazy_import.AttributeSideEffectTests, for the lazy and -X lazy_imports=all spellings, a module inside the child, a from . import sub, and two bindings that must still be overwritten. The four that keep a binding fail without the fix. test_lazy_import, test_import, test_importlib, test_pkg, test_module, test_pkgutil, test_runpy and test_zipimport pass. Under -X lazy_imports=all this adds no failure and fixes test_import.RelativeImportTests.test_rebinding. The new tests pass -R 3:3.

Cost: one C call and one dict lookup per submodule import, under 50 ns including the call from Python. On optimised builds, with and without the GIL, re-importing 2,000 submodules of a package takes 26 µs per import (33 µs free-threaded) with or without the change, within noise.

Free threading: a placeholder's fields do not change after creation, and PyDict_GetItemRef returns a strong reference, so the check takes no lock. The check and the setattr run while the import holds the child's module lock. Resolving the placeholder imports the child, so a resolution in another thread waits for both. On the free-threaded build the tests above pass. A stress test with 8 threads importing the child keeps the binding in 300 of 300 rounds, where stock loses it in all 300.

Real case: cryptography's hazmat/backends/openssl/__init__.py does from cryptography.hazmat.backends.openssl.backend import backend, so the name backend is both a submodule and the object the package exports. cryptography.hazmat.backends.default_backend() imports that submodule inside a function. The SSH library asyncssh then does from cryptography.hazmat.backends.openssl import backend and calls backend.ed25519_supported(). With cryptography's imports made lazy through __lazy_modules__ and default_backend() called first, stock 3.15.0 gives asyncssh the submodule, and the call raises AttributeError. With this fix backported to 3.15.0, asyncssh gets the backend object and generates an ed25519 key.

I used an AI assistant (Claude) to find the cause, write the patch and tests, and check the edge cases. I reviewed the change and can explain it.

• Issue: Lazy from .sub import sub in a package is overwritten when the submodule is imported elsewhere #161265

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[Lillecarl]

The "Lazy imports enabled" job found one more module this fixes: test_unittest now passes under -X lazy_imports=all.

unittest/__init__.py does from .main import main. When the submodule unittest.main was imported elsewhere first, it replaced the function, and unittest.main() raised TypeError: 'module' object is not callable in six test_program tests. 2319f59 removes test_unittest from Lib/test/lazy_imports_all_exclude.txt.