[mypyc] Synchronize native-to-native imports

[p-sawicki]

Fixes #21808

Native-to-native imports within the same compilation group were sped up in #21101 to improve mypy cold-start performance on macOS, since regular imports going through the shim significantly slowed it down. With this change a native-to-native import does not go through the regular Python import machinery.

This means that native-to-native imports are not synchronized by the module lock used by the regular Python import process, which guarantees that only one thread can import a given module at a time.

Missing this synchronization means that a thread importing a native module might see it in a partially-initialized state, leading to errors. More details about this are available in comments to the linked issue.

To add synchronization, we use the module lock from the Python import library. Writing a custom native lock might be possible, but it would be difficult because of edge cases such as circular imports, modules waiting at top-level on threads importing other native modules, or interpreted and native modules concurrently importing a native module. Using the Python module lock does not regress performance, as the mypy self-check comparison does not produce a significant difference (table below). mypy cold-start time on macOS is also pretty much the same before and after this change.

mypy self-check comparison:

Python	Baseline	PR	Difference
3.13, GIL	2.124s ± 0.020	2.118s ± 0.024	-0.3%
3.14t, no GIL	2.756s ± 0.039	2.768s ± 0.042	+0.4%

The module lock API is obtained when initializing a group shared library by importing from importlib._bootstrap. When importing a native module, this API is used to obtain the lock associated with the module. This is the same lock that would be used when importing the module in an interpreted module. To ensure compatibility in case the native module is imported both through Python and natively at the same time, the native import function now sets module.__spec__._initializing to True when the module is being imported.

There's a fast path in case the native module is imported multiple times. Each native module defines a boolean flag that is set to true after it has been imported, so future imports can immediately return and not take the module lock after reading the flag. The flag is set in a wrapper over the module exec function so it will be set even if the first import is not native-to-native.

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅