Repository navigation
fix(utilities): serialise first resolution of lazy exports and module descriptors so concurrent imports cannot deadlock (#2771) - #2781
Merged
Conversation
… descriptors so concurrent imports cannot deadlock (#2771) - Threads making first imports at once raised _DeadlockError: a family member's protocol module imports a module of its base's package, and the import system takes that module's lock before waiting on the base's package, while a thread importing the package takes the two the other way. - pcapkit.utilities.importlib.import_lock, a shared RLock, is held by every first resolution in lazy_exports and by ModuleDescriptor.klass when its module is not imported yet; resolved names and imported modules are read without it. - IGMP's base and version classes on the dispatch path resolve through descriptors rather than function-level imports, which bypassed the lock. - ProtocolBase._registry_lock goes (#2773): get_protocol resolves through the lock. - The ALL-tier race test covers lazy names, family-member dispatch and get_protocol, and fails on any first import made without the lock; it fails on the unfixed tree.
JarryShaw
added
fix
… the shared import lock (#2771) - Function-level imports of modules `import pcapkit` leaves unloaded made their first import without import_lock, so they could still deadlock against a locked one. - They resolve through module-level ModuleDescriptor constants instead, as IGMP's do: HTTP's and LinuxSLL's members and data models, L2TP's member, the TCP and UDP port registries of the __index__ hooks, the ip_pseudo helpers of IPv4, IPv6, ICMPv6, TCP, UDP and DCCP, verify_upper's AH, and the PCAP and PCAP-NG engines' toolkits. - The race test makes these first resolutions at once with checksums verified, and a new project test fails on any function body importing a module first, bar the sites it allows with their reasons.
Contributor
|
Coverage: 94.45% (unit tier, Python 3.15,
Per-file detail: the |
Owner
Author
|
GOOD TO GO at
|
JarryShaw
added
review: good-to-go
JarryShaw
removed
the
review: good-to-go
JarryShaw
added a commit
that referenced
this pull request
Nov 4, 2026
- #2777: the cut-header sentence reads the length back too, the rebuild read at the width its parse was given (__short_width__); the ruling into the conventions entry. - #2778: a PCAP-NG block with octets after opt_endofopt rebuilds an info holding its options and padding_opts, where 1.5.1 rebuilt none, into the whole-header rebuild entry; 1.5.0's frozen entry is left as written. - #2779: SCTP's reorder, the source-ordered member lists and the sorted Option.register clash warning ('opt', 'if', 'epb', ... in 1.5.1), into the member-order entry; the layer-grouped registry and the warning name restore ('put back silently' in 1.5.1), into the #2770 entry; the page wording into the conventions entry. - #2781: import_lock serialising first resolutions, into the lazy-import entry; the removed registry lock never shipped and stays unnamed. - CHANGELOG.md regenerated; util/changelog_md.py --check passes.
1 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
make pylint,make mypy,make isort)make testpasses, and a test case covers the changeWhat is the purpose of your pull request?
fix— corrects a defectDescription of your pull request and other information
Since #2757, threads that reach different lazy names or first dispatches at the same time, on a cold import, deadlock the import system (
_DeadlockError). A family member's module imports its base package's modules while another thread holds that package's import lock.pcapkit.utilities.importlib.import_lock, a re-entrant lock, now serialises the first resolution inlazy_exports. A name, once resolved, is bound on its module, so later reads take no lock.ModuleDescriptor.klasstakes the same lock, but only when its module isn't imported yet. This covers next-layer dispatch andget_protocol.ProtocolBase._registry_lockis removed, as ruled on refactor(protocols): align ProtocolBase.__registry__ handling and entry layout with the other registry maps #2773 (option c)..klassis unchanged, and warmget_protocoldrops from 460 to 348 ns.get_protocolraced with dispatch. On main they fail in 19–20 of 20 trials on 3.14; with this fix, 0 of 100. A check also fails on any first import made without the lock.__index__hooks, theip_pseudohelpers, the engines' toolkit) now go through module-level descriptors. A new static test scans every function body and fails on an unlisted first import of a moduleimport pcapkitleaves unloaded. Each remaining plain import is listed with its reason.Closes #2771
Part of #2773