Repository navigation
fix(pkgmap): keep one manifest's candidates in its parser's order (regression from #2974) - #2989
Merged
Merged
Conversation
cd31825 made the package map independent of worker scheduling by sorting every candidate of a package name by depth and path before the first-wins insertion. That also reordered the candidates of ONE manifest: a pyproject.toml pushes src/<name> first and <name> second, the first had always won, and after the sort the shallower second one did, for every Python package. Imports that used to bind their own package then fell to the same-name fallback: in flask, `from flaskr import create_app` in examples/tutorial bound the celery example's create_app. The dispatcher now stamps each entry with the manifest that yielded it and its position among that manifest's entries, and the merge orders by package name, the manifest's depth and path, and that position. Different manifests are still ordered by the repository, not by the run; one manifest's entries keep the order its parser chose. ei_pkgmap_one_manifest_keeps_its_candidate_order hands a pyproject.toml and a sibling's to the merge in both worker orders; ei_python_example_project_import_binds_its_own_package indexes the flask shape and checks the import's target file. Both fail without this change. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes a regression that #2974 introduced in the package map (commit
cd318253e, mine).That commit made the package map independent of worker scheduling by sorting every candidate of a package name by
depth and path before the first-wins insertion. The sort also reordered the candidates of one manifest: a
pyproject.tomlpushessrc/<name>first and<name>second, the first had always won, and after the sort theshallower second one did, for every Python package. Imports that used to bind their own package then fell to the
same-name fallback.
Measured with the frozen call-graph metric, main before #2974 against main today against this branch:
In flask the wrong binding is visible directly:
examples/tutorial/tests/conftest.pydoesfrom flaskr import create_app, and main today binds it to the celery example'screate_app. On the 23 Pythontruth repos this branch gives the pre-#2974 found and correct counts everywhere. The CALLS counts that still differ
from before #2974 (pydantic, matplotlib, airflow and a few smaller ones) are the same with and without this change
and come from #2977.
The fix
cbm_pkgmap_try_parsestamps each entry with the manifest that yielded it and its position among that manifest'sentries. The merge orders by package name, then the manifest's depth and path, then that position. Different
manifests are still ordered by the repository and not by the run, which is what #2974 was for: cli/cli gives 3,869
IMPORTS edges in 12 of 12 runs with this branch. One manifest's entries keep the order its parser chose.
Tests
edge_importsei_pkgmap_one_manifest_keeps_its_candidate_order(apyproject.tomland a sibling's, handed to the merge in both worker orders)edge_importsei_python_example_project_import_binds_its_own_package(the flask shape, checked by the import's target file)On main today
edge_importsgives 88 passed, 2 failed; with the changeedge_imports,pipeline,lang_contractand
registrygive 755 passed, 0 failed under ASan and UBSan. The two #2974 tests for go.mod manifests passunchanged. clang-format on the changed lines, the memory-core and NOLINT lints. The full local ladder was not run.
Why #2974 did not catch it
Its tests covered go.mod manifests only and no Python repository was scored before the merge. The scoring run that
found this now exists as a script and is run before a resolver or package-map change merges.
Checklist
git commit -s)make -f Makefile.cbm test): only the affected suites were run locally, PR CI runs the rest