Conversation
PyTorch is installed in one of three shapes (artifacts beside the Python sources, split from them, or without them), and the documentation describes the result of only one of the six commands that produce them. Code that derives asset paths from __file__ is right in some shapes and wrong in others, which is how seven call sites broke under editable installs after the scikit-build-core migration. The RFC documents the shapes from listings of the shipped 2.14.0 wheels and libtorch zips and of real editable installs on Linux, macOS and Windows; states two resolution rules for in-tree code (resources through importlib.resources behind a thin helper, directories for external tools through the existing location helper, both anchored on the import system); proposes one documentation page linked from CONTRIBUTING.md and the C++ docs; and aligns the standalone cmake --install recipe and tools/build_libtorch.py with the zips through three build defaults. Opened as a draft for discussion. Authored with assistance from Claude Code. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
The per-directory table rested on file listings that lived outside the repository. RFC-0060-assets/artifact-inventory.md now carries the sources, the per-category counts across the 2.14.0 wheels, libtorch zips and editable installs, the consumer groups, the provenance of each directory, the generated Python files, and the CI consumers of the build_libtorch.py prefix. RFC-0060-assets/list-wheel-contents.py regenerates the wheel and zip listings for any release from the zip central directories over HTTP range requests; it reproduces the 2.14.0 listings entry for entry. The RFC references both and keeps only the facts its rules and proposals rely on, so it shrinks by about 200 words with nothing dropped. Authored with assistance from Claude Code. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
…nd anchor
Sequencing item 1 said the resource helper "lands with the first call site
that reads a file", which reads as if none existed. Six do: model_dump and
the export serde schema check read package data through the functional
importlib.resources API, and the inductor aoti_runtime sources and the two
jinja template directories are read relative to __file__. Every one of
those files is tracked and also CMake-installed, so under the split shape
it exists in both trees with identical bytes, which is why they work today
and why they are the both-trees case the helper's shim orders.
The enforcement sentence and the first metric named only
importlib.resources.files("torch"); the live users call read_text(), so
the lint and the metric now cover every direct importlib.resources call.
torchgen becomes a second anchor with the same two rules. It cannot import
torch, its packaged/ data is mirrored into the install tree only, and its
public get_torchgen_root() still derives from __file__, so it needs its own
copy of the anchor chain. It is listed as a separate sequencing item.
Evidence gathered on a scikit-build-core 1.0.0 redirect editable install.
Written with an AI assistant.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
torch/_inductor/codecache.py reads csrc/inductor/aoti_runtime/model.h relative to __file__ for static-linkage AOTI. That file is the only thing PackageData.cmake installs under torch/csrc/, so under the split shape the directory exists in both trees like torch/lib does, and the read is one more both-trees resource site for the helper to take over. Written with an AI assistant. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
zklaus
added a commit
to zklaus/pytorch
that referenced
this pull request
Sep 11, 2026
Two defects in the editable-install path lookups added by pytorch#180247 share one root cause: both asked a question whose answer a source checkout can forge. `_load_global_deps` fell back to `importlib.metadata.distribution("torch")`, which returns the first torch distribution on sys.path; a checkout carrying a setuptools-era `torch.egg-info` supplies a nearer match whose `locate_file` resolves into the checkout, the `exists()` guard declines it, and CDLL proceeds with a path it already knew was missing, so `import torch` fails with an unrelated-looking dlopen error (pytorch#195293). `_compute_torch_parent` consulted the distribution only when `bin/` was absent beside the module, and `tools/build_libtorch.py` installs into `<repo>/torch` by design, so a checkout that had run it kept the fallback from firing and `get_file_path("torch")` silently returned the source tree, making every caller conditionally wrong, including the merged pytorch#195295. Both now ask the import system where torch is installed, through one helper, `_installed_torch_dir`, in the order RFC-0060 (pytorch/rfcs#109) prescribes: the editable loader's `paths` (scikit-build/scikit-build-core#1567; the entry that is not the checkout and holds `lib/`, since the order is only fixed by scikit-build/scikit-build-core#1566), then `importlib.util.find_spec("torch._C") .origin`, the directory `_C` will actually be imported from in every layout and the anchor `torch/__init__.py` already uses for the Windows DLL directory, then `__file__` when there is no spec at all (BUILD_LIBTORCH_WHL, a frozen interpreter). `torch_parent` is two levels above that directory. Instead of reasoning around leftovers, the helper detects them. In a checkout whose `_C` resolves elsewhere, an in-tree `_C`, `lib/*.so`, `bin/`, `include/`, `share/` or a `torch.egg-info` produces a warning that lists the paths and points at `spin clean`, which removes all of them. A warning rather than an error: the stale copies are never imported, they only mislead tooling that walks metadata or `__file__`. `_load_global_deps` resolves through `get_file_path` like every other consumer and raises OSError when the library is missing, which is what `ctypes.CDLL` raised and what `torch.utils.collect_env` catches. Revision history: the first revision of this change resolved the installed tree through `importlib.metadata.distributions()` with a METADATA-versus- PKG-INFO filter, and was reviewed in that form on 2026-09-04. A self-review found the indirection leaking (a built checkout shadowing a wheel could pair the wheel's `libtorch_global_deps` with the checkout's `_C`), and the review's `bin/`-probe finding pointed the same way; the import-system anchor replaces it. The review reply on the PR maps each of its points to this revision. Test Plan: Linux, redirect-mode editable install on qgpu3 (scikit-build-core 1.0.0, whose loaders have no `paths` attribute, so the `_C` spec path is the one exercised end to end; the checkout there holds a stale `_C`, 13 in-tree libraries and a `torch.egg-info`), patch applied then reverted: ``` python test/test_utils.py -k TestTorchPathResolution -k test_cmake_prefix_path -v ``` 5 passed; the warning listed all leftovers; `torch_parent` and `cmake_prefix_path` resolve to site-packages. The loader-`paths` step is pinned by `test_installed_torch_dir_prefers_the_editable_loader_paths` with a fake loader, since no released scikit-build-core exposes it yet. Windows, redirect-mode editable install (MSVC, Ninja, scikit-build-core 1.0.0), clean checkout, run against the `_C`-spec revision of this commit on 2026-09-07: ``` python test\test_utils.py -k TestTorchPathResolution -k test_cmake_prefix_path -v ``` 3 passed, 1 skipped (the global-deps test is a no-op on Windows); no warning; `cmake_prefix_path` resolves to site-packages with `TorchConfig.cmake` present. ``` spin lint -- torch/_utils_internal.py torch/__init__.py test/test_utils.py ``` Authored with an AI assistant. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
atalman
marked this pull request as ready for review
September 15, 2026 15:10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PyTorch is installed in one of three shapes (artifacts beside the Python
sources, split from them, or without them), and the documentation describes
the result of only one of the six commands that produce them. Code that
derives asset paths from file is right in some shapes and wrong in
others, which is how seven call sites broke under editable installs after
the scikit-build-core migration.
The RFC documents the shapes from listings of the shipped 2.14.0 wheels and
libtorch zips and of real editable installs on Linux, macOS and Windows;
states two resolution rules for in-tree code (resources through
importlib.resources behind a thin helper, directories for external tools
through the existing location helper, both anchored on the import system);
proposes one documentation page linked from CONTRIBUTING.md and the C++
docs; and aligns the standalone cmake --install recipe and
tools/build_libtorch.py with the zips through three build defaults.
Opened as a draft for discussion. Authored with assistance from Claude Code.
🤖 Generated with Claude Code