Repository navigation
Package page: read the manifest only for a format with a README, and build a group's graph once per request - #903
Merged
Conversation
…d a group's graph once per request The web UI's package page and its refresh from upstream resolved the latest version's manifest for every format and handed it to Format::readme, which only npm implements. For NuGet the manifest was read and discarded: a blob read on a hosted repository, and on a proxy's refresh an upstream .nuspec fetch, or the whole package where the upstream serves no manifest. A format now says in its descriptor whether its manifests carry a README (Descriptor::readme, Format::has_readme), and the page and the refresh resolve the manifest only then. A GroupRepo built its graph (memberships and the caller's grants) again for each read. It is made per request, so it keeps the graph of a group for its lifetime: the page's reads walk one graph. Fixes #537 Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012e6DJZBMgAHf8FDxUa3Kpt
…reads A GroupRepo keeps the graph it built, so the tests that change a membership's scope or the group's guards between reads make a new resolver for the reads after the change, as the server does for each request. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012e6DJZBMgAHf8FDxUa3Kpt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #537
What was wrong
Format::readme, which only npm implements. For NuGet the manifest was read and thrown away: a read of the stored.nuspecon each page view, and on a proxy's refresh (off the request once the listing is stale, or Refresh from upstream) a request for the upstream's.nuspecof a version nobody had downloaded, or a download of the whole package where the upstream serves no manifest.GroupRepobuilt its graph (the memberships below the group and the caller's grants on each member) again for every read, so a group's package page built it four times.The change
Descriptor::readme(truefor npm,falsefor NuGet, apt, raw and docker), surfaced to the host asFormat::has_readme. The page (cached_detail) andrefresh_from_upstreamresolve the manifest only then. The descriptor check refuses the flag on a format that is not a package format.docs/format-plugins.mddescribes the field.GroupRepokeeps the graph of each group it is asked about, by name, for its lifetime. It is made per request (AppState::read_ops_for,cached_read_ops_for), so a request's reads walk one graph and share its memoized per-id walks. Reads that arrive together wait for the one build, and a failed build is not kept.Showing NuGet's README and the group's package-list cost (#536) are left alone, as the issue says.
Behaviour to review
GroupRepolonger than one. Four existing tests reused one resolver across a change to a membership's scope or the group's guards (a_scoped_member_is_never_asked_about_other_ids,a_scope_on_a_nested_group_holds_for_every_member_below_it,a_scope_never_widens_what_the_guards_allow,group_search_asks_upstreams_for_the_page_window); they now make a new resolver for the reads after the change, as the server does. Their assertions are unchanged.listing_of(the page's age and what is due for a refresh) andrefresh_from_upstreameach make a resolver of their own, so a group page that also refreshes still builds the graph more than once. Sharing one across them means passing the resolver through those functions; I left that out to keep this to the issue.owned_in) still runs per read: it depends on which packages exist, not on the graph.No admin API changes, so there is no screen to add.
Tests
crates/server/tests/browse.rs,a_nuget_package_page_reads_no_manifest_for_a_readme: a NuGet proxy, and a group over it, refreshed from a mock upstream and then read. The upstream is asked for the listing and the registration, and for no.nuspecand no.nupkg. With the gate removed the test fails (one.nuspecrequest), which I checked before restoring it.crates/core/tests/group_read.rs,a_resolver_builds_its_graph_once: a member added after a resolver's first read is in none of its later reads (list_versions,registration,tags,resolve_nuspec), and is in the next resolver's.crates/core/src/format/descriptor.rs: the descriptor check acceptsreadme: trueon a package format and refuses it on a files format.What I ran
cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warnings,cargo xtask unused,cargo xtask doc-check,cargo xtask changelog --check,cargo xtask docs --check: all pass..github/scripts/test-parallel.sh --workspace(unit and integration): 132 of 134 test binaries passed on the first run; the two that failed weregroup_readandproxy_read, with the four tests named above. After adjusting them, both binaries pass (46 and 58 tests). I did not run the other 132 binaries again after that test-only change.cargo test -p forge-e2e --features forge-e2e: everything passes exceptapt_import_from_a_static_repositoryandapt_import_from_a_real_nexus, which fail in my cloud session because theirdebian:bookwormcontainer cannot resolvedeb.debian.org. They do not touch this change; CI will say.Not run locally: the web suites (no web change), the
oci/raw/apt/npmclient e2e scripts, andcargo sqlx prepare --check(no query changed).Generated by Claude Code