Skip to content

Package page: read the manifest only for a format with a README, and build a group's graph once per request - #903

Merged
schmitch merged 2 commits into
mainfrom
claude/537-readme-lookup-group-graph
Nov 17, 2026
Merged

schmitch merged 2 commits into
mainfrom
claude/537-readme-lookup-group-graph

Conversation

@schmitch

Copy link
Copy Markdown
Member

Fixes #537

What was wrong

  • The README lookup. The web UI's package page, and its refresh from upstream, resolved the latest version's manifest for every format and handed it to Format::readme, which only npm implements. For NuGet the manifest was read and thrown away: a read of the stored .nuspec on each page view, and on a proxy's refresh (off the request once the listing is stale, or Refresh from upstream) a request for the upstream's .nuspec of a version nobody had downloaded, or a download of the whole package where the upstream serves no manifest.
  • The group graph. GroupRepo built its graph (the memberships below the group and the caller's grants on each member) again for every read, so a group's package page built it four times.

The change

  • A format says in its descriptor whether its manifests carry a README: Descriptor::readme (true for npm, false for NuGet, apt, raw and docker), surfaced to the host as Format::has_readme. The page (cached_detail) and refresh_from_upstream resolve the manifest only then. The descriptor check refuses the flag on a format that is not a package format. docs/format-plugins.md describes the field.
  • GroupRepo keeps the graph of each group it is asked about, by name, for its lifetime. It is made per request (AppState::read_ops_for, cached_read_ops_for), so a request's reads walk one graph and share its memoized per-id walks. Reads that arrive together wait for the one build, and a failed build is not kept.

Showing NuGet's README and the group's package-list cost (#536) are left alone, as the issue says.

Behaviour to review

  • A resolver no longer sees a change to the group made after its first read. That is the point for a request, and nothing in the server keeps a GroupRepo longer than one. Four existing tests reused one resolver across a change to a membership's scope or the group's guards (a_scoped_member_is_never_asked_about_other_ids, a_scope_on_a_nested_group_holds_for_every_member_below_it, a_scope_never_widens_what_the_guards_allow, group_search_asks_upstreams_for_the_page_window); they now make a new resolver for the reads after the change, as the server does. Their assertions are unchanged.
  • One graph per resolver, not strictly per HTTP request. The package page's own four reads share one. listing_of (the page's age and what is due for a refresh) and refresh_from_upstream each make a resolver of their own, so a group page that also refreshes still builds the graph more than once. Sharing one across them means passing the resolver through those functions; I left that out to keep this to the issue.
  • The prefer-hosted ownership query (owned_in) still runs per read: it depends on which packages exist, not on the graph.

No admin API changes, so there is no screen to add.

Tests

  • crates/server/tests/browse.rs, a_nuget_package_page_reads_no_manifest_for_a_readme: a NuGet proxy, and a group over it, refreshed from a mock upstream and then read. The upstream is asked for the listing and the registration, and for no .nuspec and no .nupkg. With the gate removed the test fails (one .nuspec request), which I checked before restoring it.
  • crates/core/tests/group_read.rs, a_resolver_builds_its_graph_once: a member added after a resolver's first read is in none of its later reads (list_versions, registration, tags, resolve_nuspec), and is in the next resolver's.
  • crates/core/src/format/descriptor.rs: the descriptor check accepts readme: true on a package format and refuses it on a files format.

What I ran

  • cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo xtask unused, cargo xtask doc-check, cargo xtask changelog --check, cargo xtask docs --check: all pass.
  • .github/scripts/test-parallel.sh --workspace (unit and integration): 132 of 134 test binaries passed on the first run; the two that failed were group_read and proxy_read, with the four tests named above. After adjusting them, both binaries pass (46 and 58 tests). I did not run the other 132 binaries again after that test-only change.
  • cargo test -p forge-e2e --features forge-e2e: everything passes except apt_import_from_a_static_repository and apt_import_from_a_real_nexus, which fail in my cloud session because their debian:bookworm container cannot resolve deb.debian.org. They do not touch this change; CI will say.

Not run locally: the web suites (no web change), the oci/raw/apt/npm client e2e scripts, and cargo sqlx prepare --check (no query changed).


Generated by Claude Code

…d a group's graph once per request

The web UI's package page and its refresh from upstream resolved the
latest version's manifest for every format and handed it to
Format::readme, which only npm implements. For NuGet the manifest was
read and discarded: a blob read on a hosted repository, and on a proxy's
refresh an upstream .nuspec fetch, or the whole package where the
upstream serves no manifest.

A format now says in its descriptor whether its manifests carry a README
(Descriptor::readme, Format::has_readme), and the page and the refresh
resolve the manifest only then.

A GroupRepo built its graph (memberships and the caller's grants) again
for each read. It is made per request, so it keeps the graph of a group
for its lifetime: the page's reads walk one graph.

Fixes #537

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e6DJZBMgAHf8FDxUa3Kpt
…reads

A GroupRepo keeps the graph it built, so the tests that change a
membership's scope or the group's guards between reads make a new
resolver for the reads after the change, as the server does for each
request.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e6DJZBMgAHf8FDxUa3Kpt
@schmitch
schmitch added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 099c841 Nov 17, 2026
19 checks passed
@schmitch
schmitch deleted the claude/537-readme-lookup-group-graph branch November 17, 2026 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants