Skip to content

feat(origin): hosted .kpar packages by download_url and checksum qualifiers - #12

Open
devin-ai-integration[bot] wants to merge 4 commits into
developfrom
feat/download-url-archives
Open

devin-ai-integration[bot] wants to merge 4 commits into
developfrom
feat/download-url-archives

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A .kpar hosted at any http(s) URL can now be named by an ECMA-427 Package-URL with the Annex B.3.5 download_url qualifier and an optional checksum, where before only vcs_url was accepted (qualifier "download_url" is not supported; only vcs_url is):

pkg:generic/<name>[@<version>]?download_url=<url>[&checksum=<algorithm>:<hex>[,…]]
  • internal/origin: parseGeneric dispatches on the qualifiers — vcs_url (repository) or download_url (hosted archive, Kind == RemoteArchive, Origin.Hosted()); the two exclude each other, checksum is only accepted with download_url. The URL must be http(s) without credentials or #, and need not end in .kpar (a release asset or an export URL serves one). Identity() is the version-free pkg:generic/<name>?download_url=encode(url); Canonical() adds @<version> (percent-encoded, decoded again on input) and checksum= first (sorted by algorithm), which add writes to .project.json.
  • internal/digest/checksum.go: ParseChecksums reads the Annex B.3.5 value (algo:hex, comma-separated, case-folded, length-checked) for md5, sha1, sha224/256/384/512 and sha3-224/256/384/512; an algorithm the tool cannot compute is refused rather than skipped. Sums/Verify/Compare hash a file once for every declared item.
  • internal/install.resolveRemoteArchive: the same download path as a plain .kpar IRI (netrc, redirects, size bound, offline), then digest.Verify before project.OpenArchive — a mismatch is an error before any member is read, and nothing is staged. The purl <name> must equal the archive's .project.json name; the version is the archive's own, held to @<version>/versionConstraint as before. The lock entry is kind: archive, source: <url>, sha256: <archive digest>.
  • Every usage of a hosted archive is held to the checksums it declares (verifyDeclared): the resolver keeps the digests known for an archive in the run (resolver.sums: the lock's SHA-256, what the download or the installed copy verified) and compares each usage — a second usage of an already-chosen archive included — to them. A locked entry is reused only when every declared item was checked (sha256 against the lock, others against an installed copy whose SHA-256 is the lock's); otherwise, lock-only runs included, the archive is downloaded and checked, and a damaged lib/ copy is replaced rather than reported.
  • project.Marshal/Meta.Marshal no longer HTML-escape (& was written as \u0026 into .project.json), matching lock.Marshal.
  • README "Identity"/"Other sources", docs/packages.md "Hosted archives", syspi -h, changelog fragment.

Tests: origin round-trips (including @1.2.3%2Bci) and refusals, digest checksum parsing/verification, install.TestHostedArchive (wrong checksum leaves the tree untouched; lock, offline reuse, second algorithm, conflicting second usage, lock-only fresh clone, damaged copy, version and name checks), e2e.TestHostedArchive (bundle → httptest server → add with a wrong and a right checksum → offline install/lock).

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/6f003dfe90194f8a96e42bde58a29e3f
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/6f003dfe90194f8a96e42bde58a29e3f?variant=devin
Requested by: @HuiJun


🟢 Devin fixed all 4 findings · Follow Devin's work

Devin Review

…ifiers

A pkg:generic/<name>[@<version>]?download_url=<url>[&checksum=<algo>:<hex>]
purl names a .kpar at any http(s) URL (ECMA-427 Annex B.3.5). The archive
is fetched as a plain .kpar IRI is, the declared checksums are verified
before it is opened, and it installs as lib/<publisher>-<name>_<version>.kpar
with the archive digest in the lock. vcs_url and download_url exclude each
other; add writes the canonical form, and .project.json no longer escapes
& as \u0026.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration
devin-ai-integration Bot marked this pull request as ready for review November 12, 2026 06:25
devin-ai-integration Bot and others added 3 commits November 12, 2026 06:29
The canonical form writes @1.2.3%2Bci; parsing it validated the encoded
text and refused the build metadata.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…ecksums

A second usage of an archive already resolved in the run, and a locked
entry reused without its installed copy, were not checked against the
checksums they declare. The resolver now keeps the digests known for an
archive in the run, compares each usage to them, and downloads an archive
a lock-only run cannot otherwise check.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
…e declared checksums

verifyDeclared reads a copy only when its SHA-256 is the lock's; otherwise
the download replaces it and is checked.

Co-Authored-By: jason.han <hanhuijun@gmail.com>

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 4 findings. Devin automatically fixed all 4 findings.

Fixed by Devin (4)
  • Hosted build versions fail to install — parseGeneric percent-decodes the purl version before validating it, so the canonical @1.2.3%2Bci round-trips; origin.TestHostedArchive covers %2B and a raw +.
  • Lock accepts unverified hosted checksum — A locked entry is reused only when every declared checksum was checked (sha256 against the lock, others against the installed copy); otherwise, lock-only included, the archive is downloaded and checked. install.TestHostedArchive covers a lock-only fresh clone with a wrong and a right sha1.
  • Damaged archive blocks online repair — verifyDeclared reads the installed copy only when its SHA-256 is the lock's; a damaged copy is treated as absent, so the archive is downloaded again and checked. install.TestHostedArchive covers a damaged lib/ copy with a declared sha1.
  • Duplicate usages bypass archive checksum — resolve() now holds a usage of an already-chosen archive to its own checksums (verifyDeclared against the digests known this run or the staged/installed copy); a conflicting second usage is a checksum mismatch error. Covered by install.TestHostedArchive.

View findings in Devin Review · Follow Devin's work

Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant