Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,18 @@ what package managers install.
## The contract, on one screen

- **Identity** — an ECMA-427 Package-URL of type `generic` with the Annex B.3.5 `vcs_url`
qualifier: `pkg:generic/<name>?vcs_url=<vcs>+<transport>://<host>/<path>[@<version>][#<subpath>]`,
qualifier: `pkg:generic/<name>?vcs_url=<vcs>+<transport>://[<user>@]<host>/<path>[@<version>][#<subpath>]`,
the qualifier percent-encoded (`+` → `%2B`, `/` → `%2F`, `@` → `%40`, `#` → `%23`, `:` kept), e.g.
`pkg:generic/OpenSysML-Extensions-Library?vcs_url=git%2Bhttps:%2F%2Fgithub.com%2FOpen-MBEE%2FOpenSysML-Extensions-Library%401.0.0`.
`<vcs>` is `git`, `svn` or `hg`, `<transport>` one of Annex B.3.5's (`git`, `https`, `http`,
`ssh` for git; `http`, `https`, `static-http`, `ssh` for hg; `svn`, `http`, `https`, `ssh` for svn)
plus `file` for a repository on this machine; `<name>` is the repository's last path segment.
`<user>@` is the SSH login and is accepted on `<vcs>+ssh` only
(`git+ssh://git@github.com/owner/repo@1.2.0`); it is part of the identity, so it is written to
`.project.json` and the lock as `…git%2Bssh:%2F%2Fgit%40github.com%2F…`, and the same location
without it is another identity. A password, or a user name on any other transport, is refused.
Two shorter spellings are accepted as input and normalised to it: `pkg:git/<host>/<path>[@<version>][#<subpath>]`
(git over https) and the bare notation `git+https://host/path@1.2.0`, `svn://host/repo@1.2.0`,
(git over https) and the bare notation `git+https://host/path@1.2.0`, `git+ssh://git@host/path@1.2.0`, `svn://host/repo@1.2.0`,
`hg+https://…`, `git+file:///srv/repo@1.2.0` (`git+file:///C:/repo@1.2.0` on Windows). Refused with a message naming those three:
`pkg:svn/…`, `pkg:hg/…`, `pkg:github/…`, plain clone URLs (`https://host/path.git`, `ssh://…`,
`git@host:path`, `file:///path@1.2.0`) and a commit id as version. SysPI hand-parses this one
Expand Down
6 changes: 6 additions & 0 deletions changes/unreleased/ssh-user-name.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
- A user name is accepted before the host of a `<vcs>+ssh://` location —
`git+ssh://git@github.com/owner/repo@1.0.0`, `hg+ssh://…`, `svn+ssh://…` — so an SSH remote no
longer needs a `User` entry in `ssh_config`. It is the SSH login, part of the identity
(`…git%2Bssh:%2F%2Fgit%40github.com%2F…` in `.project.json` and the lock) and of the address
the VCS executable is given. A password anywhere, and a user name on any other transport,
stay refused.
8 changes: 5 additions & 3 deletions cmd/syspi/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,12 @@ Commands:

A package is any git, svn or hg repository with .project.json at a tag, identified by an
ECMA-427 Package-URL (vcs: git, svn, hg; transports per Annex B.3.5, plus file):
pkg:generic/<name>?vcs_url=<vcs>%2B<transport>:%2F%2F<host>%2F<path>[%40<version>][%23<subpath>]
pkg:generic/<name>?vcs_url=<vcs>%2B<transport>:%2F%2F[<user>%40]<host>%2F<path>[%40<version>][%23<subpath>]
Accepted as input and normalised to it:
pkg:git/<host>/<path>[@<version>][#<subpath>]
<vcs>+<transport>://<host>/<path>[@<version>][#<subpath>]
pkg:git/<host>/<path>[@<version>][#<subpath>] (git over https)
<vcs>+<transport>://[<user>@]<host>/<path>[@<version>][#<subpath>]
<user>@ is the SSH login, accepted on <vcs>+ssh only (git+ssh://git@github.com/owner/repo@1.0.0)
and part of the identity; a password, or a user name on another transport, is refused.
Run "syspi <command> -h" for the flags of a command.
`

Expand Down
62 changes: 58 additions & 4 deletions docs/packages.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,14 @@ types and go in a qualifier) of type `generic` carrying the Annex B.3.5 `vcs_url

```
resource = "pkg:generic/" name "?vcs_url=" encode(vcs_url)
vcs_url = vcs "+" transport "://" host "/" path [ "@" version ] [ "#" subpath ]
vcs_url = vcs "+" transport "://" [ user "@" ] host "/" path [ "@" version ] [ "#" subpath ]
vcs = "git" | "svn" | "hg"
transport = git: "git" | "https" | "http" | "ssh" (Annex B.3.5; "git" and "git+git" are one)
hg: "http" | "https" | "static-http" | "ssh"
svn: "svn" | "http" | "https" | "ssh" ("svn" and "svn+svn" are one)
any: "file" (a repository on this machine, pip's notation; host empty, path absolute)
name = last segment of path, a ".git" suffix stripped
user = SSH login, on transport "ssh" only; as written (case kept), never with a password
host = DNS name (lower-cased), with an optional port
path = repository path on the host as written (a ".git" suffix stays), without a leading "/"
version = semantic version (MAJOR.MINOR.PATCH[-pre][+build]) naming a tag; an exact constraint
Expand All @@ -25,13 +26,14 @@ subpath = directory within the tagged tree that is the project directory
percent-encoding is accepted on input. Examples:

- `pkg:generic/OpenSysML-Extensions-Library?vcs_url=git%2Bhttps:%2F%2Fgithub.com%2FOpen-MBEE%2FOpenSysML-Extensions-Library%401.0.0`
- `pkg:generic/OpenSysML-Extensions-Library?vcs_url=git%2Bssh:%2F%2Fgit%40github.com%2FOpen-MBEE%2FOpenSysML-Extensions-Library%401.0.0`
- `pkg:generic/rover?vcs_url=hg%2Bhttps:%2F%2Fhg.example.org%2Fmodels%2Frover%23packages%2Fpower`
- `pkg:generic/rover?vcs_url=svn%2Bsvn:%2F%2Fsvn.example.org%2Frepos%2Frover%401.0.0`
- `pkg:generic/rover?vcs_url=git%2Bfile:%2F%2F%2Fsrv%2Fgit%2Frover.git%401.0.0`

Two input spellings normalise to it (and `syspi add` writes the canonical form into
`.project.json`): `pkg:git/<host>/<path>[@<version>][#<subpath>]`, meaning `git+https`; and the
bare vcs_url notation `git+https://…`, `git+ssh://…`, `git://…`, `hg+https://…`,
bare vcs_url notation `git+https://…`, `git+ssh://[<user>@]…`, `git://…`, `hg+https://…`,
`hg+static-http://…`, `svn://…`, `svn+https://…`, `svn+ssh://…`, `<vcs>+file:///…`, each with an
optional `@<version>` and `#<subpath>`. A repository on a Windows machine is
`<vcs>+file:///C:/path` (RFC 8089); `<vcs>+file://C:/path` is accepted as input and normalised to
Expand All @@ -44,12 +46,64 @@ two locations and the one your server serves is the one to write).
Refused, each with a message naming the three accepted spellings: `pkg:svn/…` and `pkg:hg/…`
(not Package-URL types), `pkg:github/…`, `pkg:gitlab/…` and other provider vocabularies (the
host is part of the location), plain clone URLs as identities (`https://host/path.git@1.0.0`,
`ssh://…`, `git@host:path`, `file:///path@1.0.0`, `/path@1.0.0`), a user name or password in the
host (Annex B.3.5), a transport outside the lists above, `bzr` or any other VCS, a commit id or
`ssh://…`, `git@host:path`, `file:///path@1.0.0`, `/path@1.0.0`), a password in the host on any
transport and a user name in the host on any transport but `ssh` (Annex B.3.5; see below), a
transport outside the lists above, `bzr` or any other VCS, a commit id or
`@latest` as version. A plain `http(s)` IRI is only ever dereferenced as a project interchange
file — a `.kpar` — as KerML 1.1 §10.3 Table 12 describes `resource`; it is never cloned, and
`user:password@` in it is refused (credentials belong in the netrc, never in `.project.json`).

## The user name of an SSH location

`git+ssh://git@github.com/owner/repo@1.0.0`, `hg+ssh://hg@host/repo` and `svn+ssh://svn@host/repo`
are accepted: on the `ssh` transport the part before `@` is the login the server expects, the
same for everyone, and not a secret. Annex B.3.5 (informative) says "Using user names and password
in the host_name is not supported and should be reported by tools as an error", and in the same
clause that its syntax is pip's, whose own examples are `git+ssh://git@git.example.com/MyProject`
and `svn+ssh://user@svn.example.com/MyProject`. SysPI follows that recommendation wherever the
user name would be a credential and departs from it for this one case:

| written | result |
| --- | --- |
| `<vcs>+ssh://user@host/path` | accepted |
| `<vcs>+ssh://user:password@host/path`, `user:@host` | refused: a password |
| `<vcs>+<any other transport>://user@host/path` | refused: there a user name is a credential (a token) |
| `pkg:git/user@host/path` | refused: it means `git+https://user@host/path` |
| `git@host:path` (scp style) | refused: not a URL; the message names `git+ssh://git@host/path` |

The alias and the bare notation agree: both are the same `vcs_url` once normalised, and the one
rule above is applied to that `vcs_url`. An SSH location is therefore always written as
`git+ssh://…` or as the canonical `pkg:generic/…?vcs_url=git%2Bssh:…`; `pkg:git/…` has no SSH form.

**The user name is part of the identity.** RFC 3986 §3.2 makes userinfo a component of the
authority, and neither its normalisation rules (§6) nor Annex B.3.5 drop it, so SysPI keeps it:

- `.project.json` and the lock's `resource` carry it inside the percent-encoded `vcs_url`, the
`@` as `%40` like every other one:
`pkg:generic/repo?vcs_url=git%2Bssh:%2F%2Fgit%40github.com%2Fowner%2Frepo`. The lock's `source`
is `ssh://git@github.com/owner/repo`, and that is the address the VCS executable is given
(`git ls-remote --tags -- ssh://git@github.com/owner/repo`).
- The user name is kept as written: its case is significant, and a character outside RFC 3986's
userinfo set stays percent-encoded in the address (`first.last%40corp@host`, hence `%2540` in
the `resource`). An empty user name (`git+ssh://@host/path`) is none. One starting with `-` is
refused, since `ssh` would read it as an option.
- `git+ssh://git@host/owner/repo` and `git+ssh://host/owner/repo` are **two identities**, exactly
as `git+https://host/owner/repo` and `git+ssh://host/owner/repo` are: SysPI cannot know that
the login `ssh_config` supplies for the second is the `git` of the first. A usage of one never
matches a lock entry of the other; each is resolved through its own address and locked under
its own `resource`. Changing a usage from one spelling to the other is changing the identity:
the next `install` resolves it anew over the network (`-offline` refuses, as for any identity
the lock does not hold) and replaces the lock entry.
- The installed directory does not carry it: `lib/<publisher>-<name>_<version>` is named from the
package's own `.project.json`, never from the location. Two usages that differ only in the user
name — in one project, or one in the project and one in a dependency — therefore resolve to the
same directory, are locked as two entries with the same `revision` and `sha256`, and are
installed once. Were the two addresses to serve different contents for that publisher, name and
version, the run fails naming both (as for any two packages that collide on a directory)
instead of letting one overwrite the other. `lib/` is matched to a usage only through the
lock: an entry whose `resource` is the usage's identity, and an installed tree whose digest is
that entry's `sha256`.

## What a package is

The tagged tree — the repository root, or `#<subpath>` within it — is the project directory. It
Expand Down
7 changes: 4 additions & 3 deletions internal/install/install.go
Original file line number Diff line number Diff line change
Expand Up @@ -417,7 +417,7 @@ func (r *resolver) lockedRepository(o *origin.Origin, c version.Constraint) *loc
}

// locationKey normalises a repository source for comparison: a <vcs>+ scheme
// prefix and host case do not matter, a file URL is its path; .git does.
// prefix and host case do not matter, a file URL is its path; .git and a user name do.
func locationKey(src string) string {
scheme, rest, ok := strings.Cut(src, "://")
if !ok {
Expand All @@ -430,8 +430,9 @@ func locationKey(src string) string {
if scheme == "file" {
return filepath.FromSlash(rest)
}
host, p, _ := strings.Cut(rest, "/")
return scheme + "://" + strings.ToLower(host) + "/" + p
authority, p, _ := strings.Cut(rest, "/")
at := strings.LastIndexByte(authority, '@') + 1
return scheme + "://" + authority[:at] + strings.ToLower(authority[at:]) + "/" + p
}

// reuseLocked keeps a locked archive entry when only the lock is being
Expand Down
3 changes: 3 additions & 0 deletions internal/install/install_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,7 @@ func TestLocationKey(t *testing.T) {
same := [][2]string{
{"git+ssh://Example.org/team/repo.git", "ssh://example.org/team/repo.git"},
{"git+https://example.org/team/repo", "https://example.org/team/repo"},
{"git+ssh://git@Example.org/team/repo", "ssh://git@example.org/team/repo"},
{"git+file:///srv/repo.git", filepath.FromSlash("/srv/repo.git")},
{"svn+file:///srv/repo", "file:///srv/repo"},
}
Expand All @@ -351,6 +352,8 @@ func TestLocationKey(t *testing.T) {
{"https://example.org/team/repo", "https://example.org/team/repo.git"},
{"ssh://example.org/team/repo", "https://example.org/team/repo"},
{"https://example.org/Team/repo", "https://example.org/team/repo"},
{"ssh://git@example.org/team/repo", "ssh://example.org/team/repo"},
{"ssh://Git@example.org/team/repo", "ssh://git@example.org/team/repo"},
}
for _, c := range differ {
if locationKey(c[0]) == locationKey(c[1]) {
Expand Down
73 changes: 59 additions & 14 deletions internal/origin/origin.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
// A version-controlled package is identified by an ECMA-427 Package-URL of
// type generic carrying the Annex B.3.5 vcs_url qualifier:
//
// pkg:generic/<name>?vcs_url=<vcs>+<transport>://<host>/<path>[@<version>][#<subpath>]
// pkg:generic/<name>?vcs_url=<vcs>+<transport>://[<user>@]<host>/<path>[@<version>][#<subpath>]
//
// with the qualifier value percent-encoded. pkg:git/<host>/<path>[@v][#sub]
// with the qualifier value percent-encoded; <user>@ is the SSH login, on <vcs>+ssh only. pkg:git/<host>/<path>[@v][#sub]
// and the bare vcs_url notation are accepted as input and normalised to it.
package origin

Expand Down Expand Up @@ -73,13 +73,16 @@ var transports = map[string]map[string]bool{
// Spellings names the three accepted ways to write a repository package.
const Spellings = "pkg:generic/<name>?vcs_url=<vcs>%2B<transport>:%2F%2F<host>%2F<path>[%40<version>][%23<subpath>], " +
"pkg:git/<host>/<path>[@<version>][#<subpath>], or <vcs>+<transport>://<host>/<path>[@<version>][#<subpath>] " +
"(vcs: git, svn, hg; transports per ECMA-427 Annex B.3.5, plus file for a repository on this machine)"
"(vcs: git, svn, hg; transports per ECMA-427 Annex B.3.5, plus file for a repository on this machine; " +
"<user>@<host> names the SSH login on <vcs>+ssh only)"

// Origin is a parsed resource.
type Origin struct {
Kind Kind
// VCS and Transport are the vcs_url scheme parts for Kind == VCS.
VCS, Transport string
// User is the SSH login written before the host of a <vcs>+ssh location, "" when none.
User string
// Host is the repository host (empty for a repository on this machine).
Host string
// Path is the repository path on the host as written, without a leading
Expand All @@ -98,14 +101,22 @@ type Origin struct {
Resource string
}

// VCSURL is the Annex B.3.5 location: <vcs>+<transport>://<host>/<path>[@v][#sub].
// authority is [<user>@]<host>, the user name escaped as RFC 3986 userinfo.
func authority(user, host string) string {
if user == "" {
return host
}
return url.User(user).String() + "@" + host
}

// VCSURL is the Annex B.3.5 location: <vcs>+<transport>://[<user>@]<host>/<path>[@v][#sub].
func (o *Origin) VCSURL(withVersion bool) string {
var b strings.Builder
b.WriteString(o.VCS)
b.WriteByte('+')
b.WriteString(o.Transport)
b.WriteString("://")
b.WriteString(o.Host)
b.WriteString(authority(o.User, o.Host))
b.WriteByte('/')
b.WriteString(strings.TrimPrefix(o.Path, "/"))
if withVersion && o.Version != "" {
Expand Down Expand Up @@ -200,7 +211,7 @@ func Parse(resource string) (*Origin, error) {
return nil, fmt.Errorf("resource %q: unsupported URL scheme; a repository is written as %s", res, Spellings)
}
if m := scpLike.FindStringSubmatch(res); m != nil && !filepath.IsAbs(res) && !strings.HasPrefix(res, ".") {
return nil, fmt.Errorf("resource %q: scp-style clone addresses are not identities; write git+ssh://%s/%s or another of %s", res, m[2], strings.TrimSuffix(m[3], ".git"), Spellings)
return nil, fmt.Errorf("resource %q: scp-style clone addresses are not identities; write git+ssh://%s%s/%s or another of %s", res, m[1], m[2], strings.TrimSuffix(m[3], ".git"), Spellings)
}
return parseLocal(res, res)
}
Expand Down Expand Up @@ -281,12 +292,45 @@ func parseGitAlias(res, tail string) (*Origin, error) {
return parseVCSURL(res, "git+https://"+tail)
}

// parseVCSURL handles <vcs>[+<transport>]://<host>/<path>[@<version>][#<subpath>].
// userInAuthority reports whether the last @ of spec lies in its authority,
// where it ends a user name instead of starting a version.
func userInAuthority(spec string) bool {
scheme, rest, _ := strings.Cut(spec, "://")
if scheme == "file" || strings.HasSuffix(scheme, "+file") {
return false // no authority: on Windows the path may follow "://" directly
}
auth, tail, _ := strings.Cut(rest, "/")
return strings.Contains(auth, "@") && !strings.Contains(tail, "@")
}

// sshUser is the user name of a vcs_url authority: accepted on ssh, where it
// is the login and not a secret; a password, or a user name elsewhere, is refused.
func sshUser(res, vcs, transport string, info *url.Userinfo) (string, error) {
if info == nil {
return "", nil
}
if _, has := info.Password(); has {
return "", fmt.Errorf("resource %q: a password in the host is not supported (ECMA-427 Annex B.3.5); use the credentials of your %s configuration", res, vcs)
}
if transport != "ssh" {
return "", fmt.Errorf("resource %q: a user name in the host is accepted only on %s+ssh, where it is the SSH login; on %s+%s it would be a credential (ECMA-427 Annex B.3.5) — use the credentials of your %s configuration", res, vcs, vcs, transport, vcs)
}
user := info.Username()
if strings.HasPrefix(user, "-") {
return "", fmt.Errorf("resource %q: user name %q starts with \"-\" and would be read as an option by ssh", res, user)
}
return user, nil
}

// parseVCSURL handles <vcs>[+<transport>]://[<user>@]<host>/<path>[@<version>][#<subpath>].
func parseVCSURL(res, raw string) (*Origin, error) {
spec, sub := cutFragment(raw)
spec, ver, err := cutVersion(res, spec)
if err != nil {
return nil, err
ver := ""
if !userInAuthority(spec) {
var err error
if spec, ver, err = cutVersion(res, spec); err != nil {
return nil, err
}
}
scheme, rest, ok := strings.Cut(spec, "://")
if !ok {
Expand All @@ -312,13 +356,14 @@ func parseVCSURL(res, raw string) (*Origin, error) {
if err != nil {
return nil, fmt.Errorf("resource %q: not a valid URL: %v", res, err)
}
if u.User != nil {
return nil, fmt.Errorf("resource %q: a user name or password in the host is not supported (ECMA-427 Annex B.3.5); use the credentials of your %s configuration", res, vcs)
user, err := sshUser(res, vcs, transport, u.User)
if err != nil {
return nil, err
}
if u.RawQuery != "" {
return nil, fmt.Errorf("resource %q: a vcs_url must not carry a query", res)
}
o := &Origin{Kind: VCS, VCS: vcs, Transport: transport, Subpath: sub, Version: ver, Resource: res}
o := &Origin{Kind: VCS, VCS: vcs, Transport: transport, User: user, Subpath: sub, Version: ver, Resource: res}
if transport == "file" {
if u.Host != "" || !strings.HasPrefix(u.Path, "/") {
return nil, fmt.Errorf("resource %q: a repository on this machine is %s+file:///absolute/path", res, vcs)
Expand Down Expand Up @@ -348,7 +393,7 @@ func parseVCSURL(res, raw string) (*Origin, error) {
return nil, fmt.Errorf("resource %q: repository path must not contain \"..\"", res)
}
o.Host, o.Path, o.Name = strings.ToLower(u.Host), p, strings.TrimSuffix(path.Base(p), ".git")
o.Source = source(vcs, transport, o.Host, p)
o.Source = source(vcs, transport, authority(user, o.Host), p)
return o, nil
}

Expand Down
Loading
Loading