diff --git a/README.md b/README.md index 3378f69..0e8cb9e 100644 --- a/README.md +++ b/README.md @@ -331,7 +331,9 @@ the flow. An authorization server that requires PKCE refuses the code exchange without one, and there is no other way to ask for it from a declaration. `token_prefixes: []` says the service does not announce itself: a token is -taken as its own once no other connection's prefix matches it. Two such +taken as its own once no other connection's prefix matches it. A JWT is taken +only if its `iss` claim names the service's own host; one that names no +issuer is left to a connection declaring `issuer: optional` below. Two such connections cannot be told apart, so an agent may hold only one of them. Declaring a second one in `gete.yaml` is fine; naming both under one agent's `connections` is what `gete validate` refuses. @@ -366,25 +368,51 @@ so, and gete holds the connection to it — a token that is not such a JWT is refused, however little else claims it. In return the connection is no longer accepted by elimination, and an agent may hold it next to one that is: +```yaml +connections: + internal-billing: + display_name: Internal Billing + hosts: [billing.internal.example.com] + tokens: + format: jwt # tokens are JWTs whose iss names this service + oauth: + authorization_url: https://auth.billing.example.com/authorize + token_url: https://auth.billing.example.com/token + scopes: {} +``` + +An agent may then write to `internal-billing` and read `internal-api` above, +which keeps the one place a token is taken by elimination. What the two may +not share is an issuer: one authorization server in front of both services +puts its host in either connection's tokens, and `gete validate` refuses that +pairing for the same reason it refuses two anonymous connections. + +Some providers issue JWTs that name no issuer at all — Zendesk's carry only +`exp`. Such a token says nothing about whose it is, so neither elimination nor +the format alone takes it. `issuer: optional` says the connection's tokens may +name none: + ```yaml connections: zendesk: base_url: https://acme.zendesk.com tokens: - format: jwt # tokens are JWTs issued by acme.zendesk.com, nothing else + format: jwt + issuer: optional # Zendesk's JWTs name no issuer ``` -An agent may then read Zendesk and write to `internal-api` above, which keeps -the one place a token is taken by elimination. What the two may not share is -an issuer: one authorization server in front of both services puts its host -in either connection's tokens, and `gete validate` refuses that pairing for -the same reason it refuses two anonymous connections. +A JWT that does name an issuer is still held to the connection's own host. +An agent may hold Zendesk beside `internal-api` and beside `internal-billing`, +since neither takes a JWT naming no issuer, but not beside a second connection +declaring `issuer: optional`: nothing would tell their tokens apart. Whether a provider issues such tokens can be a setting rather than a promise — Zendesk does so only with token expiry turned on — so the declaration belongs in `gete.yaml`, not in a catalog entry that would promise it for every -installation. It cuts both ways: turn that setting off and every token the -connection is handed is refused until the declaration goes with it. +installation. Without it, every such token the connection is handed is +refused, and the log names the declaration to add. It cuts both ways: turn +that setting off and a token that is not a JWT is refused until the +declaration goes with it. A service whose root moves with the installation — the tenant in a subdomain, or a deployment you host — writes its URLs around `{base_url}`, and the diff --git a/src/gete/catalog/connections/zendesk.yaml b/src/gete/catalog/connections/zendesk.yaml index 9e14064..e8898b1 100644 --- a/src/gete/catalog/connections/zendesk.yaml +++ b/src/gete/catalog/connections/zendesk.yaml @@ -7,14 +7,13 @@ docs: https://developer.zendesk.com/api-reference/ # until the installation sets connections.zendesk.base_url in its gete.yaml; # hosts stays empty because the only host comes from that root. # Zendesk announces no prefix, and what its access tokens look like is the -# installation's own setting rather than the service's promise: with token -# expiry turned on they are JWTs issued by the subdomain, and without it they -# are JWTs whose claims name no issuer, which says as little about their -# origin as an opaque token. The catalog cannot promise either, so tokens are -# accepted by elimination here - and an agent may hold only one connection of -# that kind. An installation that has expiry on says so with -# connections.zendesk.tokens.format: jwt in its gete.yaml, and then this -# connection no longer takes that place. +# installation's own setting rather than the service's promise. With token +# expiry turned on they arrive as JWTs whose claims carry only exp and name no +# issuer. With it off, their shape has not been confirmed. Neither elimination +# nor the jwt format alone takes a JWT that names no issuer, so an +# installation that has expiry on declares +# connections.zendesk.tokens: {format: jwt, issuer: optional} in its +# gete.yaml; the catalog cannot promise that for every installation. token_prefixes: [] oauth: @@ -46,18 +45,30 @@ setup: | fails afterwards. Zendesk can be set to issue access tokens that expire. Those tokens are - JWTs issued by the subdomain, and an installation that has the setting on - may declare connections.zendesk.tokens.format: jwt in its gete.yaml, which - frees an agent to hold Zendesk beside another connection whose tokens - announce themselves no way at all. The declaration is held to: turn the - setting off again and every Zendesk token is refused until it is removed. + JWTs whose claims carry only exp: they name no issuer. Such a token is + taken only by a connection that says its tokens may name none, so an + installation that has the setting on declares, in its gete.yaml: -# No accepted example: which of the two shapes arrives is the installation's -# setting, and an entry fixing one of them would contradict an installation -# that has the other - the declared one most of all, whose accepted tokens -# name a subdomain the catalog cannot know. What is refused is refused either -# way. verified is absent because no authorization has been taken through -# Gemini Enterprise yet. + connections: + zendesk: + tokens: + format: jwt + issuer: optional + + Until it does, every token is refused and the log says what to declare. + The declaration also frees an agent to hold Zendesk beside a connection + whose tokens announce themselves no way at all, or beside one whose JWTs + name their own issuer. An agent may hold only one connection that + declares issuer: optional. The declaration is held to: turn the setting + off again and a token that is not a JWT is refused until the declaration + is removed. What arrives with the setting off has not been confirmed. + +# No accepted example: without the declaration only a token that is not a JWT +# could be accepted, and what Zendesk issues with expiry off has not been +# confirmed, so an example would be a claim this cannot keep. Nor is the +# expiring token a rejected one: the examples are checked against the entry an +# installation declares the format on too, and that entry takes it. verified is +# absent because no authorization has been taken through Gemini Enterprise yet. examples: rejects: - "ya29.a0AfH6SMB" # Google access token diff --git a/src/gete/connection/checks.py b/src/gete/connection/checks.py index 93b0f99..18e11e7 100644 --- a/src/gete/connection/checks.py +++ b/src/gete/connection/checks.py @@ -5,6 +5,7 @@ from gete.connection.registry import ( GOOGLE_ACCESS_TOKEN_PREFIX, + JWT_FORMAT, Connection, OAuth, Registry, @@ -41,7 +42,11 @@ "eyJhbGciOiJSUzI1NiJ9." "eyJpc3MiOiJhZ2VudEBwcm9qZWN0LmlhbS5nc2VydmljZWFjY291bnQuY29tIn0.sig", ) -_GOOGLE_ACCESS_TOKEN_EXAMPLE = GOOGLE_ACCESS_TOKEN_PREFIX + "a0AfH6SMB" +# Both shapes Google issues: the second carries two dots, as a JWT does. +_GOOGLE_ACCESS_TOKEN_EXAMPLES = ( + GOOGLE_ACCESS_TOKEN_PREFIX + "a0AfH6SMB", + GOOGLE_ACCESS_TOKEN_PREFIX + "c.b0Aaekm1K", +) def elimination_problems( @@ -65,6 +70,11 @@ def elimination_problems( naming its own authorization server as readily as a declaring one does, so a shared issuer confuses the two whichever of them declared. + A JWT that names no issuer, as Zendesk's do, is taken only by a + connection declaring tokens.issuer: optional. Nothing tells two of those + apart, so an agent may hold one, the way it may hold one anonymous + connection; neither of the others takes such a token. + Prefixes that overlap are that confusion among the connections that do announce themselves: a token carrying the shared prefix passes as either's. A service that is run in more than one place - hosted by its @@ -87,7 +97,19 @@ def elimination_problems( f"{', '.join(connection.id for connection in anonymous)} declare " "neither token_prefixes nor tokens.format; only one of an agent's " "connections may accept tokens by elimination, or a token from one " - "of them would be accepted as another's" + "of them would be accepted as another's. Declare tokens.format: jwt " + "on one whose tokens are JWTs, with issuer: optional if they never " + "name an issuer" + ) + issuerless = [ + connection for connection in connections if _takes_issuerless(connection) + ] + if len(issuerless) >= 2: + problems.append( + f"{', '.join(connection.id for connection in issuerless)} declare " + "tokens.issuer: optional; only one of an agent's connections may " + "accept JWTs that name no issuer, or a token from one of them would " + "be accepted as another's" ) # A prefix decides on its own and before any issuer does, so a connection # held to one judges no token by who issued it, and shares an issuer with @@ -128,6 +150,11 @@ def elimination_problems( return problems +def _takes_issuerless(connection: Connection) -> bool: + """Whether the connection accepts a JWT that names no issuer.""" + return connection.token_format == JWT_FORMAT and connection.issuer_optional + + def _overlapping_prefixes(connection: Connection, other: Connection) -> list[str]: """The prefixes a token can carry and be accepted by both connections. @@ -202,7 +229,9 @@ def connection_problems(connection: Connection, registry: Registry) -> list[str] prefix.startswith(GOOGLE_ACCESS_TOKEN_PREFIX) for prefix in connection.token_prefixes ) - if not claims_google and connection.accepts_token(_GOOGLE_ACCESS_TOKEN_EXAMPLE): + if not claims_google and any( + connection.accepts_token(example) for example in _GOOGLE_ACCESS_TOKEN_EXAMPLES + ): problems.append("a Google access token is accepted") # The MCP server is spoken to with the user's token, so its URL must sit # where hosts lets the token go - path scoping included. diff --git a/src/gete/connection/registry.py b/src/gete/connection/registry.py index 5489e0d..7202d9e 100644 --- a/src/gete/connection/registry.py +++ b/src/gete/connection/registry.py @@ -34,6 +34,11 @@ # names itself in every token, which is what a prefix would otherwise do. JWT_FORMAT = "jwt" +# What tokens.issuer says of a JWT format whose tokens name no issuer at all, +# as Zendesk's do. Anything else - the default, or a value this gete cannot +# read - holds every JWT to naming the connection's own service. +ISSUER_OPTIONAL = "optional" + # Stands for the root of a service that differs per installation, wherever the # connection's URLs are written around it. A definition meant to be shared # cannot spell a tenant's host, and a stand-in host would be a name a stranger @@ -220,6 +225,10 @@ class Connection: # as surely as a prefix does and is not accepted by elimination; one this # gete cannot judge takes nothing at all. token_format: str | None = None + # Whether a JWT that names no issuer keeps the format's promise. Such a + # token says nothing about whose it is, so only the connection that + # declares it takes one, and validate lets an agent hold one such. + issuer_optional: bool = False # Prefixes declared by every other connection in the registry, filled in by # Registry. A connection without prefixes of its own accepts a token only # if none of these match, so a bare from_mapping() connection judges more @@ -267,6 +276,7 @@ def from_mapping(cls, data: Mapping[str, Any]) -> "Connection": redirect_hosts=frozenset(data.get("redirect_hosts", ())), token_prefixes=tuple(data.get("token_prefixes", ())), token_format=data.get("tokens", {}).get("format"), + issuer_optional=data.get("tokens", {}).get("issuer") == ISSUER_OPTIONAL, base_url=base_url, docs=data.get("docs"), oauth_client=data.get("oauth_client"), @@ -328,9 +338,11 @@ def accepts_token(self, token: str) -> bool: issues ya29.c. access tokens that carry two dots, and a shape heuristic must not overrule a prefix the catalog vouches for. With neither, a JWT is judged by the issuer its claims name - Google ID - tokens and service account tokens always name Google's - and a JWT - that names none says as little as an opaque token, so it is judged - like one. One that cannot be read is refused rather than guessed about. + tokens and service account tokens always name Google's - and one that + names none is refused: it says nothing about whose it is, and only a + connection declaring tokens.issuer: optional takes it, so the two + never share a token. One that cannot be read is refused rather than + guessed about. """ if not token: return False @@ -338,37 +350,36 @@ def accepts_token(self, token: str) -> bool: # A resolved declaration outlives the gete that wrote it, and a # format this one cannot judge is not one it may fall back from: # elimination would accept what the declaration meant to narrow. - return self.token_format == JWT_FORMAT and self._issued_here_jwt(token) + return self.token_format == JWT_FORMAT and self._issued_here_jwt( + token, issuerless=self.issuer_optional + ) if self.token_prefixes: return token.startswith(self.token_prefixes) if token.startswith(GOOGLE_ACCESS_TOKEN_PREFIX): return False if looks_like_jwt(token): - claims = jwt_claims(token) - if claims is None: - return False - if "iss" in claims: - issuer = claims["iss"] - return isinstance(issuer, str) and self.issued_here(issuer) + return self._issued_here_jwt(token, issuerless=False) # The token does not announce itself. All that can be said is that # it is not some other service's. return not any(token.startswith(prefix) for prefix in self.foreign_prefixes) - def _issued_here_jwt(self, token: str) -> bool: + def _issued_here_jwt(self, token: str, *, issuerless: bool) -> bool: """Whether the token is a JWT this connection's own service issued. - What a connection declaring the format holds every token to. A token - of any other shape, one whose claims cannot be read, and one that - names no issuer are all refused alike: none of them is what was - promised, and the promise is the only reason the connection may sit - beside another that accepts tokens by elimination. + Its claims have to be readable and name this service as the issuer, + or - where issuerless is allowed - name no issuer at all. Google's + ya29.c. access tokens carry two dots as well, and are + refused for the prefix they announce rather than for a middle + segment that happens not to decode. """ - if not looks_like_jwt(token): + if token.startswith(GOOGLE_ACCESS_TOKEN_PREFIX) or not looks_like_jwt(token): return False claims = jwt_claims(token) if claims is None: return False - issuer = claims.get("iss") + if "iss" not in claims: + return issuerless + issuer = claims["iss"] return isinstance(issuer, str) and self.issued_here(issuer) @property diff --git a/src/gete/connection/runtime.py b/src/gete/connection/runtime.py index f89722a..70634c9 100644 --- a/src/gete/connection/runtime.py +++ b/src/gete/connection/runtime.py @@ -12,6 +12,7 @@ from typing import Any from gete.connection.registry import ( + GOOGLE_ACCESS_TOKEN_PREFIX, JWT_FORMAT, Connection, Registry, @@ -101,26 +102,38 @@ def describe_token(connection: Connection, token: str) -> str: """Which declared shape the token has, without repeating any of it. A connection without prefixes declares no shapes, so a refused JWT is - described by what refused it - claims that cannot be read, or an issuer - that does not name this service - or the shape message would leave the - refusal looking like a mystery. A connection that declares a token format - is refusing against a promise, and the promise is named: an operator who - turned the provider's expiring-token setting off has every token refused - at once, and nothing else in the log would say why. Read in the order + described by what refused it - claims that cannot be read, an issuer + that does not name this service, or no issuer at all - or the shape + message would leave the refusal looking like a mystery. A JWT naming no + issuer is taken only where tokens.issuer: optional is declared, and the + message names that declaration: a provider whose JWTs never name one has + every token refused at once, and nothing else in the log would say why. + A Google access token carries two dots as a JWT does, and is described + by the shapes it misses rather than as a JWT. Read in the order accepts_token decides in, so the message names the rule that actually refused the token. """ if connection.token_format is not None: - return _unmet_format(connection.token_format, token) + return _unmet_format(connection, token) for prefix in connection.token_prefixes: if token.startswith(prefix): return f"starts with {prefix}" - if connection.token_prefixes: + if ( + connection.token_prefixes + or token.startswith(GOOGLE_ACCESS_TOKEN_PREFIX) + or not looks_like_jwt(token) + ): return NO_DECLARED_SHAPE - return _refused_jwt(token) or NO_DECLARED_SHAPE + return _refused_jwt( + token, + issuerless=( + f"declare connections.{connection.id}.tokens.format: jwt with " + "issuer: optional if its tokens never name one" + ), + ) -def _unmet_format(token_format: str, token: str) -> str: +def _unmet_format(connection: Connection, token: str) -> str: """Which part of the declared format the token missed. A format this gete cannot judge refuses every token alike, so it is the @@ -128,24 +141,29 @@ def _unmet_format(token_format: str, token: str) -> str: issuer was wrong would go looking at a token that may be exactly right, when what refused it is a declaration newer than the gete reading it. """ + token_format = connection.token_format if token_format != JWT_FORMAT: return f"declared as {token_format}, which this gete cannot judge" - if not looks_like_jwt(token): + if token.startswith(GOOGLE_ACCESS_TOKEN_PREFIX) or not looks_like_jwt(token): return f"not the {token_format} this connection declares" - return _refused_jwt(token) or "a JWT that names no issuer" + return _refused_jwt( + token, + issuerless=( + f"declare connections.{connection.id}.tokens.issuer: optional if " + "its tokens never name one" + ), + ) -def _refused_jwt(token: str) -> str | None: - """Why a JWT-shaped token is not taken for this service's, or None when - nothing about the JWT itself says so.""" - if not looks_like_jwt(token): - return None +def _refused_jwt(token: str, *, issuerless: str) -> str: + """Why a JWT-shaped token was refused; issuerless is the declaration + that would take one naming no issuer.""" claims = jwt_claims(token) if claims is None: return "a JWT whose claims cannot be read" - if "iss" in claims: - return "a JWT whose issuer does not name this service" - return None + if "iss" not in claims: + return f"a JWT that names no issuer; {issuerless}" + return "a JWT whose issuer does not name this service" def usable_token(connection: Connection, key: str, state: Any) -> str | None: diff --git a/src/gete/connections_listing.py b/src/gete/connections_listing.py index 67aaa24..6a55488 100644 --- a/src/gete/connections_listing.py +++ b/src/gete/connections_listing.py @@ -20,8 +20,10 @@ def token_shapes(connection: Connection) -> str: """ if connection.token_prefixes: return ", ".join(connection.token_prefixes) + if connection.token_format is not None and connection.issuer_optional: + return f"(none: {connection.token_format} naming this service or no issuer)" if connection.token_format is not None: - return f"(none: {connection.token_format} issued by this service)" + return f"(none: {connection.token_format} naming this service)" return BY_ELIMINATION diff --git a/src/gete/schema/connection.json b/src/gete/schema/connection.json index 463846b..018b016 100644 --- a/src/gete/schema/connection.json +++ b/src/gete/schema/connection.json @@ -61,7 +61,7 @@ } }, "token_prefixes": { - "description": "Prefixes of issued tokens. Empty means the service does not announce itself and tokens are accepted by elimination, unless tokens.format says what they look like; an agent may declare only one connection that announces itself neither way. Connections may share a prefix, as two installations of one service do; an agent may not declare two whose prefixes overlap.", + "description": "Prefixes of issued tokens. Empty means the service does not announce itself and tokens are accepted by elimination, unless tokens.format says what they look like; a connection accepting by elimination takes no JWT that names no issuer, and an agent may declare only one connection that announces itself neither way. Connections may share a prefix, as two installations of one service do; an agent may not declare two whose prefixes overlap.", "type": "array", "uniqueItems": true, "items": { @@ -82,6 +82,13 @@ "enum": [ "jwt" ] + }, + "issuer": { + "description": "required (the default): every JWT names the connection's own service as its issuer. optional: a JWT that names no issuer at all is accepted as well, as Zendesk's are; one that names an issuer is still held to it. Such a token says nothing about whose it is, so an agent may hold only one connection that declares optional.", + "enum": [ + "required", + "optional" + ] } } }, diff --git a/tests/conformance/test_catalog.py b/tests/conformance/test_catalog.py index 30e19c7..d0f65fc 100644 --- a/tests/conformance/test_catalog.py +++ b/tests/conformance/test_catalog.py @@ -1,6 +1,7 @@ """Every bundled connection must be sound on its own and against the others.""" from importlib.resources import files +from typing import Any import pytest @@ -346,31 +347,74 @@ def test_zendesk_says_what_a_person_has_to_do_before_authorizing() -> None: assert "redirect URI" in setup +# Claims: {"exp": 0}. What Zendesk issues with token expiry turned on: a JWT +# whose claims name no issuer. +ZENDESK_EXPIRING_TOKEN = "eyJhbGciOiJIUzI1NiJ9.eyJleHAiOjB9.sig" + + def test_zendesk_promises_no_token_format_for_every_installation() -> None: - """Zendesk issues JWTs of its own only with token expiry turned on, which - is the installation's setting; the catalog must not promise it.""" + """What arrives depends on whether the installation turned token expiry + on, so the catalog must not promise a shape; setup says what to declare.""" assert "tokens" not in CATALOG["zendesk"] - assert "tokens.format" in CATALOG["zendesk"]["setup"] + setup = " ".join(CATALOG["zendesk"]["setup"].split()) + assert "format: jwt" in setup + assert "issuer: optional" in setup + + +def test_zendesk_setup_does_not_say_its_jwts_name_the_subdomain() -> None: + """They carry only exp; a setup promising an issuer sends an operator + after the wrong cause when every token is refused.""" + setup = " ".join(CATALOG["zendesk"]["setup"].split()) + assert "issued by the subdomain" not in setup + assert "only exp" in setup -def test_zendesk_records_no_accepted_shape_because_it_has_two() -> None: - """An accepted example would fix one of the shapes, and contradict the - installation that has the other. What is refused is refused either way.""" +def test_zendesk_is_not_verified_until_an_authorization_goes_through() -> None: + assert "verified" not in CATALOG["zendesk"] + + +def test_zendesk_records_no_accepted_shape() -> None: + """Without a declaration only an opaque token could be accepted, and no + one has seen Zendesk issue one; an example would be a claim this cannot + keep. What is refused is refused either way.""" assert "accepts" not in CATALOG["zendesk"]["examples"] assert CATALOG["zendesk"]["examples"]["rejects"] +@pytest.mark.parametrize( + "tokens", + [ + None, # accepted by elimination, which takes no JWT naming no issuer + {"format": "jwt"}, # the format alone requires an issuer + ], +) +def test_zendesk_refuses_an_expiring_token_until_its_issuer_is_optional( + tokens: dict[str, Any] | None, +) -> None: + """The catalog entry as it stands turns the token away, and so does the + format without the optional issuer; the log says what to declare.""" + override: dict[str, Any] = {"base_url": "https://acme.zendesk.com"} + if tokens is not None: + override["tokens"] = tokens + zendesk = Registry.from_catalog({"zendesk": override}).get("zendesk") + assert not zendesk.accepts_token(ZENDESK_EXPIRING_TOKEN) + + def test_an_installation_may_declare_that_zendesk_issues_jwts() -> None: - """With expiry on the tokens name the subdomain, and Zendesk stops being - the one connection an agent may hold by elimination.""" + """With the format and the optional issuer declared Zendesk takes its + expiring tokens, and sits beside a connection whose tokens announce + themselves no way at all.""" registry = Registry.from_catalog( { "zendesk": { "base_url": "https://acme.zendesk.com", - "tokens": {"format": "jwt"}, + "tokens": {"format": "jwt", "issuer": "optional"}, } } ) zendesk = registry.get("zendesk") + assert zendesk.accepts_token(ZENDESK_EXPIRING_TOKEN) + assert not registry.get("notion-mcp").accepts_token(ZENDESK_EXPIRING_TOKEN) assert connection_problems(zendesk, registry) == [] assert elimination_problems(["zendesk", "freee"], registry) == [] + assert elimination_problems(["zendesk", "notion-mcp"], registry) == [] diff --git a/tests/test_caller_token.py b/tests/test_caller_token.py index b9073d0..f04a098 100644 --- a/tests/test_caller_token.py +++ b/tests/test_caller_token.py @@ -25,10 +25,13 @@ GOOGLE_ISSUED_JWT = ( "eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJodHRwczovL2FjY291bnRzLmdvb2dsZS5jb20ifQ.sig" ) +# Google issues ya29.c. access tokens as well; the two dots make one +# look like a JWT. +GOOGLE_ACCESS_TOKEN_WITH_DOTS = "ya29.c.b0Aaekm1K" -# Claims: {"exp": 0}. A JWT that names no issuer says as little about its -# origin as an opaque token. +# Claims: {"exp": 0}, as Zendesk's JWTs carry. Taken only where +# tokens.issuer: optional is declared. JWT_WITHOUT_ISSUER = "eyJhbGciOiJFZERTQSJ9.eyJleHAiOjB9.sig" # A connection whose tokens are JWTs of its own making, as an installation # declares one whose provider issues them. @@ -52,6 +55,21 @@ # Claims: {"iss": "https://api.example.com"}: a token DECLARED's own service # issued, and the only kind it takes. OWN_ISSUER_JWT = "eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJodHRwczovL2FwaS5leGFtcGxlLmNvbSJ9.sig" +# DECLARED as an installation declares it whose provider's JWTs name no +# issuer at all. +ISSUERLESS = Connection.from_mapping( + { + "id": "issuerless", + "display_name": "Issuerless", + "hosts": ["api.example.com"], + "tokens": {"format": "jwt", "issuer": "optional"}, + "oauth": { + "authorization_url": "https://api.example.com/authorize", + "token_url": "https://api.example.com/token", + "scopes": {}, + }, + } +) # A format no gete of this version knows, as a resolved declaration written # by a later one carries it. UNJUDGEABLE = Connection.from_mapping( @@ -152,10 +170,6 @@ def test_describe_token_says_why_a_jwt_was_refused_without_repeating_it() -> Non describe_token(FREEE, GOOGLE_ISSUED_JWT) == "a JWT whose issuer does not name this service" ) - assert ( - describe_token(FREEE, "eyJhbGciOiJSUzI1NiJ9.x.sig") - == "a JWT whose claims cannot be read" - ) # A declared prefix is the whole judgment; the JWT is described no further. assert ( describe_token(GITHUB, GOOGLE_ISSUED_JWT) @@ -163,18 +177,54 @@ def test_describe_token_says_why_a_jwt_was_refused_without_repeating_it() -> Non ) +def test_describe_token_names_the_declaration_a_jwt_naming_no_issuer_needs() -> None: + """Only a connection declaring tokens.issuer: optional takes such a token, + so a provider whose JWTs name no issuer has every token refused until it + is declared, and the log has to say what to declare.""" + assert describe_token(FREEE, JWT_WITHOUT_ISSUER) == ( + "a JWT that names no issuer; declare connections.freee.tokens.format: " + "jwt with issuer: optional if its tokens never name one" + ) + assert describe_token(DECLARED, JWT_WITHOUT_ISSUER) == ( + "a JWT that names no issuer; declare connections.declared.tokens.issuer: " + "optional if its tokens never name one" + ) + assert ISSUERLESS.accepts_token(JWT_WITHOUT_ISSUER) + + +def test_describe_token_does_not_name_the_declaration_where_it_would_not_help() -> None: + """A Google access token with two dots looks like a JWT, and so does a + token whose claims cannot be read; a declaring connection refuses both + as well, so telling the operator to declare the format would send them + after a change that refuses the same token.""" + assert ( + describe_token(FREEE, GOOGLE_ACCESS_TOKEN_WITH_DOTS) + == "matches none of the declared shapes" + ) + assert ( + describe_token(FREEE, "eyJhbGciOiJSUzI1NiJ9.x.sig") + == "a JWT whose claims cannot be read" + ) + + def test_describe_token_says_a_declared_format_was_not_met() -> None: - """The connection promised JWTs of its own making. An operator who turned - the provider's expiring-token setting off sees every token refused, and - the message has to say which promise the token missed.""" + """The connection promised JWTs its own service issued. The message says + which part of the promise the token missed.""" assert ( describe_token(DECLARED, FREEE_TOKEN) == "not the jwt this connection declares" ) - assert describe_token(DECLARED, JWT_WITHOUT_ISSUER) == "a JWT that names no issuer" + assert ( + describe_token(DECLARED, GOOGLE_ACCESS_TOKEN_WITH_DOTS) + == "not the jwt this connection declares" + ) assert ( describe_token(DECLARED, GOOGLE_ISSUED_JWT) == "a JWT whose issuer does not name this service" ) + assert ( + describe_token(DECLARED, "eyJhbGciOiJSUzI1NiJ9.x.sig") + == "a JWT whose claims cannot be read" + ) assert FREEE_TOKEN not in describe_token(DECLARED, FREEE_TOKEN) diff --git a/tests/test_connection_registry.py b/tests/test_connection_registry.py index 8c7f1bc..392f618 100644 --- a/tests/test_connection_registry.py +++ b/tests/test_connection_registry.py @@ -99,11 +99,13 @@ def test_google_issued_jwts_are_never_accepted(catalog: Registry) -> None: assert not entry.accepts_token(jwt_with({"iss": issuer})), entry.id -def test_a_jwt_naming_no_issuer_is_accepted_by_elimination() -> None: - """Zendesk issues JWTs whose claims name no issuer. They say as little - about their origin as an opaque token, so they are judged like one.""" +def test_a_jwt_naming_no_issuer_is_not_accepted_by_elimination() -> None: + """It says nothing about whose it is. A connection whose tokens are such + JWTs says so with tokens.issuer: optional, and elimination leaves them to + it, so the two can be held by one agent without sharing a token.""" alone = Registry([connection()]).get("example") - assert alone.accepts_token(jwt_with({"exp": 0})) + assert not alone.accepts_token(jwt_with({"exp": 0})) + assert alone.accepts_token("a1b2c3d4e5f60718293a4b5c6d7e8f90") def test_a_jwt_naming_the_connections_own_service_is_accepted() -> None: @@ -151,13 +153,58 @@ def test_prefixless_connection_alone_still_refuses_google_access_tokens() -> Non def test_a_declared_jwt_format_accepts_a_token_the_service_issued() -> None: - """The declaration says the tokens are JWTs of this service's own making, - and the issuer is what says so.""" + """An issuer the claims name has to be the connection's own service.""" entry = Registry([connection(**JWT_TOKENS)]).get("example") assert entry.accepts_token(jwt_with({"iss": "https://auth.example.com"})) assert entry.accepts_token(jwt_with({"iss": "api.example.com"})) +ISSUERLESS_JWT_TOKENS: dict[str, Any] = { + "tokens": {"format": "jwt", "issuer": "optional"} +} + + +def test_a_declared_jwt_format_refuses_a_jwt_that_names_no_issuer() -> None: + """The format promises tokens of this service's own making, and an + issuer is what says so; claims naming nobody keep no part of it.""" + entry = Registry([connection(**JWT_TOKENS)]).get("example") + assert not entry.accepts_token(jwt_with({"exp": 0})) + + +def test_an_optional_issuer_accepts_a_jwt_that_names_none() -> None: + """Zendesk's JWTs carry only exp. An installation that knows its + provider's tokens never name an issuer says so, and only then are they + taken; one that does name an issuer is still held to it.""" + entry = Registry([connection(**ISSUERLESS_JWT_TOKENS)]).get("example") + assert entry.accepts_token(jwt_with({"exp": 0})) + assert entry.accepts_token(jwt_with({"iss": "https://auth.example.com"})) + assert not entry.accepts_token(jwt_with({"iss": "https://idp.example.org"})) + assert not entry.accepts_token(jwt_with({"iss": "https://accounts.google.com"})) + assert not entry.accepts_token("a1b2c3d4e5f60718293a4b5c6d7e8f90") + assert not entry.accepts_token("eyJhbGciOiJSUzI1NiJ9.x.sig") + + +def test_an_issuer_requirement_this_gete_cannot_read_is_held_as_required() -> None: + """A resolved declaration outlives the gete that wrote it. Reading an + unknown value as optional would take tokens the declaration may have + meant to refuse; as required, it takes no more than the format alone.""" + entry = Registry([connection(tokens={"format": "jwt", "issuer": "pinned"})]).get( + "example" + ) + assert not entry.accepts_token(jwt_with({"exp": 0})) + assert entry.accepts_token(jwt_with({"iss": "https://auth.example.com"})) + + +def test_a_declared_jwt_format_refuses_a_google_access_token_by_its_prefix() -> None: + """Google's ya29.c. access tokens carry two dots. One is refused + for the prefix it announces, not for a middle segment that happens not + to decode, so a payload that does decode changes nothing.""" + entry = Registry([connection(**ISSUERLESS_JWT_TOKENS)]).get("example") + assert not entry.accepts_token("ya29.c.b0Aaekm1K") + readable = "ya29." + jwt_with({"exp": 0}).split(".", 1)[1] + assert not entry.accepts_token(readable) + + def test_a_declared_jwt_format_refuses_a_token_that_is_not_a_jwt() -> None: """Without the declaration elimination would take it; with it, the connection promises a shape and holds itself to it.""" @@ -171,10 +218,13 @@ def test_a_declared_jwt_format_refuses_a_token_that_is_not_a_jwt() -> None: @pytest.mark.parametrize( "claims", [ - {"exp": 0}, # a JWT that names no issuer says nothing about its origin + {"exp": 0}, # names no issuer, and none is declared optional {"iss": "https://idp.example.org"}, {"iss": "https://accounts.google.com"}, - {"iss": None}, + {"iss": None}, # present but null names nothing a host could match + {"iss": 5}, + {"iss": "https://api.example.com.evil.example"}, + {"iss": "https://["}, ], ) def test_a_declared_jwt_format_refuses_a_jwt_from_anywhere_else(claims: Any) -> None: @@ -549,6 +599,18 @@ def test_two_prefixless_connections_are_reported_as_one_pairing() -> None: assert len(found) == 1 +def test_two_prefixless_connections_are_told_what_declaration_parts_them() -> None: + """A provider whose tokens are JWTs leaves elimination by declaring the + format, and one whose JWTs name no issuer needs the optional issuer + beside it; the report names both, or an operator adding the format alone + would see every token refused at runtime before learning of the second.""" + freee = connection(id="freee", token_prefixes=[]) + internal = connection(id="internal", token_prefixes=[]) + found = elimination_problems(["internal", "freee"], Registry([freee, internal])) + assert "tokens.format: jwt" in found[0], found + assert "issuer: optional" in found[0], found + + def test_a_prefixless_connection_is_no_problem_on_its_own() -> None: """The registry holds all of them; only what one agent holds can collide.""" freee = connection(id="freee", token_prefixes=[]) @@ -632,6 +694,57 @@ def test_two_declared_token_formats_issued_by_one_host_are_reported() -> None: assert len(found) == 1 +def test_an_optional_issuer_sits_beside_any_connection_that_requires_one() -> None: + """A JWT naming no issuer is taken by it alone: an anonymous connection + and one requiring an issuer both refuse it, and its own issuer host is + named by neither of theirs.""" + optional = connection(id="optional", **ISSUERLESS_JWT_TOKENS) + anonymous = connection( + id="anonymous", + hosts=["api.anonymous.example.com"], + oauth={ + "authorization_url": "https://auth.anonymous.example.com/authorize", + "token_url": "https://auth.anonymous.example.com/token", + "scopes": {"read": "Read data"}, + }, + ) + required = connection( + id="required", + hosts=["api.two.example.com"], + oauth={ + "authorization_url": "https://auth.two.example.com/authorize", + "token_url": "https://auth.two.example.com/token", + "scopes": {"read": "Read data"}, + }, + **JWT_TOKENS, + ) + registry = Registry([optional, anonymous, required]) + issuerless = jwt_with({"exp": 0}) + assert registry.get("optional").accepts_token(issuerless) + assert not registry.get("anonymous").accepts_token(issuerless) + assert not registry.get("required").accepts_token(issuerless) + assert elimination_problems(["optional", "anonymous", "required"], registry) == [] + + +def test_two_optional_issuers_are_reported_whatever_their_services() -> None: + """A JWT that names no issuer is taken by either, so a token from one + authorization passes as the other's even when their services differ.""" + one = connection(id="one", **ISSUERLESS_JWT_TOKENS) + two = connection( + id="two", + hosts=["api.two.example.com"], + oauth={ + "authorization_url": "https://auth.two.example.com/authorize", + "token_url": "https://auth.two.example.com/token", + "scopes": {"read": "Read data"}, + }, + **ISSUERLESS_JWT_TOKENS, + ) + found = elimination_problems(["one", "two"], Registry([one, two])) + assert len(found) == 1, found + assert "one, two" in found[0] and "tokens.issuer" in found[0], found + + def test_prefixes_next_to_a_declared_token_format_are_reported() -> None: """The format decides on its own, so the prefixes would never be read.""" entry = connection(token_prefixes=["ex_"], **JWT_TOKENS) @@ -639,10 +752,14 @@ def test_prefixes_next_to_a_declared_token_format_are_reported() -> None: assert any("tokens" in problem for problem in problems), problems -def test_a_connection_that_accepts_google_issued_jwts_is_reported() -> None: +@pytest.mark.parametrize("tokens", [{}, JWT_TOKENS]) +def test_a_connection_that_accepts_google_issued_jwts_is_reported( + tokens: dict[str, Any], +) -> None: """Naming Google's authorization server as one's own host would let ID - tokens through the issuer match; the checks must catch the declaration.""" - entry = connection(hosts=["accounts.google.com"]) + tokens through the issuer match, which elimination and a declared format + both make; the checks must catch the declaration either way.""" + entry = connection(hosts=["accounts.google.com"], **tokens) assert any( "Google-issued" in problem for problem in connection_problems(entry, Registry([entry])) diff --git a/tests/test_connections_command.py b/tests/test_connections_command.py index 182f21c..18e07d5 100644 --- a/tests/test_connections_command.py +++ b/tests/test_connections_command.py @@ -226,11 +226,18 @@ def test_a_declared_token_format_reads_as_the_format_it_declares() -> None: connection has to see which of the two it is.""" entry = prefixless(tokens={"format": "jwt"}) rows = connections_table(Registry([entry])) - assert "jwt" in rows[0]["token_prefixes"] - assert "elimination" not in rows[0]["token_prefixes"] + assert rows[0]["token_prefixes"] == "(none: jwt naming this service)" assert "jwt" in format_connection(entry) +def test_an_optional_issuer_reads_as_taking_jwts_that_name_none() -> None: + """An agent may hold only one such connection, so the person preparing + it has to see that it is one.""" + entry = prefixless(tokens={"format": "jwt", "issuer": "optional"}) + rows = connections_table(Registry([entry])) + assert rows[0]["token_prefixes"] == "(none: jwt naming this service or no issuer)" + + def test_an_app_connection_is_described_by_its_app_not_an_oauth_client() -> None: """Nobody registers an OAuth client for it; what a person prepares is the App, its key, and the limit every token is narrowed to.""" diff --git a/tests/test_resolve.py b/tests/test_resolve.py index 154182c..1bfce55 100644 --- a/tests/test_resolve.py +++ b/tests/test_resolve.py @@ -1,7 +1,5 @@ """The resolved declaration: one file that carries everything the runtime needs.""" -import base64 -import json from pathlib import Path import yaml @@ -11,15 +9,9 @@ from gete.connection import Registry from gete.declaration import load_project, load_resolved, resolve - -def jwt_issued_by(issuer: str) -> str: - """A JWT-shaped token naming this issuer. The signature is never checked.""" - payload = ( - base64.urlsafe_b64encode(json.dumps({"iss": issuer}).encode()) - .rstrip(b"=") - .decode() - ) - return f"eyJhbGciOiJSUzI1NiJ9.{payload}.sig" +# Claims: {"exp": 0}, as Zendesk's JWTs carry with token expiry turned on. +# The signature is never checked. +ISSUERLESS_JWT = "eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjB9.sig" def write_project_with_policy(project: ProjectBuilder) -> None: @@ -92,7 +84,7 @@ def test_a_declared_token_format_travels_to_the_runtime( "connections": { "zendesk": { "base_url": "https://acme.zendesk.com", - "tokens": {"format": "jwt"}, + "tokens": {"format": "jwt", "issuer": "optional"}, } }, } @@ -101,7 +93,8 @@ def test_a_declared_token_format_travels_to_the_runtime( loaded = load_project(project.root / "gete.yaml") connections = resolve(loaded, loaded.agents[0])["resolved"]["connections"] zendesk = Registry.from_documents(connections).get("zendesk") - assert zendesk.accepts_token(jwt_issued_by("https://acme.zendesk.com")) + # Without the optional issuer travelling with it, this would be refused. + assert zendesk.accepts_token(ISSUERLESS_JWT) assert not zendesk.accepts_token("a1b2c3d4e5f60718293a4b5c6d7e8f90") diff --git a/tests/test_schema.py b/tests/test_schema.py index 88eb54f..4d14f88 100644 --- a/tests/test_schema.py +++ b/tests/test_schema.py @@ -654,13 +654,27 @@ def test_connection_may_declare_that_its_tokens_are_jwts() -> None: ) +@pytest.mark.parametrize("issuer", ["required", "optional"]) +def test_connection_may_say_whether_its_jwts_name_an_issuer(issuer: str) -> None: + """Zendesk's JWTs name none; an installation says so, or holds every + token to naming its own service, which is the default.""" + validate_document( + "connection", + {**CONNECTION, "tokens": {"format": "jwt", "issuer": issuer}}, + source="c.yaml", + ) + + @pytest.mark.parametrize( "tokens", [ {"format": "opaque"}, # the only format that promises anything is jwt {"format": "JWT"}, {}, # a tokens block that declares nothing + # The issuer is the connection's hosts; a second list would be a + # second ceiling to keep in step with the first. {"format": "jwt", "issuer": "https://auth.example.com"}, + {"issuer": "optional"}, # whether a JWT names an issuer needs a JWT ], ) def test_connection_token_format_is_one_declared_shape(tokens: dict[str, Any]) -> None: diff --git a/tests/test_validate.py b/tests/test_validate.py index c31cbbc..5eb0d05 100644 --- a/tests/test_validate.py +++ b/tests/test_validate.py @@ -434,6 +434,75 @@ def test_a_declared_format_beside_a_connection_of_the_same_issuer_is_refused( assert any("auth.internal.example.com" in problem for problem in found), found +ZENDESK_ISSUERLESS: dict[str, Any] = { + "base_url": "https://acme.zendesk.com", + "tokens": {"format": "jwt", "issuer": "optional"}, +} + + +def test_zendesk_may_sit_beside_a_service_whose_jwts_name_their_issuer( + project: ProjectBuilder, +) -> None: + """Only Zendesk takes a JWT naming no issuer, and only the other takes one + naming its own authorization server, so neither token passes as the + other's.""" + project.write_project( + { + "version": 1, + "project": "example-project", + "location": "us-central1", + "connections": { + "internal-api": {**INTERNAL_API, "tokens": {"format": "jwt"}}, + "zendesk": ZENDESK_ISSUERLESS, + }, + } + ) + project.write_agent("mail-triage", {"connections": ["zendesk", "internal-api"]}) + assert problems(project) == [] + + +def test_two_optional_issuers_on_one_agent_are_reported( + project: ProjectBuilder, +) -> None: + """A JWT that names no issuer is taken by either, whichever service each is.""" + project.write_project( + { + "version": 1, + "project": "example-project", + "location": "us-central1", + "connections": { + "internal-api": { + **INTERNAL_API, + "tokens": {"format": "jwt", "issuer": "optional"}, + }, + "zendesk": ZENDESK_ISSUERLESS, + }, + } + ) + project.write_agent("mail-triage", {"connections": ["zendesk", "internal-api"]}) + found = problems(project) + assert any( + "tokens.issuer" in p and "internal-api" in p and "zendesk" in p for p in found + ), found + + +def test_zendesk_declared_issuerless_may_be_held_beside_notion_mcp( + project: ProjectBuilder, +) -> None: + """Zendesk's JWTs and Notion MCP's opaque tokens part by shape, so one + agent can search the one and write to the other.""" + project.write_project( + { + "version": 1, + "project": "example-project", + "location": "us-central1", + "connections": {"zendesk": ZENDESK_ISSUERLESS}, + } + ) + project.write_agent("mail-triage", {"connections": ["zendesk", "notion-mcp"]}) + assert problems(project) == [] + + def test_the_catalog_alone_still_leaves_zendesk_accepted_by_elimination( project: ProjectBuilder, ) -> None: