diff --git a/CHANGELOG.md b/CHANGELOG.md index 87f776b..f9aaec0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,31 @@ # Changelog +## 0.3.0 + +- Synced the vendored `v: 3` OpenAPI and fixture contract snapshot, added + `Event.getReason()` and `EventTrigger` v3 signal constants, and documented + empty `visitor_id` (`""`) on unidentifiable events. +- Enforced the `proxy.md` sanitize-and-restore contract across `FingerlyProxy` + and all servlet/WebFlux adapters: validated `X-Fingerly-SDK-Platform` + restoration after stripping `x-fingerly-*`, HTTP `400 missing_origin` when + both `Origin` and a validated native platform header are absent (never + synthesizing `X-Fingerly-Origin`), forwarding `Sec-Fetch-*` and `Sec-CH-UA-*` + headers, and optional cross-origin CORS support. +- `FingerlyProxy` requires a `fly_px_` proxy key and checks the caller's + public key on every request; a secret key is never sent upstream. +- `fingerly.deviceCheck().query(requestId)` and + `fingerly.deviceCheck().write(requestId, DeviceCheckWrite)` (with + `queryAsync` and `writeAsync`) read and write an iOS event's DeviceCheck + bits. They retry the identical request only after a `503` or a transport + failure, and `FingerlyApiException` names each DeviceCheck error code and + `getAppleStatus()`. +- Added `fingerly.events().verify(requestId, VerifyOptions)` (with + `verify(requestId)` and `verifyAsync`), which reads an event and checks it exists, is recent + (`maxAgeSeconds`, 300 by default), carries the expected `tag`, is + identifiable and scored, then consumes it once, and + `fingerly.events().consume(requestId)` (with `consumeAsync`), which throws + `EventAlreadyConsumedException` on `409 event_already_consumed`. + ## 0.2.0 - `Event.getProjectId()` returns the Fingerly project an event belongs to, on diff --git a/README.md b/README.md index 872ceec..795bde6 100644 --- a/README.md +++ b/README.md @@ -35,12 +35,12 @@ Java 11 or newer, and Kotlin and Scala through the same artifacts. Releases are io.fingerly fingerly-server - 0.2.0 + 0.3.0 ``` ```kotlin -implementation("io.fingerly:fingerly-server:0.2.0") +implementation("io.fingerly:fingerly-server:0.3.0") ``` For the proxy, add the adapter for your stack as well (see [Artifacts](#artifacts)). @@ -82,6 +82,7 @@ Full documentation: [docs.fingerly.io/docs/sdks/java](https://docs.fingerly.io/d | Read an event, or verify a backend action | [Read events](#read-events), [Verify a backend action](#verify-a-backend-action) | | Read without blocking a thread | [Blocking and non-blocking reads](#blocking-and-non-blocking-reads) | | Handle errors and unscored events | [Errors](#errors), [Unscored events](#unscored-events) | +| Allow one free trial per iOS device | [DeviceCheck bits on iOS](#devicecheck-bits-on-ios) | | Check a webhook signature | [Verify webhooks](#verify-webhooks) | | Keep identification working behind content blockers | [First-party proxy](#first-party-proxy) | | Pick the adapter for Spring Boot, Tomcat, Jetty or WebFlux | [Artifacts](#artifacts), [Adapters](#adapters) | @@ -96,6 +97,7 @@ Full documentation: [docs.fingerly.io/docs/sdks/java](https://docs.fingerly.io/d - [Unscored events](#unscored-events) - [Read an event you just identified](#read-an-event-you-just-identified) - [Verify a backend action](#verify-a-backend-action) +- [DeviceCheck bits on iOS](#devicecheck-bits-on-ios) - [Verify webhooks](#verify-webhooks) - [First-party proxy](#first-party-proxy) - [Development and release preparation](#development-and-release-preparation) @@ -119,7 +121,7 @@ The servlet APIs and Spring are `provided`: the container supplies them, so noth ## Read events -Event reads require `fly_sk_{production|staging|development}_…` keys. Requests go to `https://api.fingerly.io`; `endpoint()` overrides that for a local API. Public, proxy, management, malformed and unsupported keys are rejected before network access, by `build()` rather than by the first call. Requests do not follow redirects. `get` and `list` each perform one request and never retry. A null, empty, `.` or `..` request id is an `IllegalArgumentException`. +Event reads require `fly_sk_{production|staging|development}_…` keys. Requests go to `https://api.fingerly.io`; `endpoint()` overrides that. It takes `https://host` with an optional port and nothing else: a path, query, fragment, credentials, whitespace or control byte is refused by `build()`. A plain `http` endpoint is refused too, except for a local API with `allowInsecureLoopback(true)` and a host of `127.0.0.1`, `localhost` or `[::1]`. Public, proxy, management, malformed and unsupported keys are rejected before network access, by `build()` rather than by the first call. Requests do not follow redirects. `get` and `list` each perform one request and never retry. A null, empty, `.` or `..` request id is an `IllegalArgumentException`. `getProjectId()` is the UUID of the Fingerly project the event belongs to, on `get` and on every `list` row; it is null when an older server does not send it. @@ -137,7 +139,7 @@ Mono event = Mono.fromFuture(() -> fingerly.events().getAsync(requestId)) An answer larger than 4 MiB is refused rather than buffered: the client throws `FingerlyException`, the proxy answers 502. A gzip or deflate answer is decoded. -Every client and proxy in a process shares one HTTP/1.1 connection pool, so building one per request still reuses connections. The pool is replaced every two minutes so DNS changes are picked up; a request in flight finishes on the connection it started on. Before JDK 21 a replaced pool cannot be closed and keeps its idle connections until the JDK's keep-alive timeout (20 minutes by default), so there it is replaced every 20 minutes instead. There is no cookie jar. Pass `httpClient()` to use your own client instead; configure it with `HttpClient.Redirect.NEVER`. +Every client and proxy in a process shares one HTTP/1.1 connection pool, so building one per request still reuses connections. The pool is replaced every two minutes so DNS changes are picked up; a request in flight finishes on the connection it started on. Before JDK 21 a replaced pool cannot be closed and keeps its idle connections until the JDK's keep-alive timeout (20 minutes by default), so there it is replaced every 20 minutes instead. There is no cookie jar. Pass `httpClient()` to use your own client instead; it must be built with `HttpClient.Redirect.NEVER` (the JDK default). `FingerlyProxy` refuses any other redirect policy when it is built, since every upstream call carries the proxy key. ## Errors @@ -204,11 +206,87 @@ try { Retry a pending read a bounded number of times, for example from a scheduled job; never sleep on an event loop. `isEventNotFound()` means a request id that was never issued, or one past retention, and is never pending. -The archived document arrives moments after the event itself, so check `hasDocument()` rather than assuming it is there. +The archived document arrives moments after the event itself, so check `hasDocument()` rather than assuming it is there, and read `getDocument()` for the allow-listed archived `JsonNode` (`risk`, `attestation`, `ip_network` and `velocity`, with no raw signals, user agent or address). When a submission is unidentifiable (`identifiable == false`, such as insufficient entropy, a genuine embedded WebView, or an unmatched high-risk Web payload), `getVisitorId()` is `""`; there are no `deferred_token`, `deferred_signals` or `supplements` fields in `v: 3`. + +`EventTrigger.V3_SIGNALS` groups the `v: 3` anti-spoofing and integrity signal constants on `EventTrigger`: `SIGNAL_ATTESTATION_FAILED` (`"attestation_failed"`), `SIGNAL_ATTESTATION_DOWNGRADE` (`"attestation_downgrade"`), `SIGNAL_DEVICE_INCONSISTENT` (`"device_inconsistent"`), `SIGNAL_REPORT_INCOMPLETE` (`"report_incomplete"`), `SIGNAL_PROBE_ERRORS` (`"probe_errors"`), `SIGNAL_FETCH_METADATA_MISSING` (`"fetch_metadata_missing"`), `SIGNAL_WEB_TICKET_MISSING` (`"web_ticket_missing"`), `SIGNAL_WEB_TICKET_INVALID` (`"web_ticket_invalid"`), `SIGNAL_TAMPERING` (`"tampering"`), `SIGNAL_CUSTOM_ROOT_CA` (`"custom_root_ca"`) and `SIGNAL_SECONDARY_USER_PROFILE` (`"secondary_user_profile"`). ## Verify a backend action -Read the event with a server-only key. Check that its tag matches your action, that it is recent, that its environment is correct, and that it was scored. Apply your own policy to the server's verdict. A pending event (`isPending()`) is not a verdict: retry after `getRetryAfter()` seconds or verify asynchronously, rather than refusing the action. Do not authorize an action using scores or visitor identifiers supplied by the browser. Tags correlate events; they do not replace your application's authentication or replay protection. Never put a personal identifier (user ID, email, install ID or advertising ID) in a tag: the server keeps tags in long-lived rollups. +Send the request id from the browser or app to your backend and verify it there with a server-only key. `fingerly.events().verify(requestId, options)` reads the event and checks, in this order, stopping at the first that fails: + +1. `not_found`: no event has this request id (never issued, or past retention). +2. `too_old`: the event is older than `maxAgeSeconds` (300 by default). A missing or unreadable `occurred_at` counts as too old. +3. `tag_mismatch`: a `tag` was set and the event's tag is not exactly it. An untagged event never matches. +4. `unidentifiable`: the event's `getVisitorId()` is `""`. +5. `unscored`: the event was not scored. +6. `already_consumed`: every check passed, but another call consumed the event first. + +It consumes the event last, only when every other check passes, so one event backs one decision: a second verify of the same request id answers `already_consumed`. `VerifyOptions.builder().consume(false)` skips the consume. `getReason()` is one of the `VerifyResult.REASON_` constants, `""` when `isOk()`, and `getEvent()` is the event read (null only for `not_found`). + +`verify` never looks at the suspect score, level or triggers: that policy stays with you, applied after `isOk()`. + +```java +import io.fingerly.server.VerifyOptions; +import io.fingerly.server.VerifyResult; + +VerifyResult result; +try { + result = fingerly.events().verify(requestId, + VerifyOptions.builder().tag("checkout").maxAgeSeconds(120).build()); +} catch (FingerlyApiException e) { + if (e.isPending()) return "retry"; // not decided yet: read again after e.getRetryAfter() seconds + throw e; +} +if (!result.isOk()) { + return "refuse"; // result.getReason() says which check failed +} +Event event = result.getEvent(); +if ("high".equals(event.getSuspectLevel()) + || event.getTriggers().stream() + .anyMatch(t -> EventTrigger.SIGNAL_ATTESTATION_FAILED.equals(t.getSignal()))) { + return "refuse"; +} +``` + +`verify(requestId)` uses the defaults, and `verifyAsync` returns a `CompletableFuture` for non-blocking code. Invalid options (a `maxAgeSeconds` that is not positive, or an empty `tag`) throw `IllegalArgumentException` at `build()`, before any request. Any other error, `event_pending` included, is thrown as from `get`. A pending event (`isPending()`) is not a verdict: retry after `getRetryAfter()` seconds or verify asynchronously, rather than refusing the action. + +To consume an event yourself, call `fingerly.events().consume(requestId)` (or `consumeAsync`). The first call returns a `ConsumeResult` with `getConsumedAt()` and `isFirst()`; every later one throws `EventAlreadyConsumedException`, a `FingerlyApiException` with status `409`, code `event_already_consumed` and `getConsumedAt()`. Consumes are never retried. + +Do not authorize an action using scores or visitor identifiers supplied by the browser. Tags correlate events; they do not replace your application's authentication. Never put a personal identifier (user ID, email, install ID or advertising ID) in a tag: the server keeps tags in long-lived rollups. + +## DeviceCheck bits on iOS + +For an iOS identification whose App Attest evidence carried an Apple DeviceCheck token, Fingerly holds the token for one hour, and your backend can spend it once to read or to write the device's two bits. The bits survive a reinstall, and they mean what you decide. + +Apple treats a DeviceCheck token as single-use, so a read and a write each need their own identification. A once-per-device free trial takes two: + +1. Query on the identification that asks for the trial. If the bits say the device already had one, refuse it. +2. Otherwise grant the trial and record the grant in your own data, then write the bit on the device's next identification. Until the write lands, your own record is what stops a second trial. + +```java +import io.fingerly.server.DeviceCheckBits; +import io.fingerly.server.DeviceCheckWrite; +import io.fingerly.server.FingerlyApiException; + +// On the identification that asks for the trial. +try { + DeviceCheckBits bits = fingerly.deviceCheck().query(requestId); + if (bits.isFound() && bits.getBit0()) { + return "offer-paid-plan"; + } +} catch (FingerlyApiException e) { + if (!e.isDeviceCheckNotHeld() && !e.isDeviceCheckSpent()) throw e; + // No token for this event: decide without the bits. +} +trials.grant(accountId, event.getVisitorId()); + +// On the next identification from the same install. +fingerly.deviceCheck().write(nextRequestId, DeviceCheckWrite.builder().bit0(true).build()); +``` + +`queryAsync` and `writeAsync` return a `CompletableFuture` for non-blocking code. A write names `bit0`, `bit1` or both; `DeviceCheckWrite.builder().build()` with neither throws `IllegalArgumentException`. + +The SDK retries a call, with the identical request, only after a `503` or a transport failure, up to twice. The API keeps the token after such an answer only for a retry of the same request: the other operation, or a write with other bits, answers `409 device_check_spent`. Errors arrive as `FingerlyApiException`: `isDeviceCheckNotHeld()` (`404`), `isDeviceCheckSpent()`, `isDeviceCheckBusy()`, `isCredentialUnavailable()` (`409`), `isBitsRequired()`, `isAppleRejected()` (`422`, with Apple's status in `getAppleStatus()`) and `isDeviceCheckUnavailable()` (`503`). Never rely on the bits as the only fraud check, and give customers a way to have them reset. ## Verify webhooks @@ -238,7 +316,9 @@ FingerlyProxy proxy = FingerlyProxy.builder() `clientIp` must return exactly one IPv4 or IPv6 address. Anything else (null, empty, a list, a port, a host name) is a configuration error: the request is refused with a 500 saying "The client IP callback returned no valid address", nothing is sent upstream, and the mistake is logged once. -`path` is optional where the framework already routes only that path to the proxy, and required by the WebFlux adapter, which sees every request. +Set `path` in every setup, as the examples below do. The servlet filters accept a proxy without one where the container maps only that path to the filter, but then the container decides what reaches it, including paths it normalises, such as a trailing slash or a dot segment. With `path` set, the proxy compares the raw path itself and answers near misses with its own uncacheable `404`, or passes them to the application. The WebFlux adapter, which sees every request, requires it. + +`upstream(...)` changes the API origin. It takes `https://host` with an optional port and nothing else: a path, query, fragment, credentials, whitespace or control byte is refused when the proxy is built. A plain `http` upstream is refused too, except for local development with `allowInsecureLoopback(true)` and a host of `127.0.0.1`, `localhost` or `[::1]`. Then point the browser SDK at the same path: @@ -338,25 +418,30 @@ Build a `ProxyRequest` (method, raw path, headers, body, `remoteAddress`) and wr ### What the proxy does -The browser SDK posts both of its calls to that one path, and the proxy forwards every `POST` it answers to Fingerly's one ingest URL, `https://api.fingerly.io/api/v1/ingest` (`apiVersion` changes only the version in it). Any other method gets `405`. Event reads need a secret key and stay unreachable from a browser by construction. +Browser and proxied native client SDKs post a single `v: 3` identification envelope to that one path, and the proxy forwards every `POST` it answers with its raw request body bytes untouched to Fingerly's one ingest URL, `https://api.fingerly.io/api/v1/ingest` (`apiVersion` changes only the version in it). Any method other than `POST` (plus `OPTIONS` when `allowedOrigins(...)` is configured) gets `405`, with `Cache-Control: no-store`. Methods are compared exactly, so `post` gets `405` too. Event reads need a secret key and stay unreachable from a browser by construction. -With `path` set, the proxy answers exactly that path, compared with the whole raw request path including any context path, case-sensitively. A trailing or doubled slash, a dot segment, percent encoding or a path parameter makes a different path: it gets `404`, or passes through to the application under a filter, as does anything under the path. Without `path`, it answers whatever the framework routed to it. `path` must be a plain absolute path such as `/k7x2qm4a`. +With `path` set, the proxy answers exactly that path, compared with the whole raw request path including any context path, case-sensitively. A trailing or doubled slash, a dot segment, percent encoding or a path parameter makes a different path: it gets `404`, or passes through to the application under a filter, as does anything under the path (including retired `/supplement` subpaths). Without `path`, it answers whatever the framework routed to it. `path` must be a plain absolute path such as `/k7x2qm4a`. | Situation | Answer | | --- | --- | | Path other than the configured one | `404`, or passed through to the application under a filter | -| Method other than `POST` | `405` | -| Public key from a different environment than the proxy key | `401` | +| Method other than `POST` (or `OPTIONS` when `allowedOrigins(...)` is configured) | `405` | +| Missing `X-Api-Key`, a secret key, or a public key from a different environment than the proxy key | `401` | +| Neither an `Origin` nor a validated `X-Fingerly-SDK-Platform` | `400` (`missing_origin`) | | Body over 1 MiB (enforced while reading rather than after buffering) | `413` | | Upstream did not answer within `timeout()` (5 seconds by default) | `504` ("Gateway timeout") | -| Anything else: refused, reset, truncated, larger than 4 MiB, or compressed in a way that cannot be decoded | `502` ("Bad gateway") | +| Anything else: refused, reset, truncated, larger than 4 MiB, compressed in a way that cannot be decoded, a redirect (never followed), or a status outside 200 to 599 | `502` ("Bad gateway") | | `clientIp` returns no valid address | `500` ("The client IP callback returned no valid address") | -The browser must present a public key from the same environment as the proxy key, or the request gets `401`. Only the content type, the visitor metadata Fingerly needs, and an `Idempotency-Key` are forwarded; customer cookies and authorization headers never are, and a visitor header that could not be forwarded is dropped. The proxy reports itself upstream as `java/0.2.0`, which the dashboard shows per proxy. +The proxy key (`fly_px_{production|staging|development}_…`) is required: `build()` throws `IllegalArgumentException` when it is missing or malformed. On every request the caller's `X-Api-Key` must be a public key (`fly_pk_…`) from the same environment as the proxy key. Anything else, a missing header, another environment's key or a secret `fly_sk_` key, gets a plain-text `401` with `Cache-Control: no-store`, and nothing is forwarded. Upstream receives the proxy key as `X-Fingerly-Proxy-Key` and the caller's public key as `X-Api-Key`; a secret key is never sent. + +Incoming headers follow the sanitize-and-restore sequence: all incoming `X-Fingerly-*`, `Cookie` and `Authorization` headers are stripped, restoring `X-Fingerly-SDK-Platform` only if it matches an exact lowercase native or bridge platform (`ios`, `android`, `react-native`, `react-native-ios`, `react-native-android`, `flutter`, `flutter-ios`, `flutter-android`, `lynx-ios`, `lynx-android`, `capacitor`, `kotlin-multiplatform`). When both `Origin` and a validated `X-Fingerly-SDK-Platform` are absent, the request is rejected locally with `400` (`missing_origin`), and `X-Fingerly-Origin` is never synthesized from `Host`, `Referer` or the request URL. A request carries one channel upstream: with an `Origin`, the proxy sends `X-Fingerly-Origin` and drops the platform header, as the API does on a direct call; without one, it sends the validated `X-Fingerly-SDK-Platform`. + +Alongside verified proxy metadata (`X-Fingerly-Host` included), only `Content-Type`, `X-Api-Key`, `User-Agent`, `Idempotency-Key`, `Sec-Fetch-*` and `Sec-CH-UA-*` are forwarded; customer cookies and authorization headers never are, and a visitor header that could not be forwarded is dropped. The proxy reports itself upstream as `java/0.3.0`, which the dashboard shows per proxy. Only `Content-Type`, `X-Request-Id`, `RateLimit-Limit`, `RateLimit-Remaining` and `Retry-After` are relayed back, plus `Cache-Control: no-store`, so nothing on the response names Fingerly. The answer is read whole before anything is relayed, so a truncated one is never passed on as a success. -One line is logged per failure, without a stack trace or a key, to the `System.Logger` named `io.fingerly.server.FingerlyProxy` at `WARNING` (which reaches SLF4J, Logback or Log4j through their usual bridges), or to your own `log(...)` callback. +Every refusal and failure answer carries `Cache-Control: no-store`. One line is logged per failure, naming only the kind of failure, never the upstream address, a stack trace or a key, to the `System.Logger` named `io.fingerly.server.FingerlyProxy` at `WARNING` (which reaches SLF4J, Logback or Log4j through their usual bridges), or to your own `log(...)` callback. Every adapter asks `proxy.handles(path)` before reading anything. A filter may be mapped more broadly than the one path the proxy owns, and calling `handle()` to find out would have consumed a request body the application still needs. @@ -374,7 +459,7 @@ The vendored synthetic contract snapshot makes tests independent of sibling repo
Source structure -`FingerlyClient` holds the options; `EventsService` owns the reads and the list filters, and `Event`, `EventPage` and `EventListParams` are what they speak in. `Webhooks` verifies signatures. `FingerlyProxy` holds every proxy decision, with `ProxyRequest` and `ProxyResponse` as the neutral shapes the adapters translate to. `SharedHttp` is the process-wide connection pool, `BoundedBody` reads every answer within its limit, and `Deadlines` is the one timer thread behind every deadline. `Keys` validates every caller-supplied option before any network access, and `Json` keeps the Jackson configuration in one place so a caller's own `ObjectMapper` cannot change how responses are read. +`FingerlyClient` holds the options; `EventsService` owns the reads and the list filters, and `Event`, `EventPage` and `EventListParams` are what they speak in. It also verifies and consumes events, with `VerifyOptions`, `VerifyResult`, `ConsumeResult` and `EventAlreadyConsumedException`. `DeviceCheckService` spends an event's DeviceCheck token, with `DeviceCheckBits`, `DeviceCheckWrite` and `DeviceCheckWriteResult`. `Webhooks` verifies signatures. `FingerlyProxy` holds every proxy decision, with `ProxyRequest` and `ProxyResponse` as the neutral shapes the adapters translate to. `SharedHttp` is the process-wide connection pool, `BoundedBody` reads every answer within its limit, and `Deadlines` is the one timer thread behind every deadline. `Keys` validates every caller-supplied option before any network access, and `Json` keeps the Jackson configuration in one place so a caller's own `ObjectMapper` cannot change how responses are read.
diff --git a/contract/manifest.json b/contract/manifest.json index 7303153..f5d8ba0 100644 --- a/contract/manifest.json +++ b/contract/manifest.json @@ -1,8 +1,8 @@ { "schema": 1, - "serverCommit": "e22f96777b0d029464baa4e127d7a346b684d478", + "serverCommit": "2f24510cfb9a41f780936841cc47ca8a1c012201", "files": { - "openapi.json": "850e7627dbbf1789fe7a0a214ae28838f2cd80d42e251f9994f5cb42d72e6009", + "openapi.json": "87c3881291f8a9cba673f2a62a3bad85f6ff7f8195c01efcae54f84f6887bd93", "fixtures.json": "3fbf2efbcf3c8ebced3d2c9f2a3c1f5752ee94ece63d5c110291d9a8251ff68f" } } diff --git a/contract/openapi.json b/contract/openapi.json index e1483dd..487d545 100644 --- a/contract/openapi.json +++ b/contract/openapi.json @@ -1,6 +1,80 @@ { "components": { "schemas": { + "AttestEnvelope": { + "properties": { + "assertion": { + "description": "Base64 CBOR App Attest assertion object", + "type": "string" + }, + "attestation_object": { + "description": "Base64 CBOR App Attest attestation object", + "type": "string" + }, + "certificate_chain": { + "description": "Base64 DER X.509 certificate chain for Android key registration", + "items": { + "type": "string" + }, + "type": "array" + }, + "challenge": { + "description": "Stateless server-signed challenge ticket", + "type": "string" + }, + "device_token": { + "description": "Optional standard Base64 Apple DeviceCheck token, at most 4 KiB decoded, bound into statementDigest. Held for one hour and spent by POST /events/{request_id}/device-check/query", + "type": "string" + }, + "device_token_write": { + "description": "Optional second standard Base64 Apple DeviceCheck token, at most 4 KiB decoded, bound into statementDigest. Held for one hour and spent by PUT /events/{request_id}/device-check", + "type": "string" + }, + "install_id": { + "description": "Canonical 36-char lowercase UUIDv4 matching sources.identity.value.install_id", + "type": "string" + }, + "key_id": { + "description": "Base64 App Attest key ID (iOS) or 64-char hex SPKI SHA-256 (Android)", + "type": "string" + }, + "mode": { + "description": "app_attest_register, app_attest_assert, android_key_register, or android_key_assert", + "type": "string" + }, + "play_integrity_token": { + "description": "Optional Google Play Integrity token with requestHash equal to hex(statementDigest)", + "type": "string" + }, + "prev_assertion": { + "description": "iOS, app_attest_register only: standard Base64 CBOR App Attest assertion by the previous confirmed key (prev_key_id) with clientDataHash equal to statementDigest", + "type": "string" + }, + "prev_key_id": { + "description": "Optional key ID of the previous confirmed key: 64-char hex (Android) or standard Base64 App Attest key ID (iOS, app_attest_register only)", + "type": "string" + }, + "prev_signature": { + "description": "Optional Base64 ASN.1 DER signature by retiring Android key over identical framedPreimage", + "type": "string" + }, + "reattest_attempt": { + "description": "Inline Android lease renewal attempt counter (0 or 1)", + "format": "int64", + "type": "integer" + }, + "signature": { + "description": "Base64 ASN.1 DER SHA256withECDSA signature over framedPreimage", + "type": "string" + } + }, + "required": [ + "challenge", + "install_id", + "mode" + ], + "type": "object" + }, "AttestationChallengeOut": { "properties": { "app_attest": { @@ -15,6 +89,10 @@ "description": "Whether to send a DeviceCheck token with the App Attest evidence", "type": "boolean" }, + "digest_salt": { + "description": "The project's 32-byte name digest salt, unpadded base64url. The SDK caches it per SDK key and sends each name digest as the first 16 bytes of HMAC-SHA256(salt, name) in hex, with digest_salt_id. Absent when the server has none to give; the SDK then uses 32 zero bytes", + "type": "string" + }, "expires_at": { "description": "When the nonce becomes unusable", "format": "date-time", @@ -436,10 +514,29 @@ }, "IdentifyIn": { "properties": { + "attest": { + "anyOf": [ + { + "$ref": "#/components/schemas/AttestEnvelope" + }, + { + "type": "null" + } + ], + "description": "Hardware attestation envelope bound to raw signals bytes (v3 native SDKs)" + }, + "attest_skipped": { + "description": "Native only, and only without attest: why the SDK sent no attestation. One of not_offered, unsupported, challenge_unavailable, key_error, timeout, backoff, install_unavailable. Diagnostic; it never lowers the score", + "type": "string" + }, "consent": { "description": "The SDK's consent state; must be granted, or the request is refused with consent_required", "type": "string" }, + "op": { + "description": "Empty for a report. ticket asks for a web freshness ticket instead: a public key on the web only, with a body of v, op and sdk, answered 200 with ticket and expires_at, never charged", + "type": "string" + }, "sdk": { "$ref": "#/components/schemas/SDKInfo", "description": "Which SDK collected these signals" @@ -450,30 +547,65 @@ "type": "object" }, "tag": { - "description": "An opaque value echoed back on the event and kept in long-lived rollups. A label for your own systems, such as a client or app name; never a personal identifier (user ID, email, install ID or advertising ID)", + "description": "An opaque value echoed back on the event and kept in long-lived rollups. A label for your own systems, such as a client or app name; never a personal identifier (user ID, email, install ID or advertising ID). A value holding a control character (U+0000 to U+001F, U+007F) is refused with invalid_request", "type": "string" + }, + "ticket": { + "description": "Web only: the freshness ticket from a ticket request or the previous answer's next_ticket. Single use, bound to the key and page origin, valid for 300 seconds. A native report carrying one is refused with invalid_request", + "type": "string" + }, + "ticket_pow": { + "description": "Web only, next to a ticket whose answer carried pow_bits: the smallest decimal nonce n, from 0 up, such that SHA-256(ticket || ':' || n) starts with at least pow_bits zero bits", + "type": "string" + }, + "v": { + "description": "Wire envelope version. Must equal 3.", + "format": "int64", + "type": "integer" } }, "type": "object" }, "IdentifyOut": { "properties": { - "app_attest_key": { - "description": "unknown when the report asserted with an App Attest key the server does not hold; the iOS SDK then attests a new key on its next collection. Absent otherwise", + "attest_key_id": { + "description": "Native only, present whenever attest was sent: the key_id the server evaluated, copied from attest.key_id", "type": "string" }, - "deferred_token": { - "description": "Short-lived capability for the deferred signal report", + "attest_key_status": { + "description": "Native only, present whenever attest was sent: registered (the key was verified, stored and owns the install), valid (an assertion verified against a stored key), rekey_required (the key is unknown or expired), rejected (attestation failed)", "type": "string" }, "duplicate": { - "description": "This submission was answered before; the verdict belongs to the first one", + "description": "This submission was answered before; the verdict replays the first one", "type": "boolean" }, "identifiable": { "description": "False when the report carried too little to identify anyone", "type": "boolean" }, + "next_attestation": { + "anyOf": [ + { + "$ref": "#/components/schemas/AttestationChallengeOut" + }, + { + "type": "null" + } + ], + "description": "Fresh attestation challenge ticket and dynamic offer flags for the next native submission" + }, + "next_ticket": { + "anyOf": [ + { + "$ref": "#/components/schemas/WebTicketOut" + }, + { + "type": "null" + } + ], + "description": "Web only: a fresh freshness ticket for the next report from this page, so it needs no ticket request" + }, "reason": { "description": "Why enrichment did not happen. Only answered to a secret key; a browser or app reads it on the server-side event read", "type": "string" @@ -511,7 +643,7 @@ "type": "integer" }, "visitor_id": { - "description": "Who this visitor is, stable across their visits", + "description": "Who this visitor is, stable across their visits; empty when unidentifiable", "type": "string" }, "visitor_is_new": { @@ -556,10 +688,29 @@ }, "IngestIn": { "properties": { + "attest": { + "anyOf": [ + { + "$ref": "#/components/schemas/AttestEnvelope" + }, + { + "type": "null" + } + ], + "description": "Hardware attestation envelope bound to raw signals bytes (v3 native SDKs)" + }, + "attest_skipped": { + "description": "Native only, and only without attest: why the SDK sent no attestation. One of not_offered, unsupported, challenge_unavailable, key_error, timeout, backoff, install_unavailable. Diagnostic; it never lowers the score", + "type": "string" + }, "consent": { "description": "The SDK's consent state; must be granted, or the request is refused with consent_required", "type": "string" }, + "op": { + "description": "Empty for a report. ticket asks for a web freshness ticket instead: a public key on the web only, with a body of v, op and sdk, answered 200 with ticket and expires_at, never charged", + "type": "string" + }, "sdk": { "$ref": "#/components/schemas/SDKInfo", "description": "Which SDK collected these signals" @@ -570,12 +721,21 @@ "type": "object" }, "tag": { - "description": "An opaque value echoed back on the event and kept in long-lived rollups; never a personal identifier. Identify only", + "description": "An opaque value echoed back on the event and kept in long-lived rollups; never a personal identifier. A value holding a control character (U+0000 to U+001F, U+007F) is refused with invalid_request", "type": "string" }, - "token": { - "description": "The deferred_token returned by identify. Present only on a deferred report", + "ticket": { + "description": "Web only: the freshness ticket from a ticket request or the previous answer's next_ticket. A native report carrying one is refused with invalid_request", "type": "string" + }, + "ticket_pow": { + "description": "Web only: the proof of work for a ticket whose answer carried pow_bits", + "type": "string" + }, + "v": { + "description": "Wire envelope version. Must equal 3.", + "format": "int64", + "type": "integer" } }, "type": "object" @@ -636,6 +796,41 @@ ], "type": "object" }, + "PrivateAccessTokenIssuerIn": { + "properties": { + "directory_url": { + "description": "Instead of token_key: the issuer's HTTPS directory, read for its current type 2 keys", + "type": "string" + }, + "issuer_name": { + "description": "The issuer's server name, host and optional port. Defaults to the directory URL's host", + "type": "string" + }, + "token_key": { + "description": "The issuer's token type 2 key, base64url: a DER SubjectPublicKeyInfo of a 2048-bit RSASSA-PSS key with SHA-384", + "type": "string" + } + }, + "type": "object" + }, + "PrivateAccessTokenIssuerOut": { + "properties": { + "directory_url": { + "type": "string" + }, + "issuer_name": { + "type": "string" + }, + "token_key": { + "description": "base64url, padded", + "type": "string" + } + }, + "required": [ + "issuer_name" + ], + "type": "object" + }, "ProjectArchivedOut": { "properties": { "archived": { @@ -948,6 +1143,29 @@ ], "type": "object" }, + "SDKEventConsumeOut": { + "properties": { + "consumed_at": { + "description": "When it was consumed, RFC 3339", + "format": "date-time", + "type": "string" + }, + "first": { + "description": "Always true: a later consume is answered 409 event_already_consumed", + "type": "boolean" + }, + "request_id": { + "description": "The event consumed", + "type": "string" + } + }, + "required": [ + "consumed_at", + "first", + "request_id" + ], + "type": "object" + }, "SDKEventOut": { "properties": { "agent": { @@ -965,6 +1183,14 @@ "asn_name": { "type": "string" }, + "consumed_at": { + "description": "When the event was first consumed with POST /events/{request_id}/consume. Absent while it has not been", + "format": "date-time", + "type": [ + "string", + "null" + ] + }, "country_code": { "type": "string" }, @@ -1183,6 +1409,21 @@ "description": "The human-readable stem, such as fly_pk_production", "type": "string" }, + "private_access_token_issuer": { + "anyOf": [ + { + "$ref": "#/components/schemas/PrivateAccessTokenIssuerOut" + }, + { + "type": "null" + } + ], + "description": "Prototype: the project's token issuer, on the answer to an origins update" + }, + "private_access_tokens": { + "description": "Prototype: whether a capable Safari is asked for a Private Access Token when the page requests a web ticket", + "type": "boolean" + }, "project_id": { "type": "string" }, @@ -1203,6 +1444,11 @@ "status": { "description": "active or revoked", "type": "string" + }, + "web_ticket_pow_bits": { + "description": "Proof of work, in leading zero bits, the key's web freshness tickets ask for; 0 is none", + "format": "int64", + "type": "integer" } }, "required": [ @@ -1218,9 +1464,11 @@ "last4", "name", "prefix", + "private_access_tokens", "project_id", "require_attestation", - "status" + "status", + "web_ticket_pow_bits" ], "type": "object" }, @@ -1341,21 +1589,6 @@ ], "type": "object" }, - "SupplementOut": { - "properties": { - "duplicate": { - "description": "The identical supplement was accepted before", - "type": "boolean" - }, - "request_id": { - "type": "string" - } - }, - "required": [ - "request_id" - ], - "type": "object" - }, "UsageDayOut": { "properties": { "billable": { @@ -1571,6 +1804,29 @@ ], "type": "object" }, + "WebTicketOut": { + "properties": { + "expires_at": { + "description": "When the ticket stops being accepted, RFC 3339 UTC", + "format": "date-time", + "type": "string" + }, + "pow_bits": { + "description": "Present when the key asks for proof of work: the leading zero bits ticket_pow must produce", + "format": "int64", + "type": "integer" + }, + "ticket": { + "description": "Unpadded base64url ticket, sent as ticket on the next report", + "type": "string" + } + }, + "required": [ + "expires_at", + "ticket" + ], + "type": "object" + }, "WebhookDeliveriesOut": { "properties": { "deliveries": { @@ -2075,10 +2331,10 @@ "x-title": "Get an event" } }, - "/api/v1/events/{request_id}/device-check": { - "put": { - "description": "Spends the DeviceCheck token held for this iOS event, once, to set bit0, bit1 or both with Apple, using your project's own Apple key. A bit left out keeps its value. The bits mean what you decide; Fingerly never reads them. Tokens are held for one hour, only for events whose App Attest evidence covered them, and only when DeviceCheck is switched on for the project's Apple credential.", - "operationId": "setDeviceDevicecheckBits", + "/api/v1/events/{request_id}/consume": { + "post": { + "description": "Marks one identification event as consumed, atomically and once per organization, so a request ID handed from a page to your backend backs one decision only. The first call answers `200` with `first: true`; every later call answers `409 event_already_consumed`, with the first consume's time in the error's `details` entry for the field `consumed_at`. Read and check the event before you consume it. The flag is kept with the event and erased with it.", + "operationId": "markEventConsumedOnce", "parameters": [ { "in": "path", @@ -2089,38 +2345,12 @@ } } ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "properties": { - "bit0": { - "description": "The value to set for the first bit. Leave out to keep it", - "type": [ - "boolean", - "null" - ] - }, - "bit1": { - "description": "The value to set for the second bit. Leave out to keep it", - "type": [ - "boolean", - "null" - ] - } - }, - "type": "object" - } - } - }, - "required": true - }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/DeviceCheckWriteOut" + "$ref": "#/components/schemas/SDKEventConsumeOut" } } }, @@ -2154,7 +2384,7 @@ } } }, - "description": "`device_check_not_held`: no token is held for that event" + "description": "`event_not_found`: no event with that request ID in this key's environment, or it is older than 30 days. `event_pending`: the identification is still being recorded; retry after the `Retry-After` seconds" }, "409": { "content": { @@ -2164,7 +2394,7 @@ } } }, - "description": "`device_check_spent`, `device_check_busy` or `credential_unavailable`" + "description": "`event_already_consumed`: consumed before; `details[0]` is `{\"field\": \"consumed_at\", \"issue\": \"\"}`" }, "422": { "content": { @@ -2174,7 +2404,7 @@ } } }, - "description": "`bits_required` or `apple_rejected`" + "description": "The request could not be validated." }, "503": { "content": { @@ -2184,7 +2414,7 @@ } } }, - "description": "`device_check_unavailable`: retry within the hour" + "description": "`events_unavailable`: retry after the `Retry-After` seconds" }, "default": { "content": { @@ -2202,18 +2432,18 @@ "secretKey": [] } ], - "summary": "Set the device's DeviceCheck bits", + "summary": "Mark an event consumed, once", "tags": [ "Server API" ], "x-category": "Server API", - "x-title": "Write an event's DeviceCheck bits" + "x-title": "Consume an event" } }, - "/api/v1/events/{request_id}/device-check/query": { - "post": { - "description": "Spends the DeviceCheck token held for this iOS event, once, to read the device's two bits from Apple. A token spent on a read cannot also write; to mark a device, write instead.", - "operationId": "readDeviceDevicecheckBits", + "/api/v1/events/{request_id}/device-check": { + "put": { + "description": "Spends the write DeviceCheck token held for this iOS event, once, to set bit0, bit1 or both with Apple, using your project's own Apple key. A bit left out keeps its value. The bits mean what you decide; Fingerly never reads them. Apple answers a write with no previous value; the event's read token, spent by the query route, is separate, so one identification supports one read and one write. Tokens are held for one hour, only for events whose App Attest evidence covered them, and only when DeviceCheck is switched on for the project's Apple credential. After a `503`, retry with the same bits: a token kept for a retry accepts only the request it was first used for.", + "operationId": "setDeviceDevicecheckBits", "parameters": [ { "in": "path", @@ -2224,12 +2454,38 @@ } } ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "properties": { + "bit0": { + "description": "The value to set for the first bit. Leave out to keep it", + "type": [ + "boolean", + "null" + ] + }, + "bit1": { + "description": "The value to set for the second bit. Leave out to keep it", + "type": [ + "boolean", + "null" + ] + } + }, + "type": "object" + } + } + }, + "required": true + }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/DeviceCheckQueryOut" + "$ref": "#/components/schemas/DeviceCheckWriteOut" } } }, @@ -2263,7 +2519,7 @@ } } }, - "description": "`device_check_not_held`: no token is held for that event" + "description": "`device_check_not_held`: no write token is held for that event" }, "409": { "content": { @@ -2273,7 +2529,7 @@ } } }, - "description": "`device_check_spent`, `device_check_busy` or `credential_unavailable`" + "description": "`device_check_spent`: already used, or a retry that is not the first request; `device_check_busy` or `credential_unavailable`" }, "422": { "content": { @@ -2283,7 +2539,7 @@ } } }, - "description": "The request could not be validated." + "description": "`bits_required` or `apple_rejected`" }, "503": { "content": { @@ -2311,18 +2567,18 @@ "secretKey": [] } ], - "summary": "Read the device's DeviceCheck bits", + "summary": "Set the device's DeviceCheck bits", "tags": [ "Server API" ], "x-category": "Server API", - "x-title": "Read an event's DeviceCheck bits" + "x-title": "Write an event's DeviceCheck bits" } }, - "/api/v1/events/{request_id}/supplement": { + "/api/v1/events/{request_id}/device-check/query": { "post": { - "description": "Attaches the signals a browser collects after its initial budget to an identification that has already been answered. It never changes the visitor, the score, billing or webhooks. The JavaScript SDK sends it automatically. Resending the same report is answered as a duplicate.", - "operationId": "submitDeferredReport", + "description": "Spends the read DeviceCheck token held for this iOS event, once, to read the device's two bits from Apple. Apple asks for each token to be used once, so the event holds a second token for the write route: to read and then mark a device, read and write on the same identification. After a `503`, retry the read: a token kept for a retry accepts only the request it was first used for.", + "operationId": "readDeviceDevicecheckBits", "parameters": [ { "in": "path", @@ -2333,45 +2589,16 @@ } } ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "properties": { - "consent": { - "description": "The SDK's consent state; must be granted, or the request is refused with consent_required", - "type": "string" - }, - "sdk": { - "$ref": "#/components/schemas/SDKInfo", - "description": "Which SDK collected these signals" - }, - "signals": { - "additionalProperties": {}, - "description": "A report containing only the deferred tier", - "type": "object" - }, - "token": { - "description": "The deferred_token returned by identify", - "type": "string" - } - }, - "type": "object" - } - } - }, - "required": true - }, "responses": { - "202": { + "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/SupplementOut" + "$ref": "#/components/schemas/DeviceCheckQueryOut" } } }, - "description": "Accepted" + "description": "OK" }, "401": { "content": { @@ -2381,9 +2608,9 @@ } } }, - "description": "No usable x-api-key header was presented" + "description": "No usable secret SDK key was presented" }, - "402": { + "403": { "content": { "application/json": { "schema": { @@ -2391,7 +2618,17 @@ } } }, - "description": "The organization this key belongs to is not accepting traffic" + "description": "The organization is suspended or closed" + }, + "404": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "`device_check_not_held`: no read token is held for that event" }, "409": { "content": { @@ -2401,7 +2638,7 @@ } } }, - "description": "A different deferred report was already accepted for this request" + "description": "`device_check_spent`: already used, or a retry that is not the first request; `device_check_busy` or `credential_unavailable`" }, "422": { "content": { @@ -2411,7 +2648,7 @@ } } }, - "description": "The request could not be validated." + "description": "`apple_rejected`: Apple rejected the token; `apple_status` says how" }, "503": { "content": { @@ -2421,7 +2658,7 @@ } } }, - "description": "The report could not be accepted just now" + "description": "`device_check_unavailable`: retry within the hour" }, "default": { "content": { @@ -2436,15 +2673,15 @@ }, "security": [ { - "publicKey": [] + "secretKey": [] } ], - "summary": "Submit a deferred report", + "summary": "Read the device's DeviceCheck bits", "tags": [ - "Client API" + "Server API" ], - "x-category": "Client API", - "x-title": "Submit a deferred report" + "x-category": "Server API", + "x-title": "Read an event's DeviceCheck bits" } }, "/api/v1/identify": { @@ -2492,6 +2729,16 @@ }, "description": "The organization this key belongs to is not accepting traffic" }, + "409": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + }, + "description": "The Idempotency-Key was reused with a different payload, or the original request is still in progress" + }, "413": { "content": { "application/json": { @@ -2561,7 +2808,7 @@ }, "/api/v1/ingest": { "post": { - "description": "The one URL the client SDKs and every first-party proxy post to. A body with a `token` is a deferred report and is answered `202` with `request_id` and `duplicate`, exactly as the deferred report route answers. Any other body is an identification and is answered `200` exactly as the identify route answers.\n\nIt exists so a proxy can forward everything it receives to a single path, and so the SDK can post both calls to one path of the customer's choosing, with nothing appended that a content blocker could match across sites.\n\nThe `X-Request-Id` response header names this HTTP exchange, for support and logs. It is not the `request_id` in the body, which names the identification and is what `GET /events/{request_id}` reads.", + "description": "The one URL the client SDKs and every first-party proxy post to. Forwards the single-pass v3 identification envelope and answers `200` with the verdict, exactly as the identify route answers.\n\nIt exists so a proxy can forward everything it receives to a single path of the customer's choosing, with nothing appended that a content blocker could match across sites. An `Idempotency-Key` header is required on every call.\n\nThe `X-Request-Id` response header names this HTTP exchange, for support and logs. It is not the `request_id` in the body, which names the identification and is what `GET /events/{request_id}` reads.", "operationId": "submitFromClientSdk", "requestBody": { "content": { @@ -2584,7 +2831,7 @@ }, "description": "OK" }, - "202": { + "401": { "content": { "application/json": { "schema": { @@ -2592,9 +2839,9 @@ } } }, - "description": "The deferred report was accepted" + "description": "No usable x-api-key header was presented" }, - "401": { + "402": { "content": { "application/json": { "schema": { @@ -2602,9 +2849,9 @@ } } }, - "description": "No usable x-api-key header was presented" + "description": "The organization this key belongs to is not accepting traffic" }, - "402": { + "409": { "content": { "application/json": { "schema": { @@ -2612,7 +2859,7 @@ } } }, - "description": "The organization this key belongs to is not accepting traffic" + "description": "The Idempotency-Key was reused with a different payload, or the original request is still in progress" }, "413": { "content": { @@ -5333,7 +5580,7 @@ "type": "array" }, "allowed_android_signers": { - "description": "SHA-256 signing certificate digests, 64 hex characters", + "description": "SHA-256 signing certificate digests, 64 hex characters (colon separators accepted). Required alongside allowed_android_packages in production or when require_attestation is true", "items": { "type": "string" }, @@ -5365,8 +5612,11 @@ ] }, "hide_client_score": { - "description": "Withhold suspect_score and suspect_level from a browser, an app, a proxy and a hosted domain. Defaults to false", - "type": "boolean" + "description": "Withhold suspect_score and suspect_level from a browser, an app, a proxy and a hosted domain. Defaults to true for a public key and false for a secret key", + "type": [ + "boolean", + "null" + ] }, "kind": { "description": "public or secret", @@ -5375,6 +5625,13 @@ "name": { "description": "What the key is called. The only part a person can recognise once the secret is masked", "type": "string" + }, + "require_attestation": { + "description": "Refuse native reports whose hardware attestation did not verify", + "type": [ + "boolean", + "null" + ] } }, "type": "object" @@ -5503,7 +5760,7 @@ "type": "array" }, "allowed_android_signers": { - "description": "SHA-256 signing certificate digests, 64 hex characters. Empty means not configured", + "description": "SHA-256 signing certificate digests, 64 hex characters (colon separators accepted). Required alongside allowed_android_packages in production or when require_attestation is true", "items": { "type": "string" }, @@ -5794,6 +6051,32 @@ "type": "string" }, "type": "array" + }, + "private_access_token_issuer": { + "anyOf": [ + { + "$ref": "#/components/schemas/PrivateAccessTokenIssuerIn" + }, + { + "type": "null" + } + ], + "description": "Prototype: the token issuer of the key's whole project. Leave out to keep it; send an empty object to clear it" + }, + "private_access_tokens": { + "description": "Prototype: ask Safari on iOS 16+ and macOS 13+ for a Private Access Token when the page requests a web ticket. Needs a public key in a project that names a token issuer. Leave out to keep the current setting", + "type": [ + "boolean", + "null" + ] + }, + "web_ticket_pow_bits": { + "description": "Proof of work the key's web freshness tickets ask for, 0 (off) to 22 leading zero bits. Leave out to keep the current setting", + "format": "int64", + "type": [ + "integer", + "null" + ] } }, "type": "object" diff --git a/fingerly-server-jakarta/src/main/java/io/fingerly/server/jakarta/FingerlyProxyFilter.java b/fingerly-server-jakarta/src/main/java/io/fingerly/server/jakarta/FingerlyProxyFilter.java index f0b11ba..173ddf6 100644 --- a/fingerly-server-jakarta/src/main/java/io/fingerly/server/jakarta/FingerlyProxyFilter.java +++ b/fingerly-server-jakarta/src/main/java/io/fingerly/server/jakarta/FingerlyProxyFilter.java @@ -20,7 +20,7 @@ import java.util.concurrent.CompletableFuture; /** - * Serves the browser SDK's same-origin requests under Jakarta Servlet. + * Serves Fingerly Web and native client SDK first-party requests under Jakarta Servlet. * *

For Spring Boot 3, Tomcat 10 and newer, Jetty 11 and newer, and anything * else on {@code jakarta.servlet}. Spring Boot 2 and older containers want @@ -142,8 +142,11 @@ private static ProxyRequest toProxyRequest(HttpServletRequest http) throws IOExc .contentLength(http.getContentLengthLong()) .remoteAddress(http.getRemoteAddr()); - for (String name : Collections.list(http.getHeaderNames())) { - builder.headers(name, Collections.list(http.getHeaders(name))); + java.util.Enumeration headerNames = http.getHeaderNames(); + if (headerNames != null) { + for (String name : Collections.list(headerNames)) { + builder.headers(name, Collections.list(http.getHeaders(name))); + } } return builder.build(); } @@ -158,7 +161,10 @@ private static void write(HttpServletResponse out, ProxyResponse result) throws for (Map.Entry header : result.headers()) { out.addHeader(header.getKey(), header.getValue()); } - out.setContentLength(result.contentLength()); - out.getOutputStream().write(result.body()); + int status = result.status(); + if (status != 204 && status != 205 && status != 304) { + out.setContentLength(result.contentLength()); + out.getOutputStream().write(result.body()); + } } } diff --git a/fingerly-server-jakarta/src/test/java/io/fingerly/server/jakarta/FingerlyProxyFilterTest.java b/fingerly-server-jakarta/src/test/java/io/fingerly/server/jakarta/FingerlyProxyFilterTest.java index 955d46c..a951989 100644 --- a/fingerly-server-jakarta/src/test/java/io/fingerly/server/jakarta/FingerlyProxyFilterTest.java +++ b/fingerly-server-jakarta/src/test/java/io/fingerly/server/jakarta/FingerlyProxyFilterTest.java @@ -170,13 +170,14 @@ private static FingerlyProxy proxy(String upstream) { return FingerlyProxy.builder() .proxyKey(PROXY_KEY) .path(PATH) + .allowInsecureLoopback(true) .upstream(upstream) .clientIp(request -> request.header("CF-Connecting-IP")) .build(); } @ParameterizedTest - @ValueSource(strings = {"/orders", "/k7x2qm4a/admin"}) + @ValueSource(strings = {"/orders", "/k7x2qm4a/admin", "/k7x2qm4a/supplement"}) void passesOtherPathsAlongWithTheBodyIntact(String path) throws Exception { byte[] body = "{\"order\":1}".getBytes(StandardCharsets.UTF_8); HttpServletRequest request = @@ -216,6 +217,22 @@ void refusesAWrongMethodOnAnOwnedPathItself() throws Exception { assertEquals(List.of("POST"), written.headers.get("Allow")); } + @ParameterizedTest + @ValueSource(strings = {"post", "Post", "PUT"}) + void refusesAMethodThatIsNotExactlyPostUncacheable(String method) throws Exception { + HttpServletRequest request = request(method, PATH, Map.of(), new byte[0]); + Written written = new Written(); + List passedAlong = new ArrayList<>(); + + new FingerlyProxyFilter(proxy("https://api.fingerly.io")) + .doFilter(request, response(written), + (passed, response) -> passedAlong.add(passed)); + + assertTrue(passedAlong.isEmpty()); + assertEquals(405, written.status, method); + assertEquals(List.of("no-store"), written.headers.get("Cache-Control")); + } + @Test void answersEverythingMappedToItWithoutAPath() throws Exception { // Mapped to the one path by the container, the proxy needs no path. @@ -265,7 +282,10 @@ void readsHeadersWithoutRegardToCase() throws Exception { // The filter copies whatever names the container reports; the core // matches them case-insensitively. HttpServletRequest request = request("POST", PATH, - Map.of("x-api-key", PUBLIC_KEY, "cf-connecting-ip", "203.0.113.5"), + Map.of( + "x-api-key", PUBLIC_KEY, + "origin", "https://shop.test", + "cf-connecting-ip", "203.0.113.5"), "{}".getBytes(StandardCharsets.UTF_8)); Written written = new Written(); @@ -276,19 +296,49 @@ void readsHeadersWithoutRegardToCase() throws Exception { assertEquals(502, written.status); } + @Test + void rejectsOriginlessPostAndAllowsValidatedNativePlatform() throws Exception { + HttpServletRequest originless = request("POST", PATH, + Map.of("X-API-Key", PUBLIC_KEY, "CF-Connecting-IP", "203.0.113.5"), + "{}".getBytes(StandardCharsets.UTF_8)); + Written refused = new Written(); + + new FingerlyProxyFilter(proxy("http://127.0.0.1:1")) + .doFilter(originless, response(refused), (passed, response) -> {}); + + assertEquals(400, refused.status); + assertEquals(List.of("no-store"), refused.headers.get("Cache-Control")); + assertTrue(refused.body.toString(StandardCharsets.UTF_8).contains("missing_origin")); + + HttpServletRequest nativeReq = request("POST", PATH, + Map.of( + "X-API-Key", PUBLIC_KEY, + "X-Fingerly-SDK-Platform", "ios", + "CF-Connecting-IP", "203.0.113.5"), + "{}".getBytes(StandardCharsets.UTF_8)); + Written forwarded = new Written(); + + new FingerlyProxyFilter(proxy("http://127.0.0.1:1")) + .doFilter(nativeReq, response(forwarded), (passed, response) -> {}); + + assertEquals(502, forwarded.status); + } + @Test void givesTheCallbackThePeerAddress() throws Exception { List seen = new ArrayList<>(); FingerlyProxy proxy = FingerlyProxy.builder() .proxyKey(PROXY_KEY) .path(PATH) + .allowInsecureLoopback(true) .upstream("http://127.0.0.1:1") .clientIp(request -> { seen.add(request.remoteAddress()); return request.remoteAddress(); }) .build(); - HttpServletRequest request = request("POST", PATH, Map.of("X-API-Key", PUBLIC_KEY), + HttpServletRequest request = request("POST", PATH, + Map.of("X-API-Key", PUBLIC_KEY, "Origin", "https://shop.test"), "{}".getBytes(StandardCharsets.UTF_8)); new FingerlyProxyFilter(proxy).doFilter(request, response(new Written()), (p, r) -> {}); @@ -363,7 +413,10 @@ private static com.sun.net.httpserver.HttpServer upstream(boolean trickle, } private static Map browser() { - return Map.of("X-API-Key", PUBLIC_KEY, "CF-Connecting-IP", "203.0.113.5"); + return Map.of( + "X-API-Key", PUBLIC_KEY, + "Origin", "https://shop.test", + "CF-Connecting-IP", "203.0.113.5"); } @Test diff --git a/fingerly-server-javax/src/main/java/io/fingerly/server/javaxservlet/FingerlyProxyFilter.java b/fingerly-server-javax/src/main/java/io/fingerly/server/javaxservlet/FingerlyProxyFilter.java index 6b01791..8b449dd 100644 --- a/fingerly-server-javax/src/main/java/io/fingerly/server/javaxservlet/FingerlyProxyFilter.java +++ b/fingerly-server-javax/src/main/java/io/fingerly/server/javaxservlet/FingerlyProxyFilter.java @@ -20,7 +20,7 @@ import java.util.concurrent.CompletableFuture; /** - * Serves the browser SDK's same-origin requests under the older Servlet API. + * Serves Fingerly Web and native client SDK first-party requests under the older Servlet API. * *

For Spring Boot 2, Tomcat 9 and older, and anything else still on * {@code javax.servlet}. Spring Boot 3 and newer containers want @@ -146,8 +146,11 @@ private static ProxyRequest toProxyRequest(HttpServletRequest http) throws IOExc .contentLength(http.getContentLengthLong()) .remoteAddress(http.getRemoteAddr()); - for (String name : Collections.list(http.getHeaderNames())) { - builder.headers(name, Collections.list(http.getHeaders(name))); + java.util.Enumeration headerNames = http.getHeaderNames(); + if (headerNames != null) { + for (String name : Collections.list(headerNames)) { + builder.headers(name, Collections.list(http.getHeaders(name))); + } } return builder.build(); } @@ -162,7 +165,10 @@ private static void write(HttpServletResponse out, ProxyResponse result) throws for (Map.Entry header : result.headers()) { out.addHeader(header.getKey(), header.getValue()); } - out.setContentLength(result.contentLength()); - out.getOutputStream().write(result.body()); + int status = result.status(); + if (status != 204 && status != 205 && status != 304) { + out.setContentLength(result.contentLength()); + out.getOutputStream().write(result.body()); + } } } diff --git a/fingerly-server-javax/src/test/java/io/fingerly/server/javaxservlet/FingerlyProxyFilterTest.java b/fingerly-server-javax/src/test/java/io/fingerly/server/javaxservlet/FingerlyProxyFilterTest.java index d7d86a3..165fe4d 100644 --- a/fingerly-server-javax/src/test/java/io/fingerly/server/javaxservlet/FingerlyProxyFilterTest.java +++ b/fingerly-server-javax/src/test/java/io/fingerly/server/javaxservlet/FingerlyProxyFilterTest.java @@ -170,13 +170,14 @@ private static FingerlyProxy proxy(String upstream) { return FingerlyProxy.builder() .proxyKey(PROXY_KEY) .path(PATH) + .allowInsecureLoopback(true) .upstream(upstream) .clientIp(request -> request.header("CF-Connecting-IP")) .build(); } @ParameterizedTest - @ValueSource(strings = {"/orders", "/k7x2qm4a/admin"}) + @ValueSource(strings = {"/orders", "/k7x2qm4a/admin", "/k7x2qm4a/supplement"}) void passesOtherPathsAlongWithTheBodyIntact(String path) throws Exception { byte[] body = "{\"order\":1}".getBytes(StandardCharsets.UTF_8); HttpServletRequest request = @@ -216,6 +217,22 @@ void refusesAWrongMethodOnAnOwnedPathItself() throws Exception { assertEquals(List.of("POST"), written.headers.get("Allow")); } + @ParameterizedTest + @ValueSource(strings = {"post", "Post", "PUT"}) + void refusesAMethodThatIsNotExactlyPostUncacheable(String method) throws Exception { + HttpServletRequest request = request(method, PATH, Map.of(), new byte[0]); + Written written = new Written(); + List passedAlong = new ArrayList<>(); + + new FingerlyProxyFilter(proxy("https://api.fingerly.io")) + .doFilter(request, response(written), + (passed, response) -> passedAlong.add(passed)); + + assertTrue(passedAlong.isEmpty()); + assertEquals(405, written.status, method); + assertEquals(List.of("no-store"), written.headers.get("Cache-Control")); + } + @Test void answersEverythingMappedToItWithoutAPath() throws Exception { // Mapped to the one path by the container, the proxy needs no path. @@ -265,7 +282,10 @@ void readsHeadersWithoutRegardToCase() throws Exception { // The filter copies whatever names the container reports; the core // matches them case-insensitively. HttpServletRequest request = request("POST", PATH, - Map.of("x-api-key", PUBLIC_KEY, "cf-connecting-ip", "203.0.113.5"), + Map.of( + "x-api-key", PUBLIC_KEY, + "origin", "https://shop.test", + "cf-connecting-ip", "203.0.113.5"), "{}".getBytes(StandardCharsets.UTF_8)); Written written = new Written(); @@ -276,19 +296,49 @@ void readsHeadersWithoutRegardToCase() throws Exception { assertEquals(502, written.status); } + @Test + void rejectsOriginlessPostAndAllowsValidatedNativePlatform() throws Exception { + HttpServletRequest originless = request("POST", PATH, + Map.of("X-API-Key", PUBLIC_KEY, "CF-Connecting-IP", "203.0.113.5"), + "{}".getBytes(StandardCharsets.UTF_8)); + Written refused = new Written(); + + new FingerlyProxyFilter(proxy("http://127.0.0.1:1")) + .doFilter(originless, response(refused), (passed, response) -> {}); + + assertEquals(400, refused.status); + assertEquals(List.of("no-store"), refused.headers.get("Cache-Control")); + assertTrue(refused.body.toString(StandardCharsets.UTF_8).contains("missing_origin")); + + HttpServletRequest nativeReq = request("POST", PATH, + Map.of( + "X-API-Key", PUBLIC_KEY, + "X-Fingerly-SDK-Platform", "android", + "CF-Connecting-IP", "203.0.113.5"), + "{}".getBytes(StandardCharsets.UTF_8)); + Written forwarded = new Written(); + + new FingerlyProxyFilter(proxy("http://127.0.0.1:1")) + .doFilter(nativeReq, response(forwarded), (passed, response) -> {}); + + assertEquals(502, forwarded.status); + } + @Test void givesTheCallbackThePeerAddress() throws Exception { List seen = new ArrayList<>(); FingerlyProxy proxy = FingerlyProxy.builder() .proxyKey(PROXY_KEY) .path(PATH) + .allowInsecureLoopback(true) .upstream("http://127.0.0.1:1") .clientIp(request -> { seen.add(request.remoteAddress()); return request.remoteAddress(); }) .build(); - HttpServletRequest request = request("POST", PATH, Map.of("X-API-Key", PUBLIC_KEY), + HttpServletRequest request = request("POST", PATH, + Map.of("X-API-Key", PUBLIC_KEY, "Origin", "https://shop.test"), "{}".getBytes(StandardCharsets.UTF_8)); new FingerlyProxyFilter(proxy).doFilter(request, response(new Written()), (p, r) -> {}); @@ -363,7 +413,10 @@ private static com.sun.net.httpserver.HttpServer upstream(boolean trickle, } private static Map browser() { - return Map.of("X-API-Key", PUBLIC_KEY, "CF-Connecting-IP", "203.0.113.5"); + return Map.of( + "X-API-Key", PUBLIC_KEY, + "Origin", "https://shop.test", + "CF-Connecting-IP", "203.0.113.5"); } @Test diff --git a/fingerly-server-webflux/src/main/java/io/fingerly/server/webflux/FingerlyProxyWebFilter.java b/fingerly-server-webflux/src/main/java/io/fingerly/server/webflux/FingerlyProxyWebFilter.java index fa9c002..e901a3f 100644 --- a/fingerly-server-webflux/src/main/java/io/fingerly/server/webflux/FingerlyProxyWebFilter.java +++ b/fingerly-server-webflux/src/main/java/io/fingerly/server/webflux/FingerlyProxyWebFilter.java @@ -24,7 +24,7 @@ import reactor.core.scheduler.Schedulers; /** - * Serves the browser SDK's same-origin requests under reactive Spring. + * Serves Fingerly Web and native client SDK first-party requests under reactive Spring. * *

Compiled against Spring 5.3 and tested against 5.3 and 7, so one artifact * serves Spring Boot 2, 3 and 4. @@ -123,6 +123,10 @@ private static ProxyRequest toProxyRequest(ServerHttpRequest request, byte[] bod .path(request.getPath().value()) .remoteAddress(peer(request)) .body(body); + long declared = request.getHeaders().getContentLength(); + if (declared >= 0) { + builder.contentLength(declared); + } request.getHeaders().forEach(builder::headers); return builder.build(); } diff --git a/fingerly-server-webflux/src/test/java/io/fingerly/server/webflux/FingerlyProxyWebFilterTest.java b/fingerly-server-webflux/src/test/java/io/fingerly/server/webflux/FingerlyProxyWebFilterTest.java index 92b20ee..97533b5 100644 --- a/fingerly-server-webflux/src/test/java/io/fingerly/server/webflux/FingerlyProxyWebFilterTest.java +++ b/fingerly-server-webflux/src/test/java/io/fingerly/server/webflux/FingerlyProxyWebFilterTest.java @@ -28,6 +28,7 @@ private static FingerlyProxy proxy(String upstream) { return FingerlyProxy.builder() .proxyKey(PROXY_KEY) .path(PATH) + .allowInsecureLoopback(true) .upstream(upstream) .clientIp(request -> request.header("CF-Connecting-IP")) .build(); @@ -72,7 +73,7 @@ void refusesAProxyWithoutAPath() { } @ParameterizedTest - @ValueSource(strings = {"/orders", "/k7x2qm4a/admin"}) + @ValueSource(strings = {"/orders", "/k7x2qm4a/admin", "/k7x2qm4a/supplement"}) void passesOtherPathsAlongWithTheBodyIntact(String path) { MockServerWebExchange exchange = MockServerWebExchange.from( MockServerHttpRequest.post(path).body("{\"order\":1}")); @@ -103,6 +104,22 @@ void refusesAWrongMethodOnAnOwnedPathItself() { assertFalse(reachedApp.get(), "an owned path must not reach the application"); assertEquals(405, exchange.getResponse().getStatusCode().value()); assertEquals("POST", exchange.getResponse().getHeaders().getFirst("Allow")); + assertEquals("no-store", exchange.getResponse().getHeaders().getFirst("Cache-Control")); + } + + // Spring 5.3 represents only the standard upper-case methods, so a + // lower-case "post" never reaches this adapter there; the core refuses it. + @ParameterizedTest + @ValueSource(strings = {"PUT", "DELETE", "PATCH"}) + void refusesAMethodThatIsNotExactlyPostUncacheable(String method) { + MockServerWebExchange exchange = MockServerWebExchange.from( + MockServerHttpRequest.method(HttpMethod.valueOf(method), PATH).body("{}")); + + StepVerifier.create(filter("https://api.fingerly.io") + .filter(exchange, passed -> Mono.empty())).verifyComplete(); + + assertEquals(405, exchange.getResponse().getStatusCode().value(), method); + assertEquals("no-store", exchange.getResponse().getHeaders().getFirst("Cache-Control")); } @Test @@ -124,6 +141,7 @@ void refusesABodyPastTheLimitWhileItStreams() { FingerlyProxy proxy = FingerlyProxy.builder() .proxyKey(PROXY_KEY) .path(PATH) + .allowInsecureLoopback(true) .upstream("https://api.fingerly.io") .maxBodyBytes(8) .clientIp(request -> "203.0.113.5") @@ -144,6 +162,7 @@ void reportsAnUnreachableUpstreamAsBadGateway() { MockServerWebExchange exchange = MockServerWebExchange.from( MockServerHttpRequest.method(HttpMethod.POST, PATH) .header("X-API-Key", PUBLIC_KEY) + .header("Origin", "https://shop.test") .header("CF-Connecting-IP", "203.0.113.5") .body("{}")); @@ -153,6 +172,36 @@ void reportsAnUnreachableUpstreamAsBadGateway() { assertEquals(502, exchange.getResponse().getStatusCode().value()); } + @Test + void rejectsOriginlessPostAndAllowsValidatedNativePlatform() { + MockServerWebExchange originless = MockServerWebExchange.from( + MockServerHttpRequest.method(HttpMethod.POST, PATH) + .header("X-API-Key", PUBLIC_KEY) + .header("CF-Connecting-IP", "203.0.113.5") + .body("{}")); + + StepVerifier.create(filter("http://127.0.0.1:1") + .filter(originless, passed -> Mono.empty())).verifyComplete(); + + assertEquals(400, originless.getResponse().getStatusCode().value()); + assertEquals("no-store", + originless.getResponse().getHeaders().getFirst("Cache-Control")); + assertTrue(originless.getResponse().getBodyAsString().block() + .contains("missing_origin")); + + MockServerWebExchange nativeReq = MockServerWebExchange.from( + MockServerHttpRequest.method(HttpMethod.POST, PATH) + .header("X-API-Key", PUBLIC_KEY) + .header("X-Fingerly-SDK-Platform", "flutter-ios") + .header("CF-Connecting-IP", "203.0.113.5") + .body("{}")); + + StepVerifier.create(filter("http://127.0.0.1:1") + .filter(nativeReq, passed -> Mono.empty())).verifyComplete(); + + assertEquals(502, nativeReq.getResponse().getStatusCode().value()); + } + @ParameterizedTest @ValueSource(strings = {"/k7x2qm4a/", "/K7X2QM4A", "/k7x2qm4a%2F", "/x/../k7x2qm4a"}) void passesNearMissesToTheApplication(String path) { @@ -177,6 +226,7 @@ void givesTheCallbackThePeerAddress() { FingerlyProxy proxy = FingerlyProxy.builder() .proxyKey(PROXY_KEY) .path(PATH) + .allowInsecureLoopback(true) .upstream("http://127.0.0.1:1") .clientIp(request -> { seen.add(request.remoteAddress()); @@ -187,6 +237,7 @@ void givesTheCallbackThePeerAddress() { MockServerHttpRequest.method(HttpMethod.POST, PATH) .remoteAddress(new java.net.InetSocketAddress("198.51.100.7", 40000)) .header("X-API-Key", PUBLIC_KEY) + .header("Origin", "https://shop.test") .body("{}")); StepVerifier.create(new FingerlyProxyWebFilter(proxy) @@ -228,6 +279,7 @@ void neitherBlocksNorOutlivesAVisitorWhoLeaves() throws Exception { MockServerWebExchange exchange = MockServerWebExchange.from( MockServerHttpRequest.method(HttpMethod.POST, PATH) .header("X-API-Key", PUBLIC_KEY) + .header("Origin", "https://shop.test") .header("CF-Connecting-IP", "203.0.113.5") .body("{}")); diff --git a/fingerly-server/src/main/java/io/fingerly/server/ConsumeResult.java b/fingerly-server/src/main/java/io/fingerly/server/ConsumeResult.java new file mode 100644 index 0000000..ab67e60 --- /dev/null +++ b/fingerly-server/src/main/java/io/fingerly/server/ConsumeResult.java @@ -0,0 +1,45 @@ +package io.fingerly.server; + +import com.fasterxml.jackson.databind.JsonNode; + +/** The answer to the first consume of an event. */ +public final class ConsumeResult { + + private final String requestId; + private final String consumedAt; + private final boolean first; + + private ConsumeResult(String requestId, String consumedAt, boolean first) { + this.requestId = requestId; + this.consumedAt = consumedAt; + this.first = first; + } + + static ConsumeResult from(JsonNode tree) { + return new ConsumeResult( + tree.path("request_id").isTextual() ? tree.path("request_id").asText() : null, + tree.path("consumed_at").isTextual() ? tree.path("consumed_at").asText() : null, + tree.path("first").asBoolean(false)); + } + + /** The consumed event's request id. */ + public String getRequestId() { + return requestId; + } + + /** When the event was consumed, an RFC 3339 timestamp as the API sent it. */ + public String getConsumedAt() { + return consumedAt; + } + + /** True when this call consumed the event. */ + public boolean isFirst() { + return first; + } + + @Override + public String toString() { + return "ConsumeResult[requestId=" + requestId + ", consumedAt=" + consumedAt + + ", first=" + first + "]"; + } +} diff --git a/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckBits.java b/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckBits.java new file mode 100644 index 0000000..5033b93 --- /dev/null +++ b/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckBits.java @@ -0,0 +1,59 @@ +package io.fingerly.server; + +import com.fasterxml.jackson.databind.JsonNode; +import java.util.Optional; + +/** + * The two Apple DeviceCheck bits of an event's device, as Apple holds them. + * + *

The bits mean what you decide; Fingerly never reads or keeps them. + */ +public final class DeviceCheckBits { + + private final boolean found; + private final boolean bit0; + private final boolean bit1; + private final String lastUpdate; + + private DeviceCheckBits(boolean found, boolean bit0, boolean bit1, String lastUpdate) { + this.found = found; + this.bit0 = bit0; + this.bit1 = bit1; + this.lastUpdate = lastUpdate; + } + + static DeviceCheckBits from(JsonNode tree) { + JsonNode lastUpdate = tree.path("last_update"); + return new DeviceCheckBits( + tree.path("found").asBoolean(false), + tree.path("bit0").asBoolean(false), + tree.path("bit1").asBoolean(false), + lastUpdate.isTextual() && !lastUpdate.asText().isEmpty() ? lastUpdate.asText() : null); + } + + /** False when Apple holds no bits for the device yet; both bits are then false. */ + public boolean isFound() { + return found; + } + + /** The first bit. */ + public boolean getBit0() { + return bit0; + } + + /** The second bit. */ + public boolean getBit1() { + return bit1; + } + + /** When the bits last changed, as {@code YYYY-MM}; empty when Apple holds none. */ + public Optional getLastUpdate() { + return Optional.ofNullable(lastUpdate); + } + + @Override + public String toString() { + return "DeviceCheckBits[found=" + found + ", bit0=" + bit0 + ", bit1=" + bit1 + + ", lastUpdate=" + lastUpdate + "]"; + } +} diff --git a/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckService.java b/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckService.java new file mode 100644 index 0000000..a770621 --- /dev/null +++ b/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckService.java @@ -0,0 +1,154 @@ +package io.fingerly.server; + +import java.net.URLEncoder; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionException; +import java.util.concurrent.Future; +import java.util.concurrent.atomic.AtomicReference; + +/** + * Spends the Apple DeviceCheck token held for an iOS event, once, to read or + * to write the device's two bits. Reached as {@code fingerly.deviceCheck()}. + * + *

Apple treats a DeviceCheck token as single-use, so a token spent on a + * read cannot also write: read on the identification that asks, for example, + * for a free trial, record the grant in your own data, and write on the + * device's next identification. + * + *

A call is retried, with the identical request, only after a + * {@code 503} or a transport failure, up to twice. The API keeps the token + * only for a retry of the same request: another operation or other bits + * answer {@code 409 device_check_spent}. + */ +public final class DeviceCheckService { + + /** The waits before the second and third attempts. */ + private static final Duration[] RETRY_DELAYS = {Duration.ofMillis(250), Duration.ofSeconds(1)}; + + /** The longest {@code Retry-After} a retry waits for before giving up. */ + private static final int MAX_RETRY_AFTER_SECONDS = 5; + + private final FingerlyClient client; + + DeviceCheckService(FingerlyClient client) { + this.client = client; + } + + /** + * Reads the device's two bits from Apple, spending the event's token. + * + * @param requestId the {@code request_id} of an iOS identification made + * within the last hour + * @throws FingerlyApiException on any non-2xx answer; branch on + * {@link FingerlyApiException#isDeviceCheckNotHeld()}, + * {@link FingerlyApiException#isDeviceCheckSpent()} and the others + * @throws FingerlyTimeoutException if every attempt exceeds the client timeout + * @throws IllegalArgumentException if requestId is null, empty, "." or ".." + */ + public DeviceCheckBits query(String requestId) { + return client.await(queryAsync(requestId)); + } + + /** {@link #query} without holding a thread. */ + public CompletableFuture queryAsync(String requestId) { + String path = path(requestId) + "/query"; + return withRetries("POST", path, null) + .thenApply(body -> DeviceCheckBits.from(Json.tree(body))); + } + + /** + * Sets {@code bit0}, {@code bit1} or both with Apple, spending the event's + * token. A bit left out keeps its value. + * + * @param requestId the {@code request_id} of an iOS identification made + * within the last hour + * @param bits the bits to set, from {@link DeviceCheckWrite#builder()} + * @throws FingerlyApiException on any non-2xx answer + * @throws IllegalArgumentException if requestId is invalid or bits is null + */ + public DeviceCheckWriteResult write(String requestId, DeviceCheckWrite bits) { + return client.await(writeAsync(requestId, bits)); + } + + /** {@link #write} without holding a thread. */ + public CompletableFuture writeAsync(String requestId, DeviceCheckWrite bits) { + if (bits == null) { + throw new IllegalArgumentException("bits is required"); + } + String path = path(requestId); + return withRetries("PUT", path, bits.body()) + .thenApply(body -> DeviceCheckWriteResult.from(Json.tree(body))); + } + + /** Sends the same request again after a 503 or a transport failure. */ + private CompletableFuture withRetries(String method, String path, byte[] body) { + CompletableFuture result = new CompletableFuture<>(); + AtomicReference> current = new AtomicReference<>(); + attempt(method, path, body, 0, result, current); + result.whenComplete((ignored, failure) -> { + if (result.isCancelled()) { + Future inFlight = current.get(); + if (inFlight != null) { + inFlight.cancel(true); + } + } + }); + return result; + } + + private void attempt(String method, String path, byte[] body, int retries, + CompletableFuture result, AtomicReference> current) { + if (result.isDone()) { + return; + } + CompletableFuture pending = client.sendAsync(method, path, body); + current.set(pending); + pending.whenComplete((answer, failure) -> { + if (failure == null) { + result.complete(answer); + return; + } + Throwable cause = failure instanceof CompletionException && failure.getCause() != null + ? failure.getCause() + : failure; + Duration wait = retries < RETRY_DELAYS.length ? retryDelay(cause, retries) : null; + if (wait == null) { + result.completeExceptionally(cause); + return; + } + current.set(Deadlines.schedule( + () -> attempt(method, path, body, retries + 1, result, current), wait)); + }); + } + + /** The wait before retrying after {@code cause}, or null when it is final. */ + private static Duration retryDelay(Throwable cause, int retries) { + if (cause instanceof FingerlyConnectionException || cause instanceof FingerlyTimeoutException) { + return RETRY_DELAYS[retries]; + } + if (cause instanceof FingerlyApiException && ((FingerlyApiException) cause).getStatus() == 503) { + Integer after = ((FingerlyApiException) cause).getRetryAfter(); + if (after == null) { + return RETRY_DELAYS[retries]; + } + return after <= MAX_RETRY_AFTER_SECONDS ? Duration.ofSeconds(after) : null; + } + return null; + } + + private static String path(String requestId) { + if (requestId == null || requestId.isEmpty() + || requestId.equals(".") || requestId.equals("..")) { + throw new IllegalArgumentException("requestId is required"); + } + String segment = URLEncoder.encode(requestId, StandardCharsets.UTF_8).replace("+", "%20"); + return "/events/" + segment + "/device-check"; + } + + @Override + public String toString() { + return "DeviceCheckService[" + client + "]"; + } +} diff --git a/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckWrite.java b/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckWrite.java new file mode 100644 index 0000000..9103d80 --- /dev/null +++ b/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckWrite.java @@ -0,0 +1,86 @@ +package io.fingerly.server; + +import java.nio.charset.StandardCharsets; + +/** + * The DeviceCheck bits to set. A bit left out keeps its value; a write names + * at least one. + * + *

{@code
+ * DeviceCheckWrite.builder().bit0(true).build()
+ * }
+ */ +public final class DeviceCheckWrite { + + private final Boolean bit0; + private final Boolean bit1; + + private DeviceCheckWrite(Builder builder) { + this.bit0 = builder.bit0; + this.bit1 = builder.bit1; + } + + /** A new builder. */ + public static Builder builder() { + return new Builder(); + } + + /** The value to set for the first bit, or null to keep it. */ + public Boolean getBit0() { + return bit0; + } + + /** The value to set for the second bit, or null to keep it. */ + public Boolean getBit1() { + return bit1; + } + + /** The request body, naming only the bits that are set. */ + byte[] body() { + StringBuilder json = new StringBuilder("{"); + if (bit0 != null) { + json.append("\"bit0\":").append(bit0); + } + if (bit1 != null) { + json.append(bit0 != null ? "," : "").append("\"bit1\":").append(bit1); + } + return json.append('}').toString().getBytes(StandardCharsets.UTF_8); + } + + @Override + public String toString() { + return "DeviceCheckWrite[bit0=" + bit0 + ", bit1=" + bit1 + "]"; + } + + /** Builds a {@link DeviceCheckWrite}. */ + public static final class Builder { + private Boolean bit0; + private Boolean bit1; + + private Builder() {} + + /** Sets the first bit. */ + public Builder bit0(boolean value) { + this.bit0 = value; + return this; + } + + /** Sets the second bit. */ + public Builder bit1(boolean value) { + this.bit1 = value; + return this; + } + + /** + * Builds the write. + * + * @throws IllegalArgumentException if neither bit is set + */ + public DeviceCheckWrite build() { + if (bit0 == null && bit1 == null) { + throw new IllegalArgumentException("name bit0, bit1 or both"); + } + return new DeviceCheckWrite(this); + } + } +} diff --git a/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckWriteResult.java b/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckWriteResult.java new file mode 100644 index 0000000..d6b2969 --- /dev/null +++ b/fingerly-server/src/main/java/io/fingerly/server/DeviceCheckWriteResult.java @@ -0,0 +1,39 @@ +package io.fingerly.server; + +import com.fasterxml.jackson.databind.JsonNode; +import java.util.Optional; + +/** The DeviceCheck bits Apple accepted on a write. */ +public final class DeviceCheckWriteResult { + + private final Boolean bit0; + private final Boolean bit1; + + private DeviceCheckWriteResult(Boolean bit0, Boolean bit1) { + this.bit0 = bit0; + this.bit1 = bit1; + } + + static DeviceCheckWriteResult from(JsonNode tree) { + return new DeviceCheckWriteResult(bit(tree.path("bit0")), bit(tree.path("bit1"))); + } + + private static Boolean bit(JsonNode node) { + return node.isBoolean() ? node.asBoolean() : null; + } + + /** The value set for the first bit; empty when the write left it out. */ + public Optional getBit0() { + return Optional.ofNullable(bit0); + } + + /** The value set for the second bit; empty when the write left it out. */ + public Optional getBit1() { + return Optional.ofNullable(bit1); + } + + @Override + public String toString() { + return "DeviceCheckWriteResult[bit0=" + bit0 + ", bit1=" + bit1 + "]"; + } +} diff --git a/fingerly-server/src/main/java/io/fingerly/server/Event.java b/fingerly-server/src/main/java/io/fingerly/server/Event.java index f9ca36b..f369c4e 100644 --- a/fingerly-server/src/main/java/io/fingerly/server/Event.java +++ b/fingerly-server/src/main/java/io/fingerly/server/Event.java @@ -28,6 +28,7 @@ public final class Event { private Instant occurredAt; private String state; + private String reason; private String environment; private String platform; @@ -110,7 +111,10 @@ public String getRequestId() { return requestId; } - /** The visitor the device was identified as. */ + /** + * The visitor the device was identified as, or {@code ""} when the + * submission was unidentifiable. + */ public String getVisitorId() { return visitorId; } @@ -125,6 +129,11 @@ public String getState() { return state; } + /** Why enrichment did not happen, on an unavailable event. Null otherwise. */ + public String getReason() { + return reason; + } + /** {@code production}, {@code staging} or {@code development}. */ public String getEnvironment() { return environment; diff --git a/fingerly-server/src/main/java/io/fingerly/server/EventAlreadyConsumedException.java b/fingerly-server/src/main/java/io/fingerly/server/EventAlreadyConsumedException.java new file mode 100644 index 0000000..cefcf11 --- /dev/null +++ b/fingerly-server/src/main/java/io/fingerly/server/EventAlreadyConsumedException.java @@ -0,0 +1,30 @@ +package io.fingerly.server; + +import com.fasterxml.jackson.databind.JsonNode; + +/** + * Thrown by {@link EventsService#consume} when the event was already + * consumed: {@code 409 event_already_consumed}. Another call spent the event + * first, so it must not back a second decision. + */ +public final class EventAlreadyConsumedException extends FingerlyApiException { + + private static final long serialVersionUID = 1L; + + private final String consumedAt; + + EventAlreadyConsumedException( + int status, String code, JsonNode details, String requestId, Integer retryAfter, + String consumedAt) { + super(status, code, details, requestId, retryAfter); + this.consumedAt = consumedAt; + } + + /** + * When the event was first consumed, an RFC 3339 timestamp as the API + * sent it, or null when the answer carried none. + */ + public String getConsumedAt() { + return consumedAt; + } +} diff --git a/fingerly-server/src/main/java/io/fingerly/server/EventTrigger.java b/fingerly-server/src/main/java/io/fingerly/server/EventTrigger.java index a63a0f3..18fd89e 100644 --- a/fingerly-server/src/main/java/io/fingerly/server/EventTrigger.java +++ b/fingerly-server/src/main/java/io/fingerly/server/EventTrigger.java @@ -1,19 +1,80 @@ package io.fingerly.server; +import java.util.Set; + /** One signal group that contributed to an event's suspect score. */ public final class EventTrigger { + /** Hardware attestation failed cryptographic, chain, replay or ownership verification. */ + public static final String SIGNAL_ATTESTATION_FAILED = "attestation_failed"; + + /** Native device could have offered hardware attestation proof and did not. */ + public static final String SIGNAL_ATTESTATION_MISSING = "attestation_missing"; + + /** Unproven or rejected native report naming an install an attested key owns. */ + public static final String SIGNAL_ATTESTATION_DOWNGRADE = "attestation_downgrade"; + + /** Reported hardware, OS or runtime realms contradict each other. */ + public static final String SIGNAL_DEVICE_INCONSISTENT = "device_inconsistent"; + + /** Evidence a report always carries is missing: a required native security source or the web worker-realm cross-check. */ + public static final String SIGNAL_REPORT_INCOMPLETE = "report_incomplete"; + + /** Native security source ran but lists probes that threw, were denied or timed out. */ + public static final String SIGNAL_PROBE_ERRORS = "probe_errors"; + + /** Browser report without {@code Sec-Fetch-*} headers from a browser that always sends them. */ + public static final String SIGNAL_FETCH_METADATA_MISSING = "fetch_metadata_missing"; + + /** Browser report that carried no single-use freshness ticket. */ + public static final String SIGNAL_WEB_TICKET_MISSING = "web_ticket_missing"; + + /** Browser report whose freshness ticket did not verify: bad signature, another key's or origin's, expired, or already used. */ + public static final String SIGNAL_WEB_TICKET_INVALID = "web_ticket_invalid"; + + /** Runtime code or natives were patched or tampered with. */ + public static final String SIGNAL_TAMPERING = "tampering"; + + /** Browser tampering signal group key on stored event rows. */ + public static final String SIGNAL_BROWSER_TAMPERING = "browser_tampering"; + + /** TLS chain anchored at a non-WebPKI or user-installed root authority. */ + public static final String SIGNAL_CUSTOM_ROOT_CA = "custom_root_ca"; + + /** Managed Android work profile or secondary OS user profile without cloner indicators. */ + public static final String SIGNAL_SECONDARY_USER_PROFILE = "secondary_user_profile"; + + /** The v3 anti-spoofing signal identifiers. */ + public static final Set V3_SIGNALS = Set.of( + SIGNAL_ATTESTATION_FAILED, + SIGNAL_ATTESTATION_DOWNGRADE, + SIGNAL_DEVICE_INCONSISTENT, + SIGNAL_REPORT_INCOMPLETE, + SIGNAL_PROBE_ERRORS, + SIGNAL_FETCH_METADATA_MISSING, + SIGNAL_WEB_TICKET_MISSING, + SIGNAL_WEB_TICKET_INVALID, + SIGNAL_TAMPERING, + SIGNAL_CUSTOM_ROOT_CA, + SIGNAL_SECONDARY_USER_PROFILE); + private String signal; + private String group; private int weight; private String confidence; EventTrigger() {} - /** The signal group, such as {@code vpn} or {@code bot}. */ + /** The signal or signal group, such as {@code vpn}, {@code tampering} or {@code attestation_failed}. */ public String getSignal() { return signal; } + /** Optional group identifier when returned alongside a raw signal. */ + public String getGroup() { + return group; + } + /** The group's combined weight in this score. */ public int getWeight() { return weight; diff --git a/fingerly-server/src/main/java/io/fingerly/server/EventsService.java b/fingerly-server/src/main/java/io/fingerly/server/EventsService.java index 8cf97b0..7f4b00e 100644 --- a/fingerly-server/src/main/java/io/fingerly/server/EventsService.java +++ b/fingerly-server/src/main/java/io/fingerly/server/EventsService.java @@ -1,9 +1,18 @@ package io.fingerly.server; import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.node.ObjectNode; import java.net.URLEncoder; import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.time.Instant; +import java.time.OffsetDateTime; +import java.time.format.DateTimeFormatter; +import java.time.format.DateTimeParseException; import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionException; +import java.util.concurrent.Future; +import java.util.concurrent.atomic.AtomicReference; /** * Reads events stored for the organization and environment of the client's @@ -71,6 +80,207 @@ public EventPage list() { return list(null); } + /** + * Consumes an event, once: the first call succeeds and every later one + * throws {@link EventAlreadyConsumedException}, so one event backs one + * decision. Never retried. + * + * @param requestId the {@code request_id} returned by identification + * @throws EventAlreadyConsumedException on {@code 409 event_already_consumed}, + * with {@link EventAlreadyConsumedException#getConsumedAt()} + * @throws FingerlyApiException on any other non-2xx response, such as + * {@code 404 event_not_found} + * @throws FingerlyTimeoutException if the request exceeds the client timeout + * @throws IllegalArgumentException if requestId is null, empty, "." or ".." + */ + public ConsumeResult consume(String requestId) { + return client.await(consumeAsync(requestId)); + } + + /** {@link #consume} without holding a thread; see {@link #getAsync}. */ + public CompletableFuture consumeAsync(String requestId) { + return client.sendAsync("POST", eventPath(requestId) + "/consume", null) + .thenApply(body -> ConsumeResult.from(Json.tree(body))); + } + + /** {@link #verify(String, VerifyOptions)} with {@link VerifyOptions#defaults()}. */ + public VerifyResult verify(String requestId) { + return verify(requestId, null); + } + + /** + * Reads an event and checks, in this order, stopping at the first that + * fails: it exists ({@code not_found}), it is no older than + * {@code maxAgeSeconds} ({@code too_old}, also when {@code occurred_at} + * is missing or unreadable), it carries the expected tag when one is set + * ({@code tag_mismatch}), it was identifiable ({@code unidentifiable}) + * and scored ({@code unscored}). When every check passes and consuming + * is on, it then consumes the event ({@code already_consumed} when + * another call did first). + * + *

The suspect score, level and triggers are never looked at: apply + * your own policy to them after {@link VerifyResult#isOk()}. + * + * @param options the checks, or null for {@link VerifyOptions#defaults()} + * @throws FingerlyApiException on any answer other than + * {@code 404 event_not_found} and {@code 409 event_already_consumed}, + * including {@code 404 event_pending}, which is not a verdict: read + * again after {@link FingerlyApiException#getRetryAfter()} seconds + * @throws FingerlyTimeoutException if a request exceeds the client timeout + * @throws IllegalArgumentException if requestId is null, empty, "." or ".." + */ + public VerifyResult verify(String requestId, VerifyOptions options) { + return client.await(verifyAsync(requestId, options)); + } + + /** {@link #verify(String)} without holding a thread. */ + public CompletableFuture verifyAsync(String requestId) { + return verifyAsync(requestId, null); + } + + /** + * {@link #verify(String, VerifyOptions)} without holding a thread. + * Cancelling the future abandons the call in flight. + */ + public CompletableFuture verifyAsync(String requestId, VerifyOptions options) { + VerifyOptions checks = options != null ? options : VerifyOptions.defaults(); + String path = eventPath(requestId); + CompletableFuture result = new CompletableFuture<>(); + AtomicReference> current = new AtomicReference<>(); + result.whenComplete((ignored, failure) -> { + if (result.isCancelled()) { + Future inFlight = current.get(); + if (inFlight != null) { + inFlight.cancel(true); + } + } + }); + + CompletableFuture read = client.getAsync(path); + current.set(read); + read.whenComplete((body, failure) -> { + if (result.isDone()) { + return; + } + if (failure != null) { + Throwable cause = unwrap(failure); + if (cause instanceof FingerlyApiException + && ((FingerlyApiException) cause).isEventNotFound()) { + result.complete(VerifyResult.failed(VerifyResult.REASON_NOT_FOUND, null)); + } else { + result.completeExceptionally(cause); + } + return; + } + Event event; + String reason; + try { + event = parseVerifiedEvent(body); + reason = check(event, checks); + } catch (RuntimeException unreadable) { + result.completeExceptionally(unreadable); + return; + } + if (reason != null) { + result.complete(VerifyResult.failed(reason, event)); + return; + } + if (!checks.isConsume()) { + result.complete(VerifyResult.ok(event)); + return; + } + CompletableFuture consumed = client.sendAsync("POST", path + "/consume", null); + current.set(consumed); + if (result.isCancelled()) { + consumed.cancel(true); + return; + } + consumed.whenComplete((ignored, consumeFailure) -> { + if (consumeFailure == null) { + result.complete(VerifyResult.ok(event)); + return; + } + Throwable cause = unwrap(consumeFailure); + if (cause instanceof EventAlreadyConsumedException) { + result.complete(VerifyResult.failed(VerifyResult.REASON_ALREADY_CONSUMED, event)); + } else if (cause instanceof FingerlyApiException + && ((FingerlyApiException) cause).isEventNotFound()) { + result.complete(VerifyResult.failed(VerifyResult.REASON_NOT_FOUND, null)); + } else { + result.completeExceptionally(cause); + } + }); + }); + return result; + } + + /** The first check before consumption that fails, or null when all pass. */ + private static String check(Event event, VerifyOptions checks) { + Instant occurred = rfc3339(event.getRaw().path("occurred_at")); + if (occurred == null || Duration.between(occurred, Instant.now()) + .compareTo(Duration.ofSeconds(checks.getMaxAgeSeconds())) > 0) { + return VerifyResult.REASON_TOO_OLD; + } + if (checks.getTag() != null) { + String tag = event.getTag() == null ? "" : event.getTag(); + if (!tag.equals(checks.getTag())) { + return VerifyResult.REASON_TAG_MISMATCH; + } + } + if (event.getVisitorId() == null || event.getVisitorId().isEmpty()) { + return VerifyResult.REASON_UNIDENTIFIABLE; + } + if (!event.isScored()) { + return VerifyResult.REASON_UNSCORED; + } + return null; + } + + /** Reads an RFC 3339 timestamp, or null when it is missing or unreadable. */ + private static Instant rfc3339(JsonNode node) { + if (!node.isTextual()) { + return null; + } + try { + return OffsetDateTime.parse(node.asText(), DateTimeFormatter.ISO_OFFSET_DATE_TIME).toInstant(); + } catch (DateTimeParseException unreadable) { + return null; + } + } + + /** + * Parses an event for {@link #verifyAsync(String, VerifyOptions)}, where + * an unreadable {@code occurred_at} fails the age check rather than the + * read. {@link Event#getRaw()} keeps the body as sent. + */ + private static Event parseVerifiedEvent(byte[] body) { + JsonNode tree = Json.tree(body); + JsonNode bound = tree; + if (tree.isObject() && tree.has("occurred_at")) { + JsonNode occurred = tree.get("occurred_at"); + boolean readable; + try { + readable = occurred.isNull() || occurred.isTextual() + && (occurred.asText().isEmpty() || Instant.parse(occurred.asText()) != null); + } catch (DateTimeParseException unreadable) { + readable = false; + } + if (!readable) { + bound = tree.deepCopy(); + ((ObjectNode) bound).remove("occurred_at"); + } + } + Event event = Json.convert(bound, Event.class); + event.setRaw(tree, new String(body, StandardCharsets.UTF_8)); + return event; + } + + private static Throwable unwrap(Throwable failure) { + return failure instanceof CompletionException && failure.getCause() != null + ? failure.getCause() + : failure; + } + private static String eventPath(String requestId) { if (requestId == null || requestId.isEmpty() || requestId.equals(".") || requestId.equals("..")) { diff --git a/fingerly-server/src/main/java/io/fingerly/server/FingerlyApiException.java b/fingerly-server/src/main/java/io/fingerly/server/FingerlyApiException.java index ce5a568..345828c 100644 --- a/fingerly-server/src/main/java/io/fingerly/server/FingerlyApiException.java +++ b/fingerly-server/src/main/java/io/fingerly/server/FingerlyApiException.java @@ -106,4 +106,80 @@ public boolean isEventNotFound() { public boolean isPending() { return status == 404 && "event_pending".equals(code); } + + /** + * Reports whether a consume found the event already consumed + * ({@code 409 event_already_consumed}); such an answer arrives as an + * {@link EventAlreadyConsumedException}. + */ + public boolean isEventAlreadyConsumed() { + return status == 409 && "event_already_consumed".equals(code); + } + + /** + * {@code 404 device_check_not_held}: no DeviceCheck token is held for the + * event. Decide without the bits. + */ + public boolean isDeviceCheckNotHeld() { + return status == 404 && "device_check_not_held".equals(code); + } + + /** + * {@code 409 device_check_spent}: the event's token was already used, or + * this call is not the request the token was first used for. + */ + public boolean isDeviceCheckSpent() { + return status == 409 && "device_check_spent".equals(code); + } + + /** {@code 409 device_check_busy}: another call is using the token right now. */ + public boolean isDeviceCheckBusy() { + return status == 409 && "device_check_busy".equals(code); + } + + /** + * {@code 409 credential_unavailable}: the project's Apple key is missing, + * refused by Apple, or has DeviceCheck turned off. The token is kept. + */ + public boolean isCredentialUnavailable() { + return status == 409 && "credential_unavailable".equals(code); + } + + /** {@code 422 bits_required}: a DeviceCheck write named neither bit. */ + public boolean isBitsRequired() { + return status == 422 && "bits_required".equals(code); + } + + /** + * {@code 422 apple_rejected}: Apple rejected the DeviceCheck token, which + * is now used up. {@link #getAppleStatus()} holds Apple's HTTP status. + */ + public boolean isAppleRejected() { + return status == 422 && "apple_rejected".equals(code); + } + + /** + * {@code 503 device_check_unavailable}: Apple or storage could not + * answer. The token is kept for a retry of the same request within the hour. + */ + public boolean isDeviceCheckUnavailable() { + return status == 503 && "device_check_unavailable".equals(code); + } + + /** + * Apple's HTTP status on an {@code apple_rejected} answer, from the + * {@code apple_status} entry of the error details, or null when there is none. + */ + public Integer getAppleStatus() { + if (details == null || !details.isArray()) { + return null; + } + for (JsonNode detail : details) { + if ("apple_status".equals(detail.path("field").asText())) { + String issue = detail.path("issue").asText(); + return issue.matches("[0-9]{1,9}") ? Integer.valueOf(issue) : null; + } + } + return null; + } } diff --git a/fingerly-server/src/main/java/io/fingerly/server/FingerlyClient.java b/fingerly-server/src/main/java/io/fingerly/server/FingerlyClient.java index f1f72c7..6c378ba 100644 --- a/fingerly-server/src/main/java/io/fingerly/server/FingerlyClient.java +++ b/fingerly-server/src/main/java/io/fingerly/server/FingerlyClient.java @@ -32,7 +32,7 @@ public final class FingerlyClient { /** The SDK version, reported to the API on every request. */ - public static final String VERSION = "0.2.0"; + public static final String VERSION = "0.3.0"; private static final Duration DEFAULT_TIMEOUT = Duration.ofSeconds(10); private static final String DEFAULT_API_VERSION = "v1"; @@ -43,13 +43,15 @@ public final class FingerlyClient { private final Duration timeout; private final HttpClient httpClient; private final EventsService events; + private final DeviceCheckService deviceCheck; private FingerlyClient(Builder builder) { // Validate the key even when an endpoint is given, so a public, proxy // or management key can never reach an API that would accept it. Keys.parse(builder.secretKey, Keys.SECRET); - String origin = Keys.origin( - builder.endpoint != null ? builder.endpoint : Keys.DEFAULT_ENDPOINT); + String origin = Keys.apiOrigin( + builder.endpoint != null ? builder.endpoint : Keys.DEFAULT_ENDPOINT, + builder.allowInsecureLoopback, "endpoint"); String apiVersion = Keys.apiVersion(builder.apiVersion); this.secretKey = builder.secretKey; @@ -59,6 +61,7 @@ private FingerlyClient(Builder builder) { // recycled for DNS. this.httpClient = builder.httpClient; this.events = new EventsService(this); + this.deviceCheck = new DeviceCheckService(this); } /** @@ -79,13 +82,22 @@ public EventsService events() { return events; } + /** Reads and writes the Apple DeviceCheck bits of an iOS event's device. */ + public DeviceCheckService deviceCheck() { + return deviceCheck; + } + /** * Performs one authenticated GET and waits for the body. * *

Never retries; the deadline covers receiving the body. */ byte[] get(String path) { - CompletableFuture pending = getAsync(path); + return await(getAsync(path)); + } + + /** Waits for an exchange, unwrapping its {@link FingerlyException}. */ + T await(CompletableFuture pending) { try { return pending.get(); } catch (ExecutionException failed) { @@ -113,13 +125,31 @@ byte[] get(String path) { * with a {@link FingerlyException}. Cancelling it abandons the call. */ CompletableFuture getAsync(String path) { - HttpRequest request = HttpRequest.newBuilder(URI.create(baseUrl + path)) - .GET() + return sendAsync("GET", path, null); + } + + /** + * Performs one authenticated exchange without holding a thread, with the + * same deadline as {@link #getAsync}. + * + * @param body a JSON body, or null to send none + */ + CompletableFuture sendAsync(String method, String path, byte[] body) { + HttpRequest.Builder builder = HttpRequest.newBuilder(URI.create(baseUrl + path)) + .method(method, body == null + ? HttpRequest.BodyPublishers.noBody() + : HttpRequest.BodyPublishers.ofByteArray(body)) .timeout(timeout) .header("X-API-Key", secretKey) .header("Accept", "application/json") - .header("User-Agent", USER_AGENT) - .build(); + .header("User-Agent", USER_AGENT); + if (body != null) { + builder.header("Content-Type", "application/json"); + } + return exchange(builder.build()); + } + + private CompletableFuture exchange(HttpRequest request) { BoundedBody body = new BoundedBody(); CompletableFuture> pending; @@ -212,7 +242,8 @@ private FingerlyApiException apiError(HttpResponse response) { return new FingerlyApiException(status, null, null, headerRequestId, retryAfter); } JsonNode error = tree.path("error"); - String code = error.path("code").isTextual() ? redact(error.path("code").asText()) : null; + JsonNode codeNode = error.path("code").isTextual() ? error.path("code") : tree.path("code"); + String code = codeNode.isTextual() ? redact(codeNode.asText()) : null; String requestId = tree.path("request_id").isTextual() ? redact(tree.path("request_id").asText()) : headerRequestId; @@ -227,9 +258,37 @@ private FingerlyApiException apiError(HttpResponse response) { details = null; } } + if (status == 409 && "event_already_consumed".equals(code)) { + return new EventAlreadyConsumedException( + status, code, details, requestId, retryAfter, redact(consumedAt(tree, error))); + } return new FingerlyApiException(status, code, details, requestId, retryAfter); } + /** + * The {@code consumed_at} of an {@code event_already_consumed} answer: at + * the top level, inside {@code error}, or as an {@code error.details} + * entry whose {@code issue} holds the time. Null when none is there. + */ + private static String consumedAt(JsonNode tree, JsonNode error) { + if (tree.path("consumed_at").isTextual()) { + return tree.path("consumed_at").asText(); + } + if (error.path("consumed_at").isTextual()) { + return error.path("consumed_at").asText(); + } + JsonNode details = error.path("details"); + if (details.isArray()) { + for (JsonNode detail : details) { + if ("consumed_at".equals(detail.path("field").asText()) + && detail.path("issue").isTextual()) { + return detail.path("issue").asText(); + } + } + } + return null; + } + /** * Parses {@code Retry-After}: delta-seconds, or an HTTP date turned into * seconds from now (never negative). @@ -281,6 +340,7 @@ public String toString() { public static final class Builder { private String secretKey; private String endpoint; + private boolean allowInsecureLoopback; private String apiVersion = DEFAULT_API_VERSION; private Duration timeout = DEFAULT_TIMEOUT; private HttpClient httpClient; @@ -296,12 +356,28 @@ public Builder secretKey(String secretKey) { return this; } - /** Overrides the Fingerly API origin, for a local API. */ + /** + * Overrides the Fingerly API origin: {@code https://host} with an + * optional port and nothing else. A path, query, fragment, + * credentials, whitespace or control byte is refused at + * {@link #build()}. + */ public Builder endpoint(String endpoint) { this.endpoint = endpoint; return this; } + /** + * For local development only: lets {@link #endpoint} be plain + * {@code http} when its host is {@code 127.0.0.1}, {@code localhost} + * or {@code [::1]}. Any other plain HTTP endpoint is refused either + * way. Defaults to false. + */ + public Builder allowInsecureLoopback(boolean allow) { + this.allowInsecureLoopback = allow; + return this; + } + /** The API route version. Defaults to {@code v1}. */ public Builder apiVersion(String apiVersion) { this.apiVersion = apiVersion; diff --git a/fingerly-server/src/main/java/io/fingerly/server/FingerlyProxy.java b/fingerly-server/src/main/java/io/fingerly/server/FingerlyProxy.java index 839c98a..f38bfe6 100644 --- a/fingerly-server/src/main/java/io/fingerly/server/FingerlyProxy.java +++ b/fingerly-server/src/main/java/io/fingerly/server/FingerlyProxy.java @@ -10,6 +10,10 @@ import java.net.http.HttpTimeoutException; import java.time.Duration; import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.LinkedHashSet; import java.util.List; import java.util.Locale; import java.util.Map; @@ -21,17 +25,18 @@ import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicBoolean; import java.util.function.Consumer; +import java.util.regex.Pattern; /** - * Same-origin proxy for the browser SDK. + * First-party proxy for Fingerly Web and native client SDKs. * *

Content blockers refuse browser requests to third-party APIs, and a * refused identification is a visitor never identified. Mounting this on the * path the dashboard issued, such as {@code /k7x2qm4a}, makes those requests - * first-party. The SDK posts both of its calls to that one path, and this - * forwards each to Fingerly's one ingest URL. The proxy key is what lets that - * hop declare the visitor's real address, origin and user agent; a public key - * alone never can, so nobody can spoof them. + * first-party. Client SDKs post single-pass {@code v: 3} envelopes to that one + * path, and this forwards raw request bytes untouched to Fingerly's one ingest + * URL. The proxy key is what lets that hop declare the visitor's real address, + * origin and user agent; a public key alone never can, so nobody can spoof them. * *

This class holds every decision and speaks to no server API. Mount it * through one of the adapters, which only read a request and write a response: @@ -70,6 +75,26 @@ public final class FingerlyProxy { "Content-Type", "X-Request-Id", "RateLimit-Limit", "RateLimit-Remaining", "Retry-After"); + /** Allowlisted caller headers forwarded verbatim when present and safe. */ + private static final List FORWARDED_CLIENT_HEADERS = List.of( + "Idempotency-Key", + "Sec-Fetch-Site", + "Sec-Fetch-Mode", + "Sec-Fetch-Dest", + "Sec-CH-UA", + "Sec-CH-UA-Mobile", + "Sec-CH-UA-Platform"); + + /** Exact native/bridge SDK platform identifiers restored after stripping x-fingerly-*. */ + private static final Pattern ALLOWED_PLATFORMS = Pattern.compile( + "\\A(ios|android|react-native|react-native-ios|react-native-android" + + "|flutter|flutter-ios|flutter-android|lynx-ios|lynx-android" + + "|capacitor|kotlin-multiplatform)\\z"); + + /** Refusal payload when both Origin and a validated native platform header are absent. */ + static final String MISSING_ORIGIN_JSON = + "{\"error\":{\"code\":\"missing_origin\",\"message\":\"Missing origin\",\"status\":400}}"; + /** The 500 body, and log line, for a {@link ClientIp} that returned no address. */ static final String NO_CLIENT_IP = "The client IP callback returned no valid address"; @@ -97,9 +122,9 @@ public interface ClientIp { private final String key; private final Keys.Scope scope; - private final boolean usesProxyKey; private final String upstreamUrl; private final String path; + private final Set allowedOrigins; private final long maxBodyBytes; private final Duration timeout; private final ClientIp clientIp; @@ -107,23 +132,27 @@ public interface ClientIp { private final Consumer log; private FingerlyProxy(Builder builder) { - if (isBlank(builder.proxyKey) && isBlank(builder.secretKey)) { + if (isBlank(builder.proxyKey)) { throw new IllegalArgumentException("A Fingerly proxy key is required"); } - if (!isBlank(builder.proxyKey) && !isBlank(builder.secretKey)) { - throw new IllegalArgumentException("Pass proxyKey or deprecated secretKey, not both"); - } if (builder.clientIp == null) { throw new IllegalArgumentException("clientIp is required"); } - this.usesProxyKey = !isBlank(builder.proxyKey); - this.key = usesProxyKey ? builder.proxyKey : builder.secretKey; - this.scope = Keys.parse(key, usesProxyKey ? Keys.PROXY : Keys.SECRET); - this.upstreamUrl = Keys.origin( - builder.upstream != null ? builder.upstream : Keys.DEFAULT_ENDPOINT) + this.key = builder.proxyKey; + this.scope = Keys.parse(key, Keys.PROXY); + this.upstreamUrl = Keys.apiOrigin( + builder.upstream != null ? builder.upstream : Keys.DEFAULT_ENDPOINT, + builder.allowInsecureLoopback, "upstream") + "/api/" + Keys.apiVersion(builder.apiVersion) + "/ingest"; + // The proxy key rides on every upstream call, so a client that follows + // redirects would hand it to whatever host a redirect names. + if (builder.httpClient != null + && builder.httpClient.followRedirects() != HttpClient.Redirect.NEVER) { + throw new IllegalArgumentException("httpClient must be built with Redirect.NEVER"); + } this.path = builder.path == null ? null : normalizePath(builder.path); + this.allowedOrigins = builder.allowedOrigins; if (builder.maxBodyBytes <= 0) { throw new IllegalArgumentException("maxBodyBytes must be positive"); } @@ -213,9 +242,9 @@ public ProxyResponse handle(ProxyRequest request) { } catch (InterruptedException interrupted) { pending.cancel(true); Thread.currentThread().interrupt(); - return ProxyResponse.refusal(502, "Bad gateway"); + return ProxyResponse.refusal(502, "Bad gateway", corsHeaders(corsOriginFor(request))); } catch (ExecutionException | java.util.concurrent.CancellationException unexpected) { - return failed(unexpected); + return failed(unexpected, corsOriginFor(request)); } } @@ -231,16 +260,17 @@ public ProxyResponse handle(ProxyRequest request) { * call and closes its connection. * * @param request the incoming request, as the adapter read it - * @return the upstream response, or a 404, 405, 401, 413, 500, 502 or 504 refusal + * @return the upstream response, or a 404, 405, 401, 400, 413, 500, 502 or 504 refusal */ public CompletableFuture handleAsync(ProxyRequest request) { + String corsOrigin = corsOriginFor(request); try { - Object prepared = prepare(request); + Object prepared = prepare(request, corsOrigin); return prepared instanceof ProxyResponse ? CompletableFuture.completedFuture((ProxyResponse) prepared) - : forward((HttpRequest) prepared); + : forward((HttpRequest) prepared, corsOrigin); } catch (RuntimeException unexpected) { - return CompletableFuture.completedFuture(failed(unexpected)); + return CompletableFuture.completedFuture(failed(unexpected, corsOrigin)); } } @@ -249,45 +279,71 @@ public CompletableFuture handleAsync(ProxyRequest request) { * * @return the refusal to answer with, or the {@link HttpRequest} to send */ - private Object prepare(ProxyRequest request) { + private Object prepare(ProxyRequest request, String corsOrigin) { if (!handles(request.path())) { return ProxyResponse.refusal(404, "Not Found"); } - if (!"POST".equalsIgnoreCase(request.method())) { - ProxyResponse refusal = ProxyResponse.refusal(405, "Method Not Allowed"); - List> headers = new ArrayList<>(refusal.headers()); - headers.add(Map.entry("Allow", "POST")); - return new ProxyResponse(405, headers, refusal.body()); + + // Step 1: Capture validated x-fingerly-sdk-platform before stripping x-fingerly-*. + String rawPlatform = request.header("X-Fingerly-SDK-Platform"); + String sdkPlatform = rawPlatform != null && ALLOWED_PLATFORMS.matcher(rawPlatform).matches() + ? rawPlatform + : null; + String origin = nonBlank(safe(request.header("Origin"))); + + if (!allowedOrigins.isEmpty() && "OPTIONS".equals(request.method())) { + if (origin != null && corsOrigin == null) { + return ProxyResponse.refusal(403, "Forbidden"); + } + List> headers = new ArrayList<>(); + headers.add(Map.entry("Cache-Control", "no-store")); + headers.add(Map.entry("Allow", "POST, OPTIONS")); + headers.addAll(corsHeaders(corsOrigin)); + return new ProxyResponse(204, headers, new byte[0]); + } + + // Methods are case-sensitive: "post" is a different method, not POST. + if (!"POST".equals(request.method())) { + List> extra = new ArrayList<>(); + extra.add(Map.entry("Allow", allowedOrigins.isEmpty() ? "POST" : "POST, OPTIONS")); + extra.addAll(corsHeaders(corsOrigin)); + return ProxyResponse.refusal(405, "Method Not Allowed", extra); } String apiKey = upstreamKeyFor(request.header("X-API-Key")); if (apiKey == null) { - return ProxyResponse.refusal(401, "Unauthorized"); + return ProxyResponse.refusal(401, "Unauthorized", corsHeaders(corsOrigin)); } - String address = visitorAddress(request); - if (address == null) { - return ProxyResponse.refusal(500, NO_CLIENT_IP); + // Step 4: Enforce Origin / native-platform gate (never synthesize X-Fingerly-Origin). + if (origin == null && sdkPlatform == null) { + return ProxyResponse.jsonRefusal(400, MISSING_ORIGIN_JSON, corsHeaders(corsOrigin)); } // Refuse on the declared size first; the read below enforces the real one. if (request.contentLength() > maxBodyBytes) { - return ProxyResponse.refusal(413, "Payload Too Large"); + return ProxyResponse.refusal(413, "Payload Too Large", corsHeaders(corsOrigin)); + } + + String address = visitorAddress(request); + if (address == null) { + return ProxyResponse.refusal(500, NO_CLIENT_IP, corsHeaders(corsOrigin)); } + byte[] body; try { body = readBounded(request.body()); } catch (BodyTooLargeException tooLarge) { - return ProxyResponse.refusal(413, "Payload Too Large"); + return ProxyResponse.refusal(413, "Payload Too Large", corsHeaders(corsOrigin)); } catch (IOException unreadable) { - return ProxyResponse.refusal(400, "Bad Request"); + return ProxyResponse.refusal(400, "Bad Request", corsHeaders(corsOrigin)); } HttpRequest.Builder upstreamRequest = HttpRequest.newBuilder(URI.create(upstreamUrl)) .POST(HttpRequest.BodyPublishers.ofByteArray(body)) .timeout(timeout); - applyUpstreamHeaders(upstreamRequest, request, apiKey, address); + applyUpstreamHeaders(upstreamRequest, request, apiKey, address, origin, sdkPlatform); return upstreamRequest.build(); } @@ -306,13 +362,13 @@ private Object prepare(ProxyRequest request) { * anything is relayed, so a truncated answer is never passed on as a * success. */ - private CompletableFuture forward(HttpRequest request) { + private CompletableFuture forward(HttpRequest request, String corsOrigin) { BoundedBody body = new BoundedBody(); CompletableFuture> pending; try { pending = client().sendAsync(request, body); } catch (RuntimeException refused) { - return CompletableFuture.completedFuture(failed(refused)); + return CompletableFuture.completedFuture(failed(refused, corsOrigin)); } CompletableFuture result = new CompletableFuture<>(); @@ -322,7 +378,7 @@ private CompletableFuture forward(HttpRequest request) { if (settled.compareAndSet(false, true)) { pending.cancel(true); body.cancel(); - result.complete(failed(new TimeoutException())); + result.complete(failed(new TimeoutException(), corsOrigin)); } }, timeout); pending.whenComplete((response, failure) -> { @@ -331,13 +387,20 @@ private CompletableFuture forward(HttpRequest request) { return; } if (failure != null) { - result.complete(failed(failure)); + result.complete(failed(failure, corsOrigin)); + return; + } + int status = response.statusCode(); + if (status < 200 || status > 599 || (status >= 300 && status <= 399)) { + // A redirect is never followed or passed on, and a status + // outside 200 to 599 is not an answer a browser can use. + result.complete(failed(new UnusableStatusException(status), corsOrigin)); return; } try { - result.complete(relay(response)); + result.complete(relay(response, corsOrigin)); } catch (RuntimeException unexpected) { - result.complete(failed(unexpected)); + result.complete(failed(unexpected, corsOrigin)); } }); // A cancelled result is a visitor who left: drop the upstream call @@ -364,7 +427,7 @@ private HttpClient client() { * compressed in a way that cannot be decoded) is its 502. Neither body * names Fingerly. */ - private ProxyResponse failed(Throwable cause) { + private ProxyResponse failed(Throwable cause, String corsOrigin) { Throwable root = cause; while ((root instanceof CompletionException || root instanceof ExecutionException) && root.getCause() != null) { @@ -378,9 +441,10 @@ private ProxyResponse failed(Throwable cause) { } catch (RuntimeException ignored) { // A logging hook that throws must not turn a 502 into a 500. } + List> cors = corsHeaders(corsOrigin); return timedOut - ? ProxyResponse.refusal(504, "Gateway timeout") - : ProxyResponse.refusal(502, "Bad gateway"); + ? ProxyResponse.refusal(504, "Gateway timeout", cors) + : ProxyResponse.refusal(502, "Bad gateway", cors); } /** One short, key-free description of a failure: its type and nothing it carries. */ @@ -391,6 +455,9 @@ private static String describe(Throwable cause) { if (cause instanceof BoundedBody.UnsupportedEncodingException) { return "unsupported content coding"; } + if (cause instanceof UnusableStatusException) { + return "unusable status " + ((UnusableStatusException) cause).status; + } return cause == null ? "unknown" : cause.getClass().getSimpleName(); } @@ -433,18 +500,15 @@ private String visitorAddress(ProxyRequest request) { } /** - * Decides which key to present upstream. + * Checks the browser's key, on every request. * - *

With a proxy key the browser must supply a public key for the same - * environment, which the proxy forwards alongside its own. The deprecated - * secret-key mode authenticates on its own. + *

The browser must supply a public key for the proxy key's environment, + * which the proxy forwards alongside its own. Any other value, a secret + * key included, is refused and never sent upstream. * - * @return the key to forward, or null when the caller is unauthorised + * @return the public key to forward, or null when the caller is unauthorised */ private String upstreamKeyFor(String presented) { - if (!usesProxyKey) { - return key; - } Keys.Scope browser; try { browser = Keys.parse(presented, Keys.PUBLIC); @@ -457,26 +521,43 @@ private String upstreamKeyFor(String presented) { /** * Sets the upstream headers. * - *

Nothing is copied wholesale: only the body's content type, the - * visitor metadata Fingerly needs, this proxy's version, and an - * idempotency key when the SDK sent one. + *

Nothing is copied wholesale: all incoming {@code x-fingerly-*}, + * {@code Cookie} and {@code Authorization} headers are stripped, only a + * validated {@code X-Fingerly-SDK-Platform} is restored, and + * {@code X-Fingerly-Origin} is sent only when incoming {@code Origin} is + * non-empty (never synthesized from {@code Host} or the request URL). */ private void applyUpstreamHeaders( - HttpRequest.Builder upstream, ProxyRequest request, String apiKey, String address) { - String contentType = safe(request.header("Content-Type")); + HttpRequest.Builder upstream, + ProxyRequest request, + String apiKey, + String address, + String origin, + String sdkPlatform) { + String contentType = nonBlank(safe(request.header("Content-Type"))); upstream.header("Content-Type", contentType == null ? "application/json" : contentType); upstream.header("X-API-Key", apiKey); upstream.header("X-Fingerly-Client-IP", address); - upstream.header("X-Fingerly-Origin", orEmpty(safe(request.header("Origin")))); + // One channel only: the API refuses a proxied request that names both, + // and a page origin wins, as it does on a direct call. + if (origin != null) { + upstream.header("X-Fingerly-Origin", origin); + } else if (sdkPlatform != null) { + upstream.header("X-Fingerly-SDK-Platform", sdkPlatform); + } + String host = nonBlank(safe(request.header("Host"))); + if (host != null) { + upstream.header("X-Fingerly-Host", host); + } upstream.header("X-Fingerly-User-Agent", orEmpty(safe(request.header("User-Agent")))); upstream.header("User-Agent", USER_AGENT); upstream.header("X-Fingerly-Proxy-Version", PROXY_VERSION); - if (usesProxyKey) { - upstream.header("X-Fingerly-Proxy-Key", key); - } - String idempotency = safe(request.header("Idempotency-Key")); - if (idempotency != null && !idempotency.isEmpty()) { - upstream.header("Idempotency-Key", idempotency); + upstream.header("X-Fingerly-Proxy-Key", key); + for (String name : FORWARDED_CLIENT_HEADERS) { + String value = nonBlank(safe(request.header(name))); + if (value != null) { + upstream.header(name, value); + } } } @@ -485,7 +566,7 @@ private void applyUpstreamHeaders( * headers. A value naming Fingerly is dropped too, since a blocker can * match that as well as a name. */ - private static ProxyResponse relay(HttpResponse response) { + private static ProxyResponse relay(HttpResponse response, String corsOrigin) { List> headers = new ArrayList<>(); for (String name : RELAYED_HEADERS) { for (String value : response.headers().allValues(name)) { @@ -495,6 +576,7 @@ private static ProxyResponse relay(HttpResponse response) { } } headers.add(Map.entry("Cache-Control", "no-store")); + headers.addAll(corsHeaders(corsOrigin)); byte[] body = BODILESS.contains(response.statusCode()) ? new byte[0] @@ -502,6 +584,31 @@ private static ProxyResponse relay(HttpResponse response) { return new ProxyResponse(response.statusCode(), headers, body); } + private String corsOriginFor(ProxyRequest request) { + if (allowedOrigins.isEmpty() || request == null) { + return null; + } + String origin = nonBlank(safe(request.header("Origin"))); + return origin != null && allowedOrigins.contains(origin) ? origin : null; + } + + private static List> corsHeaders(String corsOrigin) { + if (corsOrigin == null) { + return List.of(); + } + return List.of( + Map.entry("Access-Control-Allow-Origin", corsOrigin), + Map.entry("Access-Control-Allow-Methods", "POST, OPTIONS"), + Map.entry( + "Access-Control-Allow-Headers", + "content-type, x-api-key, idempotency-key, x-fingerly-sdk-platform"), + Map.entry( + "Access-Control-Expose-Headers", + "x-request-id, ratelimit-limit, ratelimit-remaining, retry-after"), + Map.entry("Access-Control-Max-Age", "600"), + Map.entry("Vary", "Origin")); + } + /** * Reads a body without buffering more than the limit. * @@ -522,6 +629,17 @@ private byte[] readBounded(InputStream stream) throws IOException { return collected.toByteArray(); } + /** An upstream status the proxy answers 502 for: a redirect, or outside 200 to 599. */ + private static final class UnusableStatusException extends RuntimeException { + private static final long serialVersionUID = 1L; + final int status; + + UnusableStatusException(int status) { + super(null, null, false, false); + this.status = status; + } + } + /** Raised internally when a streamed body passes the limit. */ private static final class BodyTooLargeException extends IOException { private static final long serialVersionUID = 1L; @@ -563,6 +681,10 @@ private static String safe(String value) { return Keys.isHeaderSafe(value) ? value : null; } + private static String nonBlank(String value) { + return value == null || value.trim().isEmpty() ? null : value; + } + private static String orEmpty(String value) { return value == null ? "" : value; } @@ -571,16 +693,17 @@ private static String orEmpty(String value) { @Override public String toString() { return "FingerlyProxy[path=" + path + ", upstream=" + upstreamUrl - + ", " + (usesProxyKey ? "proxyKey" : "secretKey") + "=[redacted]]"; + + ", proxyKey=[redacted]]"; } /** Builds a {@link FingerlyProxy}. */ public static final class Builder { private String proxyKey; - private String secretKey; private String path; + private Set allowedOrigins = Collections.emptySet(); private String apiVersion = "v1"; private String upstream; + private boolean allowInsecureLoopback; private long maxBodyBytes = DEFAULT_MAX_BODY_BYTES; private Duration timeout = DEFAULT_TIMEOUT; private ClientIp clientIp; @@ -589,21 +712,13 @@ public static final class Builder { private Builder() {} - /** A proxy key, {@code fly_px_…}, held only on your server. Required. */ - public Builder proxyKey(String proxyKey) { - this.proxyKey = proxyKey; - return this; - } - /** - * The deprecated alternative to {@link #proxyKey}. Secret read keys - * will be refused in the next major version; do not pass both. - * - * @deprecated use {@link #proxyKey} + * A proxy key, {@code fly_px_{environment}_…}, held only on your + * server. Required: {@link #build()} refuses a missing or malformed + * one. Browsers must present a public key of the same environment. */ - @Deprecated - public Builder secretKey(String secretKey) { - this.secretKey = secretKey; + public Builder proxyKey(String proxyKey) { + this.proxyKey = proxyKey; return this; } @@ -629,18 +744,55 @@ public Builder path(String path) { return this; } + /** + * Cross-origin page origins permitted to post to this proxy. Leave + * unset when the proxy serves the page's own origin. + */ + public Builder allowedOrigins(Collection origins) { + if (origins == null || origins.isEmpty()) { + this.allowedOrigins = Collections.emptySet(); + return this; + } + Set normalized = new LinkedHashSet<>(); + for (String origin : origins) { + normalized.add(Keys.origin(origin)); + } + this.allowedOrigins = Collections.unmodifiableSet(normalized); + return this; + } + + /** Cross-origin page origins permitted to post to this proxy. */ + public Builder allowedOrigins(String... origins) { + return allowedOrigins(origins == null ? Collections.emptyList() : Arrays.asList(origins)); + } + /** The API version forwarded upstream. Defaults to {@code v1}. */ public Builder apiVersion(String apiVersion) { this.apiVersion = apiVersion; return this; } - /** Overrides the API origin implied by the key. */ + /** + * Overrides the API origin: {@code https://host} with an optional + * port and nothing else. A path, query, fragment, credentials, + * whitespace or control byte is refused at {@link #build()}. + */ public Builder upstream(String upstream) { this.upstream = upstream; return this; } + /** + * For local development only: lets {@link #upstream} be plain + * {@code http} when its host is {@code 127.0.0.1}, {@code localhost} + * or {@code [::1]}. Any other plain HTTP upstream is refused either + * way. Defaults to false. + */ + public Builder allowInsecureLoopback(boolean allow) { + this.allowInsecureLoopback = allow; + return this; + } + /** Larger request bodies are refused with 413. Defaults to 1 MiB. */ public Builder maxBodyBytes(long maxBodyBytes) { this.maxBodyBytes = maxBodyBytes; @@ -668,8 +820,9 @@ public Builder log(Consumer log) { } /** - * Supplies the HTTP client for the upstream call. Configure it with - * {@link HttpClient.Redirect#NEVER}. Without one, every proxy and + * Supplies the HTTP client for the upstream call. It must be built + * with {@link HttpClient.Redirect#NEVER}, the JDK default; any other + * policy is refused at {@link #build()}. Without one, every proxy and * client in the process shares one HTTP/1.1 pool that is recycled * every two minutes so DNS changes are picked up. */ @@ -682,8 +835,7 @@ public Builder httpClient(HttpClient httpClient) { @Override public String toString() { return "FingerlyProxy.Builder[path=" + path + ", upstream=" + upstream - + ", proxyKey=" + (proxyKey == null ? "null" : "[redacted]") - + ", secretKey=" + (secretKey == null ? "null" : "[redacted]") + "]"; + + ", proxyKey=" + (proxyKey == null ? "null" : "[redacted]") + "]"; } /** diff --git a/fingerly-server/src/main/java/io/fingerly/server/Keys.java b/fingerly-server/src/main/java/io/fingerly/server/Keys.java index bbc4ff6..0ec8284 100644 --- a/fingerly-server/src/main/java/io/fingerly/server/Keys.java +++ b/fingerly-server/src/main/java/io/fingerly/server/Keys.java @@ -102,6 +102,68 @@ static String origin(String value) { return value.replaceAll("/+$", ""); } + /** Printable ASCII with no space: nothing a URL parser could tolerate and pass on. */ + private static final Pattern PRINTABLE = Pattern.compile("\\A[\\x21-\\x7e]+\\z"); + + /** The only hosts a plain HTTP API origin may name, and only with the development flag. */ + private static final java.util.Set LOOPBACK_HOSTS = + java.util.Set.of("127.0.0.1", "localhost", "[::1]"); + + /** + * Validates an API origin option (the proxy's {@code upstream}, the + * client's {@code endpoint}): a bare origin, nothing else. + * + *

Only scheme, host and an optional port are accepted. A path, query, + * fragment, credentials, whitespace or control byte is refused, and the + * returned origin is rebuilt from the parsed parts rather than the input. + * The scheme is {@code https}; {@code http} is accepted only when + * {@code allowInsecureLoopback} is set and the host is {@code 127.0.0.1}, + * {@code localhost} or {@code [::1]}. No message echoes the value. + * + * @param option the option's name, for the error message + * @return the origin, as {@code scheme://host[:port]} + * @throws IllegalArgumentException if the value is anything else + */ + static String apiOrigin(String value, boolean allowInsecureLoopback, String option) { + String bare = option + " must be an https origin (scheme, host and optional port only)"; + if (value == null || !PRINTABLE.matcher(value).matches()) { + throw new IllegalArgumentException(bare); + } + URI uri; + try { + uri = new URI(value); + } catch (Exception cause) { + throw new IllegalArgumentException(bare); + } + String scheme = uri.getScheme() == null ? "" : uri.getScheme().toLowerCase(Locale.ROOT); + String host = uri.getHost() == null ? null : uri.getHost().toLowerCase(Locale.ROOT); + String path = uri.getRawPath(); + if (uri.isOpaque() + || host == null + || host.isEmpty() + || uri.getRawUserInfo() != null + || uri.getRawQuery() != null + || uri.getRawFragment() != null + || value.indexOf('?') >= 0 + || value.indexOf('#') >= 0 + || uri.getPort() == 0 + || uri.getPort() > 65535 + || !(path == null || path.isEmpty() || path.equals("/"))) { + throw new IllegalArgumentException(bare); + } + if (scheme.equals("http")) { + if (!allowInsecureLoopback || !LOOPBACK_HOSTS.contains(host)) { + throw new IllegalArgumentException( + "a plain HTTP " + option + + " is allowed only for 127.0.0.1, localhost or [::1]" + + " with allowInsecureLoopback(true)"); + } + } else if (!scheme.equals("https")) { + throw new IllegalArgumentException(bare); + } + return scheme + "://" + host + (uri.getPort() == -1 ? "" : ":" + uri.getPort()); + } + /** * Validates an API route version before it is spliced into a URL. * diff --git a/fingerly-server/src/main/java/io/fingerly/server/ProxyResponse.java b/fingerly-server/src/main/java/io/fingerly/server/ProxyResponse.java index 4e7b6f7..70f47d3 100644 --- a/fingerly-server/src/main/java/io/fingerly/server/ProxyResponse.java +++ b/fingerly-server/src/main/java/io/fingerly/server/ProxyResponse.java @@ -50,10 +50,28 @@ void writeBodyTo(java.io.OutputStream sink) throws java.io.IOException { /** Builds a plain-text response that never reached Fingerly. */ static ProxyResponse refusal(int status, String message) { + return refusal(status, message, List.of()); + } + + /** Builds a plain-text response with extra headers. */ + static ProxyResponse refusal( + int status, String message, List> extraHeaders) { byte[] body = message.getBytes(java.nio.charset.StandardCharsets.UTF_8); List> headers = new ArrayList<>(); headers.add(Map.entry("Content-Type", "text/plain; charset=utf-8")); headers.add(Map.entry("Cache-Control", "no-store")); + headers.addAll(extraHeaders); + return new ProxyResponse(status, Collections.unmodifiableList(headers), body); + } + + /** Builds a JSON error response that never reached Fingerly. */ + static ProxyResponse jsonRefusal( + int status, String json, List> extraHeaders) { + byte[] body = json.getBytes(java.nio.charset.StandardCharsets.UTF_8); + List> headers = new ArrayList<>(); + headers.add(Map.entry("Content-Type", "application/json")); + headers.add(Map.entry("Cache-Control", "no-store")); + headers.addAll(extraHeaders); return new ProxyResponse(status, Collections.unmodifiableList(headers), body); } } diff --git a/fingerly-server/src/main/java/io/fingerly/server/VerifyOptions.java b/fingerly-server/src/main/java/io/fingerly/server/VerifyOptions.java new file mode 100644 index 0000000..f9e9a8e --- /dev/null +++ b/fingerly-server/src/main/java/io/fingerly/server/VerifyOptions.java @@ -0,0 +1,109 @@ +package io.fingerly.server; + +/** + * The checks {@link EventsService#verify(String, VerifyOptions)} applies. + * + *

{@code
+ * VerifyOptions options = VerifyOptions.builder()
+ *     .maxAgeSeconds(120)
+ *     .tag("checkout")
+ *     .build();
+ * }
+ */ +public final class VerifyOptions { + + /** The age limit when none is set: five minutes. */ + public static final long DEFAULT_MAX_AGE_SECONDS = 300; + + private static final VerifyOptions DEFAULTS = builder().build(); + + private final long maxAgeSeconds; + private final String tag; + private final boolean consume; + + private VerifyOptions(Builder builder) { + this.maxAgeSeconds = builder.maxAgeSeconds; + this.tag = builder.tag; + this.consume = builder.consume; + } + + /** A new builder, starting from the defaults. */ + public static Builder builder() { + return new Builder(); + } + + /** The defaults: 300 seconds, no tag check, consume. */ + public static VerifyOptions defaults() { + return DEFAULTS; + } + + /** The oldest event, in seconds since {@code occurred_at}, that passes. */ + public long getMaxAgeSeconds() { + return maxAgeSeconds; + } + + /** The tag the event must carry exactly, or null for no tag check. */ + public String getTag() { + return tag; + } + + /** Whether a passing event is consumed. */ + public boolean isConsume() { + return consume; + } + + @Override + public String toString() { + return "VerifyOptions[maxAgeSeconds=" + maxAgeSeconds + ", tag=" + tag + + ", consume=" + consume + "]"; + } + + /** Builds {@link VerifyOptions}. */ + public static final class Builder { + private long maxAgeSeconds = DEFAULT_MAX_AGE_SECONDS; + private String tag; + private boolean consume = true; + + private Builder() {} + + /** The oldest event that passes, in whole seconds. Defaults to 300. */ + public Builder maxAgeSeconds(long maxAgeSeconds) { + this.maxAgeSeconds = maxAgeSeconds; + return this; + } + + /** + * The tag the event must carry exactly. Null, the default, skips the + * tag check. + */ + public Builder tag(String tag) { + this.tag = tag; + return this; + } + + /** + * Whether to consume an event that passes every other check, so it + * backs one decision only. Defaults to true. + */ + public Builder consume(boolean consume) { + this.consume = consume; + return this; + } + + /** + * Builds the options. + * + * @throws IllegalArgumentException if maxAgeSeconds is not positive, + * or the tag is empty, which would match untagged events + */ + public VerifyOptions build() { + if (maxAgeSeconds <= 0) { + throw new IllegalArgumentException("maxAgeSeconds must be positive"); + } + if (tag != null && tag.isEmpty()) { + throw new IllegalArgumentException("tag must not be empty; leave it null to skip the check"); + } + return new VerifyOptions(this); + } + } +} diff --git a/fingerly-server/src/main/java/io/fingerly/server/VerifyResult.java b/fingerly-server/src/main/java/io/fingerly/server/VerifyResult.java new file mode 100644 index 0000000..f9465c0 --- /dev/null +++ b/fingerly-server/src/main/java/io/fingerly/server/VerifyResult.java @@ -0,0 +1,65 @@ +package io.fingerly.server; + +/** + * The outcome of {@link EventsService#verify(String, VerifyOptions)}. + * + *

{@link #getReason()} is one of the {@code REASON_} constants, or + * {@code ""} when {@link #isOk()}. It never reflects the suspect score: score + * policy stays with the caller. + */ +public final class VerifyResult { + + /** No event has this request id: never issued, or past retention. */ + public static final String REASON_NOT_FOUND = "not_found"; + + /** The event is older than the age limit, or its time is unreadable. */ + public static final String REASON_TOO_OLD = "too_old"; + + /** The event's tag is not the expected one. */ + public static final String REASON_TAG_MISMATCH = "tag_mismatch"; + + /** The submission was unidentifiable: its {@code visitor_id} is empty. */ + public static final String REASON_UNIDENTIFIABLE = "unidentifiable"; + + /** The event was not scored. */ + public static final String REASON_UNSCORED = "unscored"; + + /** Another call consumed the event first. */ + public static final String REASON_ALREADY_CONSUMED = "already_consumed"; + + private final String reason; + private final Event event; + + private VerifyResult(String reason, Event event) { + this.reason = reason; + this.event = event; + } + + static VerifyResult ok(Event event) { + return new VerifyResult("", event); + } + + static VerifyResult failed(String reason, Event event) { + return new VerifyResult(reason, event); + } + + /** True when every check passed and, unless turned off, the event was consumed. */ + public boolean isOk() { + return reason.isEmpty(); + } + + /** The first check that failed, as a {@code REASON_} value, or {@code ""} when ok. */ + public String getReason() { + return reason; + } + + /** The event read, or null for {@link #REASON_NOT_FOUND}. */ + public Event getEvent() { + return event; + } + + @Override + public String toString() { + return "VerifyResult[ok=" + isOk() + ", reason=" + reason + ", event=" + event + "]"; + } +} diff --git a/fingerly-server/src/test/java/io/fingerly/server/ContractTest.java b/fingerly-server/src/test/java/io/fingerly/server/ContractTest.java index 093518b..bb3b0e2 100644 --- a/fingerly-server/src/test/java/io/fingerly/server/ContractTest.java +++ b/fingerly-server/src/test/java/io/fingerly/server/ContractTest.java @@ -38,6 +38,7 @@ private static JsonNode fixtures() throws IOException { private static FingerlyClient client(FakeApi api) { return FingerlyClient.builder() .secretKey("fly_sk_development_contract") + .allowInsecureLoopback(true) .endpoint(api.url()) .build(); } @@ -198,4 +199,56 @@ void refusesAStaleWebhookFixture() throws IOException { Webhooks.DEFAULT_TOLERANCE, stale)); } + + @Test + void v3ContractRetiresSupplementAndModelsSignalsAndEmptyVisitorId() throws IOException { + JsonNode openapi = Json.MAPPER.readTree(Files.readAllBytes(CONTRACT.resolve("openapi.json"))); + assertTrue(openapi.path("paths").path("/api/v1/events/{request_id}/supplement").isMissingNode()); + assertTrue(openapi.path("components").path("schemas").path("SupplementOut").isMissingNode()); + assertTrue(openapi.path("components").path("schemas").path("IdentifyOut") + .path("properties").path("deferred_token").isMissingNode()); + + String unidentifiable = "{" + + "\"request_id\":\"0199a963-e400-7000-8000-000000000099\"," + + "\"visitor_id\":\"\"," + + "\"occurred_at\":\"2026-09-17T12:00:00Z\"," + + "\"state\":\"unavailable\"," + + "\"reason\":\"enrichment_unavailable\"," + + "\"environment\":\"production\"," + + "\"platform\":\"android\"," + + "\"sdk_platform\":\"android\"," + + "\"sdk_version\":\"0.3.0\"," + + "\"suspect_score\":48," + + "\"suspect_level\":\"high\"," + + "\"triggers\":[" + + "{\"signal\":\"attestation_failed\",\"weight\":30,\"confidence\":\"high\"}," + + "{\"signal\":\"device_inconsistent\",\"weight\":18,\"confidence\":\"high\"}," + + "{\"signal\":\"report_incomplete\",\"weight\":16,\"confidence\":\"medium\"}," + + "{\"signal\":\"tampering\",\"weight\":15,\"confidence\":\"high\"}," + + "{\"signal\":\"custom_root_ca\",\"weight\":0,\"confidence\":\"low\"}," + + "{\"signal\":\"secondary_user_profile\",\"weight\":0,\"confidence\":\"high\"}," + + "{\"signal\":\"attestation_downgrade\",\"weight\":30,\"confidence\":\"high\"}," + + "{\"signal\":\"probe_errors\",\"weight\":4,\"confidence\":\"low\"}," + + "{\"signal\":\"fetch_metadata_missing\",\"weight\":2,\"confidence\":\"low\"}," + + "{\"signal\":\"web_ticket_missing\",\"weight\":4,\"confidence\":\"low\"}," + + "{\"signal\":\"web_ticket_invalid\",\"weight\":20,\"confidence\":\"high\"}" + + "]," + + "\"has_document\":false" + + "}"; + assertEquals("attestation_downgrade", EventTrigger.SIGNAL_ATTESTATION_DOWNGRADE); + assertEquals("probe_errors", EventTrigger.SIGNAL_PROBE_ERRORS); + assertEquals("fetch_metadata_missing", EventTrigger.SIGNAL_FETCH_METADATA_MISSING); + assertEquals("web_ticket_missing", EventTrigger.SIGNAL_WEB_TICKET_MISSING); + assertEquals("web_ticket_invalid", EventTrigger.SIGNAL_WEB_TICKET_INVALID); + try (FakeApi api = FakeApi.json(200, unidentifiable)) { + Event event = client(api).events().get("0199a963-e400-7000-8000-000000000099"); + assertEquals("", event.getVisitorId()); + assertEquals("enrichment_unavailable", event.getReason()); + assertEquals(11, event.getTriggers().size()); + for (EventTrigger trigger : event.getTriggers()) { + assertTrue(EventTrigger.V3_SIGNALS.contains(trigger.getSignal()), + trigger.getSignal()); + } + } + } } diff --git a/fingerly-server/src/test/java/io/fingerly/server/DeviceCheckServiceTest.java b/fingerly-server/src/test/java/io/fingerly/server/DeviceCheckServiceTest.java new file mode 100644 index 0000000..edfc2c7 --- /dev/null +++ b/fingerly-server/src/test/java/io/fingerly/server/DeviceCheckServiceTest.java @@ -0,0 +1,216 @@ +package io.fingerly.server; + +import static io.fingerly.server.Fixtures.REQUEST_ID; +import static io.fingerly.server.Fixtures.SECRET_KEY; +import static org.junit.jupiter.api.Assertions.assertArrayEquals; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.io.IOException; +import java.net.ServerSocket; +import java.time.Duration; +import java.util.Optional; +import java.util.concurrent.atomic.AtomicInteger; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.CsvSource; + +class DeviceCheckServiceTest { + + private static final String DEVICE_CHECK = "/api/v1/events/" + REQUEST_ID + "/device-check"; + + private static FingerlyClient client(String url) { + return FingerlyClient.builder() + .secretKey(SECRET_KEY) + .allowInsecureLoopback(true) + .endpoint(url) + .timeout(Duration.ofSeconds(2)) + .build(); + } + + private static String error(int status, String code, String details) { + return "{\"error\":{\"status\":" + status + ",\"code\":\"" + code + "\",\"message\":\"m\"" + + (details == null ? "" : ",\"details\":" + details) + "}}"; + } + + @Test + void queriesTheBits() throws IOException { + try (FakeApi api = FakeApi.json(200, + "{\"found\":true,\"bit0\":true,\"bit1\":false,\"last_update\":\"2026-10\"}")) { + DeviceCheckBits bits = client(api.url()).deviceCheck().query(REQUEST_ID); + + assertTrue(bits.isFound()); + assertTrue(bits.getBit0()); + assertFalse(bits.getBit1()); + assertEquals(Optional.of("2026-10"), bits.getLastUpdate()); + FakeApi.Recorded sent = api.first(); + assertEquals("POST", sent.method); + assertEquals(DEVICE_CHECK + "/query", sent.path); + assertEquals(SECRET_KEY, sent.header("X-API-Key")); + assertEquals(0, sent.body.length); + } + } + + @Test + void readsBitsApplesHoldsNoneFor() throws IOException { + try (FakeApi api = FakeApi.json(200, "{\"found\":false,\"bit0\":false,\"bit1\":false}")) { + DeviceCheckBits bits = client(api.url()).deviceCheck().queryAsync(REQUEST_ID).join(); + + assertFalse(bits.isFound()); + assertEquals(Optional.empty(), bits.getLastUpdate()); + } + } + + @Test + void writesOnlyTheNamedBits() throws IOException { + try (FakeApi api = FakeApi.json(200, "{\"bit0\":true}")) { + DeviceCheckWriteResult result = client(api.url()).deviceCheck() + .write(REQUEST_ID, DeviceCheckWrite.builder().bit0(true).build()); + + assertEquals(Optional.of(true), result.getBit0()); + assertEquals(Optional.empty(), result.getBit1()); + FakeApi.Recorded sent = api.first(); + assertEquals("PUT", sent.method); + assertEquals(DEVICE_CHECK, sent.path); + assertEquals("application/json", sent.header("Content-Type")); + assertEquals("{\"bit0\":true}", sent.bodyText()); + } + } + + @Test + void writesBothBits() throws IOException { + try (FakeApi api = FakeApi.json(200, "{\"bit0\":false,\"bit1\":true}")) { + client(api.url()).deviceCheck() + .write(REQUEST_ID, DeviceCheckWrite.builder().bit0(false).bit1(true).build()); + + assertEquals("{\"bit0\":false,\"bit1\":true}", api.first().bodyText()); + } + } + + @Test + void aWriteNamesABit() { + assertThrows(IllegalArgumentException.class, () -> DeviceCheckWrite.builder().build()); + FingerlyClient fingerly = client("http://127.0.0.1:1"); + assertThrows(IllegalArgumentException.class, () -> fingerly.deviceCheck().write(REQUEST_ID, null)); + assertThrows(IllegalArgumentException.class, () -> fingerly.deviceCheck().query("..")); + } + + @ParameterizedTest + @CsvSource({ + "404, device_check_not_held", + "409, device_check_spent", + "409, device_check_busy", + "409, credential_unavailable", + "422, bits_required", + "422, apple_rejected", + }) + void typesEachErrorAndNeverRetriesIt(int status, String code) throws IOException { + try (FakeApi api = FakeApi.json(status, error(status, code, null))) { + FingerlyApiException e = assertThrows(FingerlyApiException.class, + () -> client(api.url()).deviceCheck().write(REQUEST_ID, DeviceCheckWrite.builder().bit1(true).build())); + + assertEquals(status, e.getStatus()); + assertEquals(code, e.getCode()); + assertEquals("device_check_not_held".equals(code), e.isDeviceCheckNotHeld()); + assertEquals("device_check_spent".equals(code), e.isDeviceCheckSpent()); + assertEquals("device_check_busy".equals(code), e.isDeviceCheckBusy()); + assertEquals("credential_unavailable".equals(code), e.isCredentialUnavailable()); + assertEquals("bits_required".equals(code), e.isBitsRequired()); + assertEquals("apple_rejected".equals(code), e.isAppleRejected()); + assertFalse(e.isDeviceCheckUnavailable()); + assertEquals(1, api.callCount()); + } + } + + @ParameterizedTest + @CsvSource({"429", "500", "502"}) + void doesNotRetryOtherFailures(int status) throws IOException { + try (FakeApi api = FakeApi.json(status, error(status, "other", null))) { + assertThrows(FingerlyApiException.class, () -> client(api.url()).deviceCheck().query(REQUEST_ID)); + assertEquals(1, api.callCount()); + } + } + + @Test + void readsApplesStatus() throws IOException { + String details = "[{\"field\":\"apple_status\",\"location\":\"upstream\",\"issue\":\"400\"}]"; + try (FakeApi api = FakeApi.json(422, error(422, "apple_rejected", details))) { + FingerlyApiException e = assertThrows(FingerlyApiException.class, + () -> client(api.url()).deviceCheck().query(REQUEST_ID)); + + assertTrue(e.isAppleRejected()); + assertEquals(Integer.valueOf(400), e.getAppleStatus()); + } + try (FakeApi api = FakeApi.json(409, error(409, "device_check_spent", null))) { + FingerlyApiException e = assertThrows(FingerlyApiException.class, + () -> client(api.url()).deviceCheck().query(REQUEST_ID)); + assertNull(e.getAppleStatus()); + } + } + + @Test + void retriesA503WithTheSameRequest() throws IOException { + AtomicInteger calls = new AtomicInteger(); + try (FakeApi api = FakeApi.start(exchange -> { + if (calls.incrementAndGet() == 1) { + FakeApi.respond(exchange, 503, error(503, "device_check_unavailable", null)); + } else { + FakeApi.respond(exchange, 200, "{\"bit0\":true,\"bit1\":false}"); + } + })) { + DeviceCheckWriteResult result = client(api.url()).deviceCheck() + .write(REQUEST_ID, DeviceCheckWrite.builder().bit0(true).bit1(false).build()); + + assertEquals(Optional.of(true), result.getBit0()); + assertEquals(2, api.callCount()); + FakeApi.Recorded first = api.requests.get(0); + FakeApi.Recorded second = api.requests.get(1); + assertEquals(first.method, second.method); + assertEquals(first.path, second.path); + assertArrayEquals(first.body, second.body); + } + } + + @Test + void givesUpAfterThreeAttempts() throws IOException { + try (FakeApi api = FakeApi.json(503, error(503, "device_check_unavailable", null))) { + FingerlyApiException e = assertThrows(FingerlyApiException.class, + () -> client(api.url()).deviceCheck().query(REQUEST_ID)); + + assertTrue(e.isDeviceCheckUnavailable()); + assertEquals(3, api.callCount()); + for (FakeApi.Recorded sent : api.requests) { + assertEquals(DEVICE_CHECK + "/query", sent.path); + } + } + } + + @Test + void retriesATransportFailure() throws IOException { + AtomicInteger calls = new AtomicInteger(); + try (FakeApi api = FakeApi.start(exchange -> { + if (calls.incrementAndGet() > 1) { + FakeApi.respond(exchange, 200, "{\"found\":false,\"bit0\":false,\"bit1\":false}"); + } + // The first exchange closes without an answer. + })) { + DeviceCheckBits bits = client(api.url()).deviceCheck().query(REQUEST_ID); + + assertFalse(bits.isFound()); + assertEquals(2, api.callCount()); + } + } + + @Test + void stopsRetryingWhenNothingListens() throws IOException { + int port; + try (ServerSocket closed = new ServerSocket(0)) { + port = closed.getLocalPort(); + } + assertThrows(FingerlyConnectionException.class, + () -> client("http://127.0.0.1:" + port).deviceCheck().query(REQUEST_ID)); + } +} diff --git a/fingerly-server/src/test/java/io/fingerly/server/EventsServiceTest.java b/fingerly-server/src/test/java/io/fingerly/server/EventsServiceTest.java index 0325b83..91fb92b 100644 --- a/fingerly-server/src/test/java/io/fingerly/server/EventsServiceTest.java +++ b/fingerly-server/src/test/java/io/fingerly/server/EventsServiceTest.java @@ -20,7 +20,7 @@ class EventsServiceTest { private static FingerlyClient client(FakeApi api) { - return FingerlyClient.builder().secretKey(SECRET_KEY).endpoint(api.url()).build(); + return FingerlyClient.builder().secretKey(SECRET_KEY).allowInsecureLoopback(true).endpoint(api.url()).build(); } @Test @@ -272,7 +272,7 @@ void completesAsyncReadsWithTheSdkErrors() throws Exception { } })) { FingerlyClient slow = FingerlyClient.builder().secretKey(SECRET_KEY) - .endpoint(api.url()).timeout(Duration.ofMillis(200)).build(); + .allowInsecureLoopback(true).endpoint(api.url()).timeout(Duration.ofMillis(200)).build(); java.util.concurrent.ExecutionException failure = assertThrows( java.util.concurrent.ExecutionException.class, () -> slow.events().getAsync(REQUEST_ID).get(3, java.util.concurrent.TimeUnit.SECONDS)); diff --git a/fingerly-server/src/test/java/io/fingerly/server/EventsVerifyTest.java b/fingerly-server/src/test/java/io/fingerly/server/EventsVerifyTest.java new file mode 100644 index 0000000..c9c35fd --- /dev/null +++ b/fingerly-server/src/test/java/io/fingerly/server/EventsVerifyTest.java @@ -0,0 +1,467 @@ +package io.fingerly.server; + +import static io.fingerly.server.Fixtures.REQUEST_ID; +import static io.fingerly.server.Fixtures.SECRET_KEY; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertSame; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import com.sun.net.httpserver.HttpExchange; +import java.io.IOException; +import java.time.Duration; +import java.time.Instant; +import java.util.concurrent.CompletionException; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; + +class EventsVerifyTest { + + private static final String EVENT = "/api/v1/events/" + REQUEST_ID; + private static final String CONSUME = EVENT + "/consume"; + private static final String CONSUMED = + "{\"request_id\":\"" + REQUEST_ID + "\",\"consumed_at\":\"2026-11-17T10:00:00Z\",\"first\":true}"; + private static final String ALREADY_LITERAL = + "{\"code\":\"event_already_consumed\",\"consumed_at\":\"2026-11-17T10:00:00Z\"}"; + private static final String ALREADY_ENVELOPE = + "{\"error\":{\"code\":\"event_already_consumed\",\"message\":\"m\",\"status\":409," + + "\"details\":[{\"field\":\"consumed_at\",\"issue\":\"2026-11-17T10:00:00Z\"," + + "\"location\":\"body\"}]}}"; + + private static FingerlyClient client(String url) { + return FingerlyClient.builder() + .secretKey(SECRET_KEY) + .allowInsecureLoopback(true) + .endpoint(url) + .timeout(Duration.ofSeconds(2)) + .build(); + } + + private static String error(int status, String code) { + return "{\"error\":{\"status\":" + status + ",\"code\":\"" + code + "\",\"message\":\"m\"}}"; + } + + /** An event whose fields are given as JSON members, aged {@code ageSeconds}. */ + private static String event(long ageSeconds, String members) { + return event("\"" + Instant.now().minusSeconds(ageSeconds) + "\"", members); + } + + private static String event(String occurredAt, String members) { + return "{\"request_id\":\"" + REQUEST_ID + "\"" + + (occurredAt == null ? "" : ",\"occurred_at\":" + occurredAt) + + (members.isEmpty() ? "" : "," + members) + "}"; + } + + private static final String GOOD = + "\"visitor_id\":\"v1\",\"tag\":\"checkout\",\"suspect_score\":12,\"suspect_level\":\"low\""; + + /** A server answering the read with {@code read} and the consume with {@code consume}. */ + private static FakeApi api(int readStatus, String read, int consumeStatus, String consume) + throws IOException { + return FakeApi.start(exchange -> answer(exchange, readStatus, read, consumeStatus, consume)); + } + + private static void answer(HttpExchange exchange, int readStatus, String read, + int consumeStatus, String consume) { + if (exchange.getRequestURI().getRawPath().endsWith("/consume")) { + FakeApi.respond(exchange, consumeStatus, consume); + } else { + FakeApi.respond(exchange, readStatus, read); + } + } + + private static long consumeCount(FakeApi api) { + return api.requests.stream().filter(r -> r.path.endsWith("/consume")).count(); + } + + // consume + + @Test + void consumeDecodesTheFirstConsume() throws IOException { + try (FakeApi api = FakeApi.json(200, CONSUMED)) { + ConsumeResult result = client(api.url()).events().consume(REQUEST_ID); + + assertEquals(REQUEST_ID, result.getRequestId()); + assertEquals("2026-11-17T10:00:00Z", result.getConsumedAt()); + assertTrue(result.isFirst()); + FakeApi.Recorded sent = api.first(); + assertEquals("POST", sent.method); + assertEquals(CONSUME, sent.path); + assertEquals(SECRET_KEY, sent.header("X-API-Key")); + assertEquals(0, sent.body.length); + } + } + + @Test + void consumeEscapesTheRequestId() throws IOException { + try (FakeApi api = FakeApi.json(200, CONSUMED)) { + client(api.url()).events().consume("a b/c"); + + assertEquals("/api/v1/events/a%20b%2Fc/consume", api.first().path); + } + } + + @ParameterizedTest + @ValueSource(strings = {ALREADY_LITERAL, ALREADY_ENVELOPE}) + void consumeTwiceThrowsTheTypedError(String body) throws IOException { + try (FakeApi api = FakeApi.json(409, body)) { + EventAlreadyConsumedException e = assertThrows(EventAlreadyConsumedException.class, + () -> client(api.url()).events().consume(REQUEST_ID)); + + assertEquals(409, e.getStatus()); + assertEquals("event_already_consumed", e.getCode()); + assertTrue(e.isEventAlreadyConsumed()); + assertEquals("2026-11-17T10:00:00Z", e.getConsumedAt()); + assertFalse(e.getMessage().contains("2026-11-17")); + assertEquals(1, api.callCount()); + } + } + + @Test + void consumeAlreadyConsumedWithoutATime() throws IOException { + try (FakeApi api = FakeApi.json(409, error(409, "event_already_consumed"))) { + EventAlreadyConsumedException e = assertThrows(EventAlreadyConsumedException.class, + () -> client(api.url()).events().consume(REQUEST_ID)); + + assertNull(e.getConsumedAt()); + } + } + + @Test + void consumeOfAnUnknownEventThrowsTheApiError() throws IOException { + try (FakeApi api = FakeApi.json(404, error(404, "event_not_found"))) { + FingerlyApiException e = assertThrows(FingerlyApiException.class, + () -> client(api.url()).events().consume(REQUEST_ID)); + + assertFalse(e instanceof EventAlreadyConsumedException); + assertTrue(e.isEventNotFound()); + } + } + + @Test + void consumeIsNeverRetried() throws IOException { + try (FakeApi api = FakeApi.json(503, error(503, "unavailable"))) { + assertThrows(FingerlyApiException.class, + () -> client(api.url()).events().consume(REQUEST_ID)); + + assertEquals(1, api.callCount()); + } + } + + @ParameterizedTest + @ValueSource(strings = {"", ".", ".."}) + void consumeRefusesABadRequestIdWithoutARequest(String requestId) throws IOException { + try (FakeApi api = FakeApi.json(200, CONSUMED)) { + FingerlyClient fingerly = client(api.url()); + assertThrows(IllegalArgumentException.class, () -> fingerly.events().consume(requestId)); + assertThrows(IllegalArgumentException.class, () -> fingerly.events().consume(null)); + + assertEquals(0, api.callCount()); + } + } + + // verify + + @Test + void verifyPassesAndConsumesOnce() throws IOException { + try (FakeApi api = api(200, event(10, GOOD), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events() + .verify(REQUEST_ID, VerifyOptions.builder().tag("checkout").build()); + + assertTrue(result.isOk()); + assertEquals("", result.getReason()); + assertEquals("v1", result.getEvent().getVisitorId()); + assertEquals(2, api.callCount()); + assertEquals("GET", api.requests.get(0).method); + assertEquals(EVENT, api.requests.get(0).path); + assertEquals("POST", api.requests.get(1).method); + assertEquals(CONSUME, api.requests.get(1).path); + assertEquals(SECRET_KEY, api.requests.get(1).header("X-API-Key")); + } + } + + @Test + void verifyLeavesScorePolicyToTheCaller() throws IOException { + String high = "\"visitor_id\":\"v1\",\"suspect_score\":99,\"suspect_level\":\"high\"," + + "\"triggers\":[{\"signal\":\"tampering\"}]"; + try (FakeApi api = api(200, event(10, high), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events().verify(REQUEST_ID); + + assertTrue(result.isOk()); + assertEquals("high", result.getEvent().getSuspectLevel()); + } + } + + @Test + void verifyAsyncPasses() throws IOException { + try (FakeApi api = api(200, event(10, GOOD), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events().verifyAsync(REQUEST_ID).join(); + + assertTrue(result.isOk()); + assertEquals(1, consumeCount(api)); + } + } + + @Test + void verifyWithoutConsumeNeverPosts() throws IOException { + try (FakeApi api = api(200, event(10, GOOD), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events() + .verify(REQUEST_ID, VerifyOptions.builder().consume(false).build()); + + assertTrue(result.isOk()); + assertEquals(1, api.callCount()); + assertEquals(0, consumeCount(api)); + } + } + + @Test + void verifyNotFound() throws IOException { + try (FakeApi api = api(404, error(404, "event_not_found"), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events().verify(REQUEST_ID); + + assertFalse(result.isOk()); + assertEquals(VerifyResult.REASON_NOT_FOUND, result.getReason()); + assertEquals("not_found", result.getReason()); + assertNull(result.getEvent()); + assertEquals(0, consumeCount(api)); + } + } + + @Test + void verifyTooOld() throws IOException { + try (FakeApi api = api(200, event(4000, GOOD), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events().verify(REQUEST_ID); + + assertEquals("too_old", result.getReason()); + assertNotNull(result.getEvent()); + assertEquals(0, consumeCount(api)); + } + } + + @Test + void verifyDefaultsToFiveMinutes() throws IOException { + try (FakeApi api = api(200, event(200, GOOD), 200, CONSUMED)) { + assertTrue(client(api.url()).events().verify(REQUEST_ID).isOk()); + } + try (FakeApi api = api(200, event(400, GOOD), 200, CONSUMED)) { + assertEquals("too_old", client(api.url()).events().verify(REQUEST_ID).getReason()); + assertEquals(0, consumeCount(api)); + } + } + + @Test + void verifyHonoursMaxAge() throws IOException { + try (FakeApi api = api(200, event(200, GOOD), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events() + .verify(REQUEST_ID, VerifyOptions.builder().maxAgeSeconds(60).build()); + + assertEquals("too_old", result.getReason()); + assertEquals(0, consumeCount(api)); + } + } + + @ParameterizedTest + @ValueSource(strings = {"\"yesterday\"", "\"\"", "12", "null"}) + void verifyUnreadableTimeIsTooOld(String occurredAt) throws IOException { + try (FakeApi api = api(200, event(occurredAt, GOOD), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events().verify(REQUEST_ID); + + assertEquals("too_old", result.getReason()); + assertNotNull(result.getEvent()); + assertEquals(0, consumeCount(api)); + } + } + + @Test + void verifyMissingTimeIsTooOld() throws IOException { + try (FakeApi api = api(200, event(null, GOOD), 200, CONSUMED)) { + assertEquals("too_old", client(api.url()).events().verify(REQUEST_ID).getReason()); + assertEquals(0, consumeCount(api)); + } + } + + @Test + void verifyFutureTimePasses() throws IOException { + try (FakeApi api = api(200, event(-600, GOOD), 200, CONSUMED)) { + assertTrue(client(api.url()).events().verify(REQUEST_ID).isOk()); + } + } + + @Test + void verifyTagMismatch() throws IOException { + try (FakeApi api = api(200, event(10, GOOD), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events() + .verify(REQUEST_ID, VerifyOptions.builder().tag("login").build()); + + assertEquals("tag_mismatch", result.getReason()); + assertEquals(0, consumeCount(api)); + } + } + + @Test + void verifyUntaggedEventMismatchesAnyTag() throws IOException { + String untagged = "\"visitor_id\":\"v1\",\"suspect_score\":12"; + try (FakeApi api = api(200, event(10, untagged), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events() + .verify(REQUEST_ID, VerifyOptions.builder().tag("checkout").build()); + + assertEquals("tag_mismatch", result.getReason()); + assertEquals(0, consumeCount(api)); + } + } + + @Test + void verifyWithoutATagSkipsTheTagCheck() throws IOException { + try (FakeApi api = api(200, event(10, GOOD), 200, CONSUMED)) { + assertTrue(client(api.url()).events().verify(REQUEST_ID).isOk()); + } + } + + @ParameterizedTest + @ValueSource(strings = {"\"visitor_id\":\"\",\"suspect_score\":12", "\"suspect_score\":12"}) + void verifyUnidentifiable(String members) throws IOException { + try (FakeApi api = api(200, event(10, members), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events().verify(REQUEST_ID); + + assertEquals("unidentifiable", result.getReason()); + assertEquals(0, consumeCount(api)); + } + } + + @ParameterizedTest + @ValueSource(strings = {"\"visitor_id\":\"v1\",\"suspect_score\":null", "\"visitor_id\":\"v1\""}) + void verifyUnscored(String members) throws IOException { + try (FakeApi api = api(200, event(10, members), 200, CONSUMED)) { + VerifyResult result = client(api.url()).events().verify(REQUEST_ID); + + assertEquals("unscored", result.getReason()); + assertEquals(0, consumeCount(api)); + } + } + + @ParameterizedTest + @ValueSource(strings = {ALREADY_LITERAL, ALREADY_ENVELOPE}) + void verifyAlreadyConsumed(String body) throws IOException { + try (FakeApi api = api(200, event(10, GOOD), 409, body)) { + VerifyResult result = client(api.url()).events().verify(REQUEST_ID); + + assertFalse(result.isOk()); + assertEquals("already_consumed", result.getReason()); + assertEquals("v1", result.getEvent().getVisitorId()); + assertEquals(1, consumeCount(api)); + } + } + + @Test + void verifyNotFoundOnConsume() throws IOException { + try (FakeApi api = api(200, event(10, GOOD), 404, error(404, "event_not_found"))) { + VerifyResult result = client(api.url()).events().verify(REQUEST_ID); + + assertEquals("not_found", result.getReason()); + assertNull(result.getEvent()); + } + } + + @Test + void verifyRaisesOtherConsumeErrors() throws IOException { + try (FakeApi api = api(200, event(10, GOOD), 500, error(500, "internal"))) { + FingerlyApiException e = assertThrows(FingerlyApiException.class, + () -> client(api.url()).events().verify(REQUEST_ID)); + + assertEquals(500, e.getStatus()); + assertEquals(1, consumeCount(api)); + } + } + + @Test + void verifyChecksInOrder() throws IOException { + String bad = "\"visitor_id\":\"\",\"tag\":\"other\""; + VerifyOptions tagged = VerifyOptions.builder().tag("checkout").build(); + try (FakeApi api = api(200, event(4000, bad), 200, CONSUMED)) { + assertEquals("too_old", client(api.url()).events().verify(REQUEST_ID, tagged).getReason()); + assertEquals(0, consumeCount(api)); + } + try (FakeApi api = api(200, event(10, bad), 200, CONSUMED)) { + assertEquals("tag_mismatch", client(api.url()).events().verify(REQUEST_ID, tagged).getReason()); + assertEquals(0, consumeCount(api)); + } + try (FakeApi api = api(200, event(10, bad), 200, CONSUMED)) { + assertEquals("unidentifiable", client(api.url()).events().verify(REQUEST_ID).getReason()); + assertEquals(0, consumeCount(api)); + } + } + + @Test + void verifyRaisesAPendingRead() throws IOException { + try (FakeApi api = FakeApi.start(exchange -> { + exchange.getResponseHeaders().add("Retry-After", "2"); + FakeApi.respond(exchange, 404, error(404, "event_pending")); + })) { + FingerlyApiException e = assertThrows(FingerlyApiException.class, + () -> client(api.url()).events().verify(REQUEST_ID)); + + assertTrue(e.isPending()); + assertEquals(Integer.valueOf(2), e.getRetryAfter()); + assertEquals(0, consumeCount(api)); + } + } + + @ParameterizedTest + @ValueSource(ints = {401, 429, 500, 503}) + void verifyRaisesOtherReadErrors(int status) throws IOException { + try (FakeApi api = api(status, error(status, "e"), 200, CONSUMED)) { + FingerlyApiException e = assertThrows(FingerlyApiException.class, + () -> client(api.url()).events().verify(REQUEST_ID)); + + assertEquals(status, e.getStatus()); + assertEquals(1, api.callCount()); + } + } + + @Test + void verifyAsyncRaisesTheSameErrors() throws IOException { + try (FakeApi api = api(404, error(404, "event_pending"), 200, CONSUMED)) { + CompletionException e = assertThrows(CompletionException.class, + () -> client(api.url()).events().verifyAsync(REQUEST_ID).join()); + + assertInstanceOf(FingerlyApiException.class, e.getCause()); + } + } + + @Test + void verifyRefusesBadInputBeforeAnyRequest() throws IOException { + try (FakeApi api = api(200, event(10, GOOD), 200, CONSUMED)) { + FingerlyClient fingerly = client(api.url()); + assertThrows(IllegalArgumentException.class, () -> fingerly.events().verify("")); + assertThrows(IllegalArgumentException.class, () -> fingerly.events().verify("..")); + assertThrows(IllegalArgumentException.class, () -> fingerly.events().verifyAsync(".")); + + assertEquals(0, api.callCount()); + } + } + + @Test + void verifyOptionsRejectInvalidValues() { + assertThrows(IllegalArgumentException.class, + () -> VerifyOptions.builder().maxAgeSeconds(0).build()); + assertThrows(IllegalArgumentException.class, + () -> VerifyOptions.builder().maxAgeSeconds(-5).build()); + assertThrows(IllegalArgumentException.class, + () -> VerifyOptions.builder().tag("").build()); + } + + @Test + void verifyOptionsDefaults() { + VerifyOptions defaults = VerifyOptions.defaults(); + + assertEquals(300, defaults.getMaxAgeSeconds()); + assertNull(defaults.getTag()); + assertTrue(defaults.isConsume()); + assertSame(defaults, VerifyOptions.defaults()); + } +} diff --git a/fingerly-server/src/test/java/io/fingerly/server/FingerlyClientTest.java b/fingerly-server/src/test/java/io/fingerly/server/FingerlyClientTest.java index 6762249..e3961d9 100644 --- a/fingerly-server/src/test/java/io/fingerly/server/FingerlyClientTest.java +++ b/fingerly-server/src/test/java/io/fingerly/server/FingerlyClientTest.java @@ -20,7 +20,7 @@ class FingerlyClientTest { private static FingerlyClient client(FakeApi api) { - return FingerlyClient.builder().secretKey(SECRET_KEY).endpoint(api.url()).build(); + return FingerlyClient.builder().secretKey(SECRET_KEY).allowInsecureLoopback(true).endpoint(api.url()).build(); } @ParameterizedTest @@ -52,6 +52,47 @@ void rejectsInvalidEndpoints() { } } + @ParameterizedTest + @ValueSource(strings = { + "http://api.example.invalid", + "http://192.0.2.10:8080", + "http://127.0.0.2", + "https://api.example.invalid/prefix", + "https://api.example.invalid/a/../b", + "https://api.example.invalid?", + "https://api.example.invalid#", + "https://api.example.invalid\n", + "https://api.example.invalid ", + "https://api.example.invalid\r\nx-injected: 1", + "https://api.example.invalid:0", + }) + void rejectsAnEndpointThatIsNotABareHttpsOrigin(String endpoint) { + for (boolean loopbackFlag : new boolean[] {false, true}) { + IllegalArgumentException refused = assertThrows(IllegalArgumentException.class, + () -> FingerlyClient.builder().secretKey(SECRET_KEY) + .allowInsecureLoopback(loopbackFlag).endpoint(endpoint).build(), + endpoint); + assertFalse(refused.getMessage().contains("example.invalid"), refused.getMessage()); + assertFalse(refused.getMessage().contains(SECRET_KEY), refused.getMessage()); + } + } + + @ParameterizedTest + @ValueSource(strings = {"http://127.0.0.1:9", "http://localhost:9", "http://[::1]:9"}) + void acceptsAPlainHttpLoopbackEndpointOnlyWithTheDevelopmentFlag(String endpoint) { + assertThrows(IllegalArgumentException.class, + () -> FingerlyClient.builder().secretKey(SECRET_KEY).endpoint(endpoint).build()); + FingerlyClient.builder().secretKey(SECRET_KEY) + .allowInsecureLoopback(true).endpoint(endpoint).build(); + } + + @Test + void storesOnlyTheParsedPartsOfAnEndpoint() { + assertTrue(FingerlyClient.builder().secretKey(SECRET_KEY) + .endpoint("https://API.example.invalid:8443/").build().toString() + .contains("baseUrl=https://api.example.invalid:8443/api/v1")); + } + @ParameterizedTest @ValueSource(strings = {"", "1", "v0", "v1/../v2", "v2?key=value", "v2\n"}) void rejectsInvalidApiVersions(String apiVersion) { @@ -95,7 +136,7 @@ void appliesTheApiVersionToEveryRequest() throws IOException { try (FakeApi api = FakeApi.json(200, Fixtures.event())) { FingerlyClient fingerly = FingerlyClient.builder() .secretKey(SECRET_KEY) - .endpoint(api.url()) + .allowInsecureLoopback(true).endpoint(api.url()) .apiVersion("v2") .build(); fingerly.events().get(REQUEST_ID); @@ -190,7 +231,7 @@ void timesOutWhileReceivingTheBody() throws IOException { })) { FingerlyClient fingerly = FingerlyClient.builder() .secretKey(SECRET_KEY) - .endpoint(api.url()) + .allowInsecureLoopback(true).endpoint(api.url()) .timeout(Duration.ofMillis(200)) .build(); assertThrows(FingerlyTimeoutException.class, () -> fingerly.events().get(REQUEST_ID)); @@ -217,7 +258,7 @@ void closesTheConnectionWhenTheDeadlinePasses() throws Exception { })) { FingerlyClient fingerly = FingerlyClient.builder() .secretKey(SECRET_KEY) - .endpoint(api.url()) + .allowInsecureLoopback(true).endpoint(api.url()) .timeout(Duration.ofMillis(300)) .build(); assertThrows(FingerlyTimeoutException.class, () -> fingerly.events().get(REQUEST_ID)); @@ -346,7 +387,7 @@ void reportsAnUnreachableApi() { // Port 1 on loopback refuses immediately. FingerlyClient fingerly = FingerlyClient.builder() .secretKey(SECRET_KEY) - .endpoint("http://127.0.0.1:1") + .allowInsecureLoopback(true).endpoint("http://127.0.0.1:1") .build(); assertThrows(FingerlyConnectionException.class, () -> fingerly.events().get(REQUEST_ID)); } diff --git a/fingerly-server/src/test/java/io/fingerly/server/FingerlyProxyTest.java b/fingerly-server/src/test/java/io/fingerly/server/FingerlyProxyTest.java index 98bc7d3..edd652f 100644 --- a/fingerly-server/src/test/java/io/fingerly/server/FingerlyProxyTest.java +++ b/fingerly-server/src/test/java/io/fingerly/server/FingerlyProxyTest.java @@ -28,6 +28,7 @@ private static FingerlyProxy proxy(FakeApi upstream, Consumer request.header("CF-Connecting-IP")); if (tweak != null) { @@ -68,15 +69,12 @@ private static String headerOf(ProxyResponse response, String name) { } @Test - @SuppressWarnings("deprecation") // The deprecated secretKey path still has to be tested. void rejectsInvalidOptions() { FingerlyProxy.ClientIp clientIp = request -> "203.0.113.5"; assertThrows(IllegalArgumentException.class, () -> FingerlyProxy.builder().clientIp(clientIp).build(), "no key"); assertThrows(IllegalArgumentException.class, () -> FingerlyProxy.builder().proxyKey(PROXY_KEY).build(), "no clientIp"); - assertThrows(IllegalArgumentException.class, () -> FingerlyProxy.builder() - .proxyKey(PROXY_KEY).secretKey(SECRET_KEY).clientIp(clientIp).build(), "both keys"); assertThrows(IllegalArgumentException.class, () -> FingerlyProxy.builder() .proxyKey(SECRET_KEY).clientIp(clientIp).build(), "a secret key as proxy key"); for (String path : new String[] {"k7x2qm4a", "/", "///", " ", "/k7x2qm4a?a=1", "/k7x2qm4a#a", @@ -92,8 +90,11 @@ void rejectsInvalidOptions() { } @ParameterizedTest - @ValueSource(strings = {"{\"signals\":{}}", "{\"token\":\"t\",\"signals\":{}}"}) - void forwardsEveryCallToTheOneIngestUrl(String body) throws IOException { + @ValueSource(strings = { + "{\"v\":3,\"signals\":{\"elapsed_ms\":142.50,\"raw\":1e-05}}", + "{ \"v\" : 3 , \"signals\" : { \"b\" : 1 , \"a\" : 2 } }", + }) + void forwardsEveryCallVerbatimToTheOneIngestUrl(String body) throws IOException { try (FakeApi upstream = FakeApi.json(200, "{\"request_id\":\"identified\"}")) { ProxyResponse response = proxy(upstream, null) .handle(request("POST", PATH, body, browserHeaders())); @@ -142,6 +143,7 @@ void dropsAVisitorHeaderThatCouldNotBeForwardedInsteadOfFailing() throws IOExcep .method("POST") .path(PATH) .header("X-API-Key", PUBLIC_KEY) + .header("X-Fingerly-SDK-Platform", "ios") .header("CF-Connecting-IP", "203.0.113.5") .header("User-Agent", "Mozilla/5.0\u007f") .header("Origin", "https://shop.test\u0001") @@ -151,7 +153,8 @@ void dropsAVisitorHeaderThatCouldNotBeForwardedInsteadOfFailing() throws IOExcep assertEquals(200, proxy(upstream, null).handle(odd).status()); assertEquals("", upstream.first().header("X-Fingerly-User-Agent")); - assertEquals("", upstream.first().header("X-Fingerly-Origin")); + assertNull(upstream.first().header("X-Fingerly-Origin")); + assertEquals("ios", upstream.first().header("X-Fingerly-Sdk-Platform")); assertNull(upstream.first().header("Idempotency-Key")); assertNull(upstream.first().header("X-Evil")); } @@ -216,6 +219,9 @@ void refusesAnythingButPost() throws IOException { @ParameterizedTest @ValueSource(strings = { "/k7x2qm4a/admin", + "/k7x2qm4a/supplement", + "/k7x2qm4a/events/req_1/supplement", + "/api/v1/events/req_1/supplement", "/k7x2qm4a/api/v1/identify", "/k7x2qm4ab", "/other", @@ -378,7 +384,8 @@ void offersThePeerAddressToTheCallback() throws IOException { .path(PATH) .remoteAddress("[2001:db8::7%3]") .body("{}".getBytes(StandardCharsets.UTF_8)) - .header("X-API-Key", PUBLIC_KEY); + .header("X-API-Key", PUBLIC_KEY) + .header("Origin", "https://shop.test"); assertEquals(200, proxy.handle(direct.build()).status()); assertEquals("2001:db8::7", upstream.first().header("X-Fingerly-Client-Ip")); } @@ -389,6 +396,7 @@ void reportsAnUnreachableUpstreamAsBadGateway() { FingerlyProxy proxy = FingerlyProxy.builder() .proxyKey(PROXY_KEY) .path(PATH) + .allowInsecureLoopback(true) .upstream("http://127.0.0.1:1") .clientIp(request -> "203.0.113.5") .build(); @@ -462,6 +470,7 @@ void logsOneKeyFreeLinePerFailure() { FingerlyProxy proxy = FingerlyProxy.builder() .proxyKey(PROXY_KEY) .path(PATH) + .allowInsecureLoopback(true) .upstream("http://127.0.0.1:1") .clientIp(request -> "203.0.113.5") .log(lines::add) @@ -482,6 +491,7 @@ void aThrowingLogHookStillAnswersBadGateway() { FingerlyProxy proxy = FingerlyProxy.builder() .proxyKey(PROXY_KEY) .path(PATH) + .allowInsecureLoopback(true) .upstream("http://127.0.0.1:1") .clientIp(request -> "203.0.113.5") .log(line -> { @@ -601,20 +611,189 @@ void cancellingHandleAsyncClosesTheUpstreamConnectionQuietly() throws Exception } @Test - @SuppressWarnings("deprecation") // The deprecated secretKey path still has to be tested. - void forwardsTheSecretKeyWithoutAPublicKey() throws IOException { + void builderOffersNoSecretKeyOption() { + for (java.lang.reflect.Method method : FingerlyProxy.Builder.class.getMethods()) { + assertFalse(method.getName().toLowerCase(Locale.ROOT).contains("secret"), + "the proxy builder must not take a secret key: " + method); + } + } + + @ParameterizedTest + @ValueSource(strings = { + "", " ", "fly_px_development_", "fly_px_unknown_test", "fly_px_development_test\n", + "fly_pk_development_test", "fly_sk_development_test", + }) + void buildRequiresAWellFormedProxyKey(String proxyKey) { + FingerlyProxy.ClientIp clientIp = request -> "203.0.113.5"; + assertThrows(IllegalArgumentException.class, + () -> FingerlyProxy.builder().clientIp(clientIp).build(), "no proxy key"); + assertThrows(IllegalArgumentException.class, () -> FingerlyProxy.builder() + .proxyKey(proxyKey).clientIp(clientIp).build(), proxyKey); + } + + @Test + void neverForwardsASecretKeyUpstream() throws IOException { try (FakeApi upstream = FakeApi.json(200, "{}")) { - FingerlyProxy proxy = FingerlyProxy.builder() - .secretKey(SECRET_KEY) + FingerlyProxy proxy = proxy(upstream, null); + + Map withSecret = new java.util.HashMap<>(browserHeaders()); + withSecret.put("X-API-Key", SECRET_KEY); + ProxyResponse refused = proxy.handle(request("POST", PATH, "{}", withSecret)); + assertEquals(401, refused.status()); + assertEquals("text/plain; charset=utf-8", headerOf(refused, "Content-Type")); + assertEquals("no-store", headerOf(refused, "Cache-Control")); + assertEquals(0, upstream.callCount(), "a secret key reached upstream"); + + assertEquals(200, proxy.handle(request("POST", PATH, "{}", browserHeaders())).status()); + assertEquals(1, upstream.callCount()); + assertEquals(PUBLIC_KEY, upstream.first().header("X-Api-Key")); + assertEquals(PROXY_KEY, upstream.first().header("X-Fingerly-Proxy-Key")); + assertFalse(upstream.first().header("X-Api-Key").startsWith("fly_sk_")); + } + } + + @Test + void rejectsOriginlessPostWithMissingOriginAndNeverSynthesizesOrigin() throws IOException { + try (FakeApi upstream = FakeApi.json(200, "{}")) { + ProxyRequest originless = request("POST", PATH, "{}", Map.of( + "X-API-Key", PUBLIC_KEY, + "Host", "shop.test", + "Referer", "https://shop.test/checkout", + "CF-Connecting-IP", "203.0.113.5")); + + ProxyResponse response = proxy(upstream, null).handle(originless); + + assertEquals(400, response.status()); + assertEquals("no-store", headerOf(response, "Cache-Control")); + assertTrue(new String(response.body(), StandardCharsets.UTF_8) + .contains("\"missing_origin\"")); + assertEquals(0, upstream.callCount()); + } + } + + @ParameterizedTest + @ValueSource(strings = { + "web", "iOS", "ANDROID", "ios, android", " ios", "ios ", "electron", "" + }) + void discardsInvalidSdkPlatformAndRefusesWhenOriginIsMissing(String invalidPlatform) + throws IOException { + try (FakeApi upstream = FakeApi.json(200, "{}")) { + ProxyRequest forged = request("POST", PATH, "{}", Map.of( + "X-API-Key", PUBLIC_KEY, + "X-Fingerly-SDK-Platform", invalidPlatform, + "X-Fingerly-Origin", "https://forged.test", + "X-Fingerly-Client-IP", "6.6.6.6", + "X-Fingerly-Proxy-Key", "fly_px_development_forged", + "CF-Connecting-IP", "203.0.113.5")); + + ProxyResponse response = proxy(upstream, null).handle(forged); + + assertEquals(400, response.status()); + assertEquals("no-store", headerOf(response, "Cache-Control")); + assertEquals(0, upstream.callCount()); + } + } + + @ParameterizedTest + @ValueSource(strings = { + "ios", + "android", + "react-native", + "react-native-ios", + "react-native-android", + "flutter", + "flutter-ios", + "flutter-android", + "lynx-ios", + "lynx-android", + "capacitor", + "kotlin-multiplatform", + }) + void restoresValidatedNativePlatformWithoutOriginAndStripsSpoofedFingerlyHeaders( + String platform) throws IOException { + String rawBody = "{\"v\":3,\"signals\":{\"elapsed_ms\":142.50}}"; + try (FakeApi upstream = FakeApi.json(200, "{\"request_id\":\"native-1\"}")) { + ProxyRequest nativeCall = ProxyRequest.builder() + .method("POST") .path(PATH) - .upstream(upstream.url()) - .clientIp(request -> "203.0.113.5") + .header("X-API-Key", PUBLIC_KEY) + .header("x-fingerly-sdk-platform", platform) + .header("x-fingerly-origin", "https://forged.test") + .header("x-fingerly-client-ip", "6.6.6.6") + .header("x-fingerly-proxy-key", "fly_px_development_forged") + .header("x-fingerly-host", "forged.host") + .header("x-fingerly-edge-tls-version", "TLSv1.0") + .header("Host", "app.shop.test") + .header("Idempotency-Key", "idem_0199a963e40070008000") + .header("CF-Connecting-IP", "203.0.113.5") + .body(rawBody.getBytes(StandardCharsets.UTF_8)) .build(); - assertEquals(200, proxy.handle(request("POST", PATH, "{}", Map.of())).status()); - assertEquals(SECRET_KEY, upstream.first().header("X-Api-Key")); - assertNull(upstream.first().header("X-Fingerly-Proxy-Key")); - assertEquals("/api/v1/ingest", upstream.first().path); + ProxyResponse response = proxy(upstream, null).handle(nativeCall); + + assertEquals(200, response.status()); + FakeApi.Recorded forwarded = upstream.first(); + assertEquals(platform, forwarded.header("X-Fingerly-Sdk-Platform")); + assertNull(forwarded.header("X-Fingerly-Origin")); + assertEquals("203.0.113.5", forwarded.header("X-Fingerly-Client-Ip")); + assertEquals(PROXY_KEY, forwarded.header("X-Fingerly-Proxy-Key")); + assertEquals("app.shop.test", forwarded.header("X-Fingerly-Host")); + assertNull(forwarded.header("X-Fingerly-Edge-Tls-Version")); + assertEquals("idem_0199a963e40070008000", forwarded.header("Idempotency-Key")); + assertEquals(rawBody, forwarded.bodyText()); + } + } + + @Test + void forwardsBrowserFetchMetadataAndClientHintsAndSupportsCors() throws IOException { + try (FakeApi upstream = FakeApi.start(exchange -> { + exchange.getResponseHeaders().add("Content-Type", "application/json"); + exchange.getResponseHeaders().add("X-Request-Id", "req-409"); + exchange.getResponseHeaders().add("Retry-After", "1"); + FakeApi.respond(exchange, 409, "{\"error\":{\"code\":\"request_in_progress\"}}"); + })) { + FingerlyProxy corsProxy = proxy(upstream, + b -> b.allowedOrigins("https://app.shop.test")); + + ProxyResponse preflight = corsProxy.handle(request("OPTIONS", PATH, "", + Map.of("Origin", "https://app.shop.test"))); + assertEquals(204, preflight.status()); + assertEquals("https://app.shop.test", + headerOf(preflight, "Access-Control-Allow-Origin")); + assertTrue(headerOf(preflight, "Access-Control-Allow-Headers") + .contains("x-fingerly-sdk-platform")); + assertTrue(headerOf(preflight, "Access-Control-Expose-Headers") + .contains("retry-after")); + + ProxyRequest browser = ProxyRequest.builder() + .method("POST") + .path(PATH) + .header("X-API-Key", PUBLIC_KEY) + .header("Origin", "https://app.shop.test") + .header("CF-Connecting-IP", "203.0.113.5") + .header("Idempotency-Key", "idem_browser_0199a963e400") + .header("Sec-Fetch-Site", "same-site") + .header("Sec-Fetch-Mode", "cors") + .header("Sec-Fetch-Dest", "empty") + .header("Sec-CH-UA", "\"Chromium\";v=\"130\"") + .header("Sec-CH-UA-Mobile", "?0") + .header("Sec-CH-UA-Platform", "\"macOS\"") + .body("{\"v\":3,\"signals\":{}}".getBytes(StandardCharsets.UTF_8)) + .build(); + + ProxyResponse response = corsProxy.handle(browser); + assertEquals(409, response.status()); + assertEquals("1", headerOf(response, "Retry-After")); + assertEquals("https://app.shop.test", + headerOf(response, "Access-Control-Allow-Origin")); + + FakeApi.Recorded forwarded = upstream.first(); + assertEquals("same-site", forwarded.header("Sec-Fetch-Site")); + assertEquals("cors", forwarded.header("Sec-Fetch-Mode")); + assertEquals("empty", forwarded.header("Sec-Fetch-Dest")); + assertEquals("\"Chromium\";v=\"130\"", forwarded.header("Sec-Ch-Ua")); + assertEquals("?0", forwarded.header("Sec-Ch-Ua-Mobile")); + assertEquals("\"macOS\"", forwarded.header("Sec-Ch-Ua-Platform")); } } } diff --git a/fingerly-server/src/test/java/io/fingerly/server/ProxyHardeningTest.java b/fingerly-server/src/test/java/io/fingerly/server/ProxyHardeningTest.java new file mode 100644 index 0000000..3fd1b14 --- /dev/null +++ b/fingerly-server/src/test/java/io/fingerly/server/ProxyHardeningTest.java @@ -0,0 +1,367 @@ +package io.fingerly.server; + +import static io.fingerly.server.Fixtures.PROXY_KEY; +import static io.fingerly.server.Fixtures.PUBLIC_KEY; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.net.InetAddress; +import java.net.ServerSocket; +import java.net.Socket; +import java.net.http.HttpClient; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.atomic.AtomicInteger; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; + +/** + * Channel choice, upstream validation, redirect, status range, exact method + * and failure log rules of the proxy. Mock upstreams listen on 127.0.0.1 only. + */ +class ProxyHardeningTest { + + private static final String PATH = "/k7x2qm4a"; + private static final String BODY = "{\"v\":3}"; + + private static FingerlyProxy.Builder builder(String upstream) { + return FingerlyProxy.builder() + .proxyKey(PROXY_KEY) + .path(PATH) + .allowInsecureLoopback(true) + .upstream(upstream) + .clientIp(request -> "203.0.113.5"); + } + + private static ProxyRequest request(String method, String path, Map headers) { + ProxyRequest.Builder b = ProxyRequest.builder() + .method(method) + .path(path) + .body(new ByteArrayInputStream(BODY.getBytes(StandardCharsets.UTF_8))) + .contentLength(BODY.length()); + headers.forEach(b::header); + return b.build(); + } + + private static Map browser() { + Map h = new LinkedHashMap<>(); + h.put("X-API-Key", PUBLIC_KEY); + h.put("User-Agent", "Firefox"); + h.put("Origin", "https://shop.test"); + return h; + } + + private static String headerOf(ProxyResponse response, String name) { + for (Map.Entry header : response.headers()) { + if (header.getKey().equalsIgnoreCase(name)) { + return header.getValue(); + } + } + return null; + } + + // BE-16: one channel only, and the page origin wins. + + @Test + void bothChannelHeadersForwardTheOriginOnly() throws IOException { + try (FakeApi upstream = FakeApi.json(200, "{}")) { + Map h = browser(); + h.put("X-Fingerly-SDK-Platform", "ios"); + ProxyResponse response = builder(upstream.url()).build().handle(request("POST", PATH, h)); + assertEquals(200, response.status()); + assertEquals("https://shop.test", upstream.first().header("X-Fingerly-Origin")); + assertNull(upstream.first().header("X-Fingerly-SDK-Platform")); + } + } + + @Test + void aNativeCallWithoutOriginForwardsThePlatformOnly() throws IOException { + try (FakeApi upstream = FakeApi.json(200, "{}")) { + Map h = new LinkedHashMap<>(); + h.put("X-API-Key", PUBLIC_KEY); + h.put("X-Fingerly-SDK-Platform", "android"); + assertEquals(200, builder(upstream.url()).build().handle(request("POST", PATH, h)).status()); + assertEquals("android", upstream.first().header("X-Fingerly-SDK-Platform")); + assertNull(upstream.first().header("X-Fingerly-Origin")); + } + } + + // BE-16 and PRX-05: the upstream option is a bare https origin. + + @ParameterizedTest + @ValueSource(strings = { + "http://upstream.invalid", + "http://192.0.2.10:8080", + "http://127.0.0.2", + "http://localhost.invalid", + "https://upstream.invalid/prefix", + "https://upstream.invalid/a/../b", + "https://upstream.invalid//", + "https://upstream.invalid?", + "https://upstream.invalid?a=1", + "https://upstream.invalid#", + "https://upstream.invalid#f", + "https://user:pass@upstream.invalid", + "https://upstream.invalid\n", + "https://upstream.invalid ", + " https://upstream.invalid", + "https://upstream.invalid\r\nx-injected: 1", + "https://upstream.invalid\t", + "ftp://upstream.invalid", + "upstream.invalid", + "https://", + "https://upstream.invalid:0", + "", + }) + void refusesAnUpstreamThatIsNotABareHttpsOrigin(String value) { + IllegalArgumentException refused = assertThrows(IllegalArgumentException.class, + () -> builder(value).build(), value); + assertFalse(refused.getMessage().contains("upstream.invalid"), refused.getMessage()); + } + + @ParameterizedTest + @ValueSource(strings = {"http://127.0.0.1:9", "http://localhost:9", "http://[::1]:9"}) + void refusesAPlainHttpLoopbackUpstreamWithoutTheDevelopmentFlag(String value) { + assertThrows(IllegalArgumentException.class, () -> FingerlyProxy.builder() + .proxyKey(PROXY_KEY) + .clientIp(request -> "203.0.113.5") + .upstream(value) + .build(), value); + // With the flag, loopback only. + builder(value).build(); + } + + @Test + void acceptsAnHttpsOriginAndStoresOnlyItsParsedParts() { + assertTrue(builder("https://API.example.invalid:8443/").build().toString() + .contains("upstream=https://api.example.invalid:8443/api/v1/ingest")); + assertTrue(builder("https://api.example.invalid").build().toString() + .contains("upstream=https://api.example.invalid/api/v1/ingest")); + } + + // PRX-01: redirects are never followed, whoever supplies the client. + + @Test + void refusesASuppliedClientThatFollowsRedirects() throws IOException { + for (HttpClient.Redirect policy : new HttpClient.Redirect[] { + HttpClient.Redirect.ALWAYS, HttpClient.Redirect.NORMAL}) { + HttpClient following = HttpClient.newBuilder().followRedirects(policy).build(); + IllegalArgumentException refused = assertThrows(IllegalArgumentException.class, + () -> builder("https://api.example.invalid").httpClient(following).build()); + assertFalse(refused.getMessage().contains(PROXY_KEY)); + } + builder("https://api.example.invalid") + .httpClient(HttpClient.newBuilder().followRedirects(HttpClient.Redirect.NEVER).build()) + .build(); + } + + @ParameterizedTest + @ValueSource(ints = {301, 302, 303, 307, 308}) + void aRedirectFromUpstreamIsABadGatewayAndIsNotFollowed(int status) throws IOException { + List leaked = new ArrayList<>(); + try (FakeApi target = FakeApi.start(exchange -> { + leaked.add(String.valueOf(exchange.getRequestHeaders().getFirst("X-Fingerly-Proxy-Key"))); + FakeApi.respond(exchange, 200, "{}"); + }); + FakeApi upstream = FakeApi.start(exchange -> { + try { + exchange.getResponseHeaders().add("Location", target.url() + "/elsewhere"); + exchange.sendResponseHeaders(status, -1); + } catch (IOException cause) { + throw new IllegalStateException(cause); + } + })) { + HttpClient never = HttpClient.newBuilder().version(HttpClient.Version.HTTP_1_1).build(); + for (FingerlyProxy proxy : new FingerlyProxy[] { + builder(upstream.url()).build(), builder(upstream.url()).httpClient(never).build()}) { + ProxyResponse response = proxy.handle(request("POST", PATH, browser())); + assertEquals(502, response.status()); + assertEquals("no-store", headerOf(response, "Cache-Control")); + assertNull(headerOf(response, "Location")); + } + assertEquals(List.of(), leaked, "the redirect target was called"); + } + } + + // PRX-03: a status outside 200 to 599 is a bad gateway. + + @ParameterizedTest + @ValueSource(strings = {"099", "000", "600", "700", "999"}) + void anUpstreamStatusOutOfRangeIsABadGateway(String status) throws Exception { + try (RawUpstream upstream = new RawUpstream( + "HTTP/1.1 " + status + " Odd\r\nContent-Type: application/json\r\n" + + "Content-Length: 2\r\nConnection: close\r\n\r\n{}")) { + ProxyResponse response = builder(upstream.url()).build() + .handle(request("POST", PATH, browser())); + assertEquals(502, response.status(), "upstream status " + status); + assertEquals("no-store", headerOf(response, "Cache-Control")); + } + } + + @ParameterizedTest + @ValueSource(ints = {200, 400, 409, 503}) + void anUpstreamStatusInRangeIsRelayed(int status) throws IOException { + try (FakeApi upstream = FakeApi.json(status, "{}")) { + assertEquals(status, + builder(upstream.url()).build().handle(request("POST", PATH, browser())).status()); + } + } + + // PRX-04: exact method, exact path, and no-store on every refusal. + + @ParameterizedTest + @ValueSource(strings = {"post", "Post", "pOST", "GET", "PUT", "DELETE", "options", "PATCH"}) + void onlyAnExactPostIsForwarded(String method) throws IOException { + try (FakeApi upstream = FakeApi.json(200, "{}")) { + ProxyResponse response = builder(upstream.url()) + .allowedOrigins("https://shop.test") + .build() + .handle(request(method, PATH, browser())); + assertEquals(405, response.status(), method); + assertEquals("no-store", headerOf(response, "Cache-Control")); + assertEquals("POST, OPTIONS", headerOf(response, "Allow")); + assertEquals(0, upstream.callCount(), method); + } + } + + @Test + void optionsIsAPreflightOnlyWithCorsConfigured() throws IOException { + try (FakeApi upstream = FakeApi.json(200, "{}")) { + ProxyResponse plain = builder(upstream.url()).build().handle(request("OPTIONS", PATH, browser())); + assertEquals(405, plain.status()); + assertEquals("POST", headerOf(plain, "Allow")); + assertEquals("no-store", headerOf(plain, "Cache-Control")); + ProxyResponse preflight = builder(upstream.url()).allowedOrigins("https://shop.test").build() + .handle(request("OPTIONS", PATH, browser())); + assertEquals(204, preflight.status()); + assertEquals("no-store", headerOf(preflight, "Cache-Control")); + } + } + + @Test + void nearMissPathsAreUncacheable404sAndNeverReachUpstream() throws IOException { + try (FakeApi upstream = FakeApi.json(200, "{}")) { + FingerlyProxy proxy = builder(upstream.url()).build(); + for (String path : new String[] {PATH + "/", PATH + "/..", PATH + "/%2e%2e/api/v1/events", + "/K7X2QM4A", PATH + ";x=1", "/" + PATH, PATH + "?a=1", PATH + "%2f", "/%6b7x2qm4a", + "/x/../k7x2qm4a", "/./k7x2qm4a"}) { + ProxyResponse response = proxy.handle(request("POST", path, browser())); + assertEquals(404, response.status(), path); + assertEquals("no-store", headerOf(response, "Cache-Control"), path); + } + assertEquals(0, upstream.callCount()); + } + } + + // Visitor headers the API trusts are never passed through. + + @Test + void aVisitorCannotSetAHeaderTheApiTrusts() throws IOException { + try (FakeApi upstream = FakeApi.json(200, "{}")) { + Map h = browser(); + h.put("X-Fingerly-Client-IP", "198.51.100.66"); + h.put("X-Forwarded-For", "198.51.100.66"); + h.put("X-Fingerly-Origin", "https://evil.test"); + h.put("X-Fingerly-Proxy-Key", "fly_px_development_attacker"); + h.put("X-Fingerly-User-Agent", "Spoofed"); + h.put("X-Fingerly-Edge-TLS-Version", "TLSv1.3"); + h.put("X-Fingerly-Proxy-Version", "spoof/9"); + h.put("Idempotency-Key", "a\r\nX-Fingerly-Client-IP: 198.51.100.66"); + assertEquals(200, builder(upstream.url()).build().handle(request("POST", PATH, h)).status()); + FakeApi.Recorded seen = upstream.first(); + assertEquals(List.of("203.0.113.5"), seen.headers.get("X-Fingerly-Client-IP")); + assertEquals(List.of("https://shop.test"), seen.headers.get("X-Fingerly-Origin")); + assertEquals(List.of(PROXY_KEY), seen.headers.get("X-Fingerly-Proxy-Key")); + assertEquals(List.of("Firefox"), seen.headers.get("X-Fingerly-User-Agent")); + assertNull(seen.header("X-Forwarded-For")); + assertNull(seen.header("X-Fingerly-Edge-TLS-Version")); + assertNull(seen.header("Idempotency-Key")); + assertFalse(seen.header("X-Fingerly-Proxy-Version").startsWith("spoof")); + } + } + + // PRX-05: a failure line names the kind of failure, not the upstream. + + @Test + void aFailureLineNamesNeitherTheUpstreamNorTheKey() throws Exception { + int closedPort; + try (ServerSocket probe = new ServerSocket(0, 1, InetAddress.getByName("127.0.0.1"))) { + closedPort = probe.getLocalPort(); + } + List lines = new ArrayList<>(); + FingerlyProxy proxy = builder("http://127.0.0.1:" + closedPort).log(lines::add).build(); + ProxyResponse response = proxy.handle(request("POST", PATH, browser())); + assertEquals(502, response.status()); + assertEquals(1, lines.size(), lines.toString()); + for (String text : List.of(lines.get(0), new String(response.body(), StandardCharsets.UTF_8))) { + assertFalse(text.contains("127.0.0.1"), text); + assertFalse(text.contains(String.valueOf(closedPort)), text); + assertFalse(text.contains("ingest"), text); + assertFalse(text.contains(PROXY_KEY) || text.contains(PUBLIC_KEY), text); + } + } + + /** A one-shot raw socket upstream on 127.0.0.1 that answers fixed bytes. */ + private static final class RawUpstream implements AutoCloseable { + private final ServerSocket socket; + private final Thread worker; + private final AtomicInteger served = new AtomicInteger(); + + RawUpstream(String answer) throws IOException { + socket = new ServerSocket(0, 8, InetAddress.getByName("127.0.0.1")); + worker = new Thread(() -> { + while (!socket.isClosed()) { + try (Socket client = socket.accept()) { + drainHead(client.getInputStream()); + OutputStream out = client.getOutputStream(); + out.write(answer.getBytes(StandardCharsets.ISO_8859_1)); + out.flush(); + served.incrementAndGet(); + } catch (IOException closed) { + return; + } + } + }); + worker.setDaemon(true); + worker.start(); + } + + String url() { + return "http://127.0.0.1:" + socket.getLocalPort(); + } + + /** Reads the request headers and a Content-Length body, enough to answer. */ + private static void drainHead(InputStream in) throws IOException { + StringBuilder head = new StringBuilder(); + int b; + while ((b = in.read()) != -1) { + head.append((char) b); + if (head.toString().endsWith("\r\n\r\n")) { + break; + } + } + for (String line : head.toString().split("\r\n")) { + if (line.toLowerCase(java.util.Locale.ROOT).startsWith("content-length:")) { + int length = Integer.parseInt(line.substring(15).trim()); + in.readNBytes(length); + } + } + } + + @Override + public void close() throws IOException { + socket.close(); + } + } +} diff --git a/fingerly-server/src/test/java/io/fingerly/server/SharedHttpTest.java b/fingerly-server/src/test/java/io/fingerly/server/SharedHttpTest.java index 30fe6f4..68a8876 100644 --- a/fingerly-server/src/test/java/io/fingerly/server/SharedHttpTest.java +++ b/fingerly-server/src/test/java/io/fingerly/server/SharedHttpTest.java @@ -42,10 +42,12 @@ void speaksHttp11AndNeverFollowsRedirects() { void reusesConnectionsAcrossInstances() throws IOException { try (FakeApi api = FakeApi.json(200, Fixtures.event())) { for (int i = 0; i < 20; i++) { - FingerlyClient.builder().secretKey(SECRET_KEY).endpoint(api.url()).build() + FingerlyClient.builder().secretKey(SECRET_KEY) + .allowInsecureLoopback(true).endpoint(api.url()).build() .events().get(REQUEST_ID); FingerlyProxy.builder() .proxyKey(PROXY_KEY) + .allowInsecureLoopback(true) .upstream(api.url()) .clientIp(request -> "203.0.113.5") .build() @@ -53,6 +55,7 @@ void reusesConnectionsAcrossInstances() throws IOException { .method("POST") .path("/p") .header("X-API-Key", PUBLIC_KEY) + .header("Origin", "https://shop.test") .body("{}".getBytes(StandardCharsets.UTF_8)) .build()); } diff --git a/gradle.properties b/gradle.properties index 830f10a..1160177 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,4 +1,4 @@ -version=0.2.0 +version=0.3.0 org.gradle.jvmargs=-Xmx2g org.gradle.caching=true org.gradle.parallel=true