Skip to content
84 changes: 62 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,10 @@ Boolean questions answer as `noul`, a probability from 0 to 1 that the answer is
| Export | Description |
| --------------------------------- | ----------------------------------------------------------------------------------- |
| `evaluate(input, options?)` | Runs one evaluation. Returns `EvaluateResult`: decisions or a `FallbackResult`. |
| `systemOne(input, options?)` | Keyed decisions. Returns `answers` by question name, or the same fallback. |
| `choice`, `noul`, `score`, `box` | Build one keyed question. `box` locates a region and needs model `jev-preview`. |
| `image`, `Image.fromBase64` | PNG or JPEG bytes, a `Blob`, or base64. Other formats are rejected before send. |
| `withAssets` | Tag a template so interpolated images are sent. Also `withAssets.join`. |
| `createSystemOneAdapter(config?)` | Returns a `ProviderAdapter` for `runInference`. `config` is an `EvaluateConfig`. |
| `createSystemOneAdapterFactory` | Interchange `AdapterFactory` for `SIDECAR_ADAPTER_MANIFEST`; takes offering quirks. |
| `SYSTEM_ONE_PROVIDER` | The `"corbits-system-one"` provider id. |
Expand All @@ -82,44 +86,80 @@ Boolean questions answer as `noul`, a probability from 0 to 1 that the answer is

### Questions

| `type` | Fields | Decision fields |
| --------- | ------------------------------------------------------------------------------- | ------------------------------------------------ |
| `choice` | `instructions`, `criteria` (option name → description or `null`, 1–255 options) | `choice`, `probabilities`, `confidence` |
| `score` | `instructions`, `criteria` (2–10 descriptions) | `score`, `legend`, `probabilities`, `confidence` |
| `boolean` | `instructions`, optional `criteria: { true?, false? }` | `noul` |
| `noul` | Same as `boolean` | `noul` |
| `type` | Fields | Decision fields |
| --------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| `choice` | `instructions`, `criteria` (option name → description or `null`, 1–255 options) | `choice`, `probabilities`, `confidence` |
| `score` | `instructions`, `criteria` (2–10 descriptions) | `score`, `legend`, `probabilities`, `confidence` |
| `boolean` | `instructions`, optional `criteria: { true?, false? }` | `noul` |
| `noul` | Same as `boolean` | `noul` |
| `box` | `instructions` describing what to locate. Use model `jev-preview`. | `box` (`top`, `bottom`, `left`, `right` as fractions), `confidence` |

Every question has an `id`, unique within the list.
Every list question has an `id`, unique within the list. `box` edges outside `[0, 1]` are rejected.

### Keyed questions

`systemOne` takes the same state and a record of questions built with `choice`, `noul`, `score`, and `box`. The names are the answer keys. Those helpers compile to the list `evaluate` already sends. The same fallback reasons apply, and those failures do not throw. The helpers are also exported from `@corbits/system-one/decisions`.

```ts
import { choice, noul, score, systemOne } from "@corbits/system-one";

const result = await systemOne({
state: { document: "I was charged twice." },
questions: {
category: choice("What is this?", { billing: null, technical: null }),
escalate: noul("Must this be escalated?"),
severity: score("How severe?", ["none", "cosmetic", "unusable"]),
},
});
```

On success, read `result.answers.category.choice` (plus `probabilities` and `confidence`), `result.answers.escalate.noul`, and `result.answers.severity.score`.

`image()` accepts a `Blob` or `Uint8Array`. `Image.fromBase64()` accepts base64 file bytes. Only PNG and JPEG are sent. Place an image in `state`, instructions, or criteria, or interpolate it with `withAssets` (and `withAssets.join` for a list). An image in an ordinary template string is rejected before the request. `assets` maps a name to an image for plain strings; every name must appear as its own token in the request. A request with any image is `multipart/form-data` and sends `TypeSafe-Danger-Zone-Images: true`. Otherwise the body stays JSON.

```ts
import { box, image, systemOne, withAssets } from "@corbits/system-one";

const photo = image(pngBytes);
const result = await systemOne({
state: withAssets`Returned item: ${photo}`,
questions: { damage: box("The damaged area") },
model: "jev-preview",
});
```

`providerOptions.systemOne.questions` accepts this keyed record as well as a question list.

### Config

`input.config` for `evaluate`, or the argument to `createSystemOneAdapter`:

| Field | Type | Default | Description |
| ----------- | ---------------- | ---------------------- | -------------------------------------------------------------------------------- |
| `endpoint` | `EndpointConfig` | `{ kind: "official" }` | `official`, `gateway`, or `custom` with a `url`. Each takes an optional `model`. |
| `timeoutMs` | `number` | `1500` | Bound for the whole round trip. |
| `apiKey` | `string` | from the environment | Overrides the environment key. |
| Field | Type | Default | Description |
| ----------- | ---------------- | ---------------------- | ----------------------------------------------------------------------------------------------------------------- |
| `endpoint` | `EndpointConfig` | `{ kind: "official" }` | `official`, `gateway`, or `custom` with a `url`. Each takes an optional `model`. `custom` may set `auth: "none"`. |
| `timeoutMs` | `number` | `1500` | Bound for the whole round trip. |
| `apiKey` | `string` | from the environment | Overrides the environment key. |

### Endpoints and keys

| `kind` | URL | Default model | Key from the environment |
| ---------- | ---------------------------------------------------- | ----------------- | ---------------------------------------------- |
| `official` | `https://api.typesafe.ai/v1/systemone` | `jev-latest` | `TYPESAFE_API_KEY` |
| `gateway` | `https://ai-gateway.vercel.sh/typesafe/v1/systemone` | `typesafe-ai/jev` | `AI_GATEWAY_API_KEY`, then `VERCEL_OIDC_TOKEN` |
| `custom` | `url` | `typesafe-ai/jev` | `SYSTEM_ONE_API_KEY` |
| `custom` | `url` | `typesafe-ai/jev` | `SYSTEM_ONE_API_KEY`, unless `auth: "none"` |

`SYSTEM_ONE_API_KEY` is also a supported alias on every endpoint, read after the endpoint's own key.
A custom endpoint with `auth: "none"` does not require a key and does not send `Authorization`. Official and gateway always require a key. Any other `auth` value is invalid input and throws.

### Fallback reasons

| `reason` | When |
| ------------- | ------------------------------------------------------------- |
| `no-key` | No key in `apiKey` or the environment. No request is sent. |
| `timeout` | The round trip, including the body read, exceeds `timeoutMs`. |
| `network` | The request fails before a response. |
| `http-error` | Non-2xx response. `httpStatus` holds the status. |
| `parse-error` | The body is not JSON or fails validation. |
| `reason` | When |
| ------------- | -------------------------------------------------------------------------------------------------------------- |
| `no-key` | No key in `apiKey` or the environment, and the endpoint is not custom with `auth: "none"`. No request is sent. |
| `timeout` | The round trip, including the body read, exceeds `timeoutMs`. |
| `network` | The request fails before a response. |
| `http-error` | Non-2xx response. `httpStatus` holds the status. |
| `parse-error` | The body is not JSON or fails validation. |

`detail` carries the underlying message.

Expand Down Expand Up @@ -208,7 +248,7 @@ The hub offering's `quirks` are validated and all optional:
}
```

Without `endpoint`, the adapter posts to the catalog provider's `baseURL` + `/systemone`, the host the offering's credential was issued for. A `model` quirk (top level or on `endpoint`) is the model sent on the wire, so one catalog model can be served by providers that name it differently. `questions` and `state` are per-call defaults; `providerOptions.systemOne` on a call wins field by field. Unknown keys throw. The adapter sends the bearer sentinel and the harness injects the offering's credential.
Without `endpoint`, the adapter posts to the catalog provider's `baseURL` + `/systemone`, the host the offering's credential was issued for. A `model` quirk (top level or on `endpoint`) is the model sent on the wire, so one catalog model can be served by providers that name it differently. `questions` and `state` are per-call defaults; `providerOptions.systemOne` on a call wins field by field. Unknown keys throw. The adapter sends the bearer sentinel and the harness injects the offering's credential, except a custom endpoint with `auth: "none"`, which omits `Authorization` so the host does not attach a key.

| Provider | Provider `baseURL` | `model` quirk |
| ----------------- | ------------------------------------------ | ----------------------------- |
Expand All @@ -221,7 +261,7 @@ Without `endpoint`, the adapter posts to the catalog provider's `baseURL` + `/sy

- `createSystemOneAdapterFactory` without an `endpoint` quirk now posts to the catalog provider's `baseURL` + `/systemone` instead of the official TypeSafe URL. Set the provider `baseURL` to the API root (for example `https://api.typesafe.ai/v1`), or keep the old behavior with `{"endpoint":{"kind":"official"}}`. The harness sends the offering's credential to whatever URL the adapter returns, so only set an explicit `endpoint` on a host that credential belongs to.
- A `model` quirk now overrides the model the harness passes (the catalog canonical name).
- `createSystemOneAdapter` and `evaluate` are unchanged.
- `createSystemOneAdapter` and `evaluate` accept `auth: "none"` on a custom endpoint and omit `Authorization`. Any other `auth` value throws `SystemOneError` before a request is built.

## Upgrading from 0.1

Expand Down
12 changes: 11 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@corbits/system-one",
"version": "0.3.2",
"version": "0.4.0",
"description": "Typed-decision evaluation client for System One (Jev-class) models: arktype question/decision schemas, endpoint configuration with gateway override, and timeout/fallback semantics.",
"keywords": [
"corbits",
Expand Down Expand Up @@ -32,6 +32,16 @@
"corbits-system-one-src": "./src/index.ts",
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
},
"./decisions": {
"corbits-system-one-src": "./src/decisions.ts",
"types": "./dist/decisions.d.ts",
"default": "./dist/decisions.js"
},
"./extraction": {
"corbits-system-one-src": "./src/extraction.ts",
"types": "./dist/extraction.d.ts",
"default": "./dist/extraction.js"
}
},
"publishConfig": {
Expand Down
117 changes: 117 additions & 0 deletions src/adapter.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import {
createSystemOneAdapter,
createSystemOneAdapterFactory,
} from "./adapter";
import { SystemOneError } from "./errors";

describe("adapter", () => {
function turns(): ConversationTurn[] {
Expand Down Expand Up @@ -122,6 +123,54 @@ describe("adapter factory", () => {
{ role: "user", timestamp: 0, content: [{ type: "text", text: "Hi" }] },
];

test("buildRequest omits Authorization for custom auth none and keeps the sentinel otherwise", () => {
const turns = (): ConversationTurn[] => [
{ role: "user", timestamp: 0, content: [{ type: "text", text: "Hi" }] },
];
const keyless = createSystemOneAdapter({
endpoint: {
kind: "custom",
url: "https://local.test/eval",
auth: "none",
},
}).buildRequest(turns(), "jev-latest", {});
expect(keyless.headers["authorization"]).toBeUndefined();
expect(Object.keys(keyless.headers)).not.toContain("authorization");

const official = createSystemOneAdapter({
endpoint: { kind: "official" },
}).buildRequest(turns(), "jev-latest", {});
expect(official.headers["authorization"]).toBe(BEARER_CREDENTIAL_SENTINEL);

const gateway = createSystemOneAdapter({
endpoint: { kind: "gateway" },
}).buildRequest(turns(), "jev-latest", {});
expect(gateway.headers["authorization"]).toBe(BEARER_CREDENTIAL_SENTINEL);

const keyedCustom = createSystemOneAdapter({
endpoint: { kind: "custom", url: "https://h.test/e" },
}).buildRequest(turns(), "jev-latest", {});
expect(keyedCustom.headers["authorization"]).toBe(
BEARER_CREDENTIAL_SENTINEL,
);
});

test("invalid auth throws before a request is built", () => {
let thrown: unknown;
try {
createSystemOneAdapter({
endpoint: {
kind: "custom",
url: "https://h.test/e",
auth: "bearer",
},
} as unknown as Parameters<typeof createSystemOneAdapter>[0]);
} catch (cause) {
thrown = cause;
}
expect(thrown).toBeInstanceOf(SystemOneError);
});

test("no quirks routes to the provider baseURL with sentinel auth", () => {
const request = createSystemOneAdapterFactory(source).buildRequest(
turns(),
Expand Down Expand Up @@ -183,4 +232,72 @@ describe("adapter factory", () => {
/model must not be empty/,
);
});

test("providerOptions.systemOne accepts keyed questions", () => {
const adapter = createSystemOneAdapter();
const request = adapter.buildRequest([], "jev-latest", {
providerOptions: {
systemOne: {
state: "ticket",
questions: {
category: {
type: "choice",
instructions: "What?",
criteria: { a: null },
},
},
},
},
});
const body = JSON.parse(request.body) as {
questions: Record<string, { type: string }>;
};
expect(body.questions["category"]?.type).toBe("choice");
});

test("parseJSONResponse rejects a box outside the image", () => {
const adapter = createSystemOneAdapter();
expect(() =>
adapter.parseJSONResponse(
JSON.stringify({
answers: {
region: {
type: "box",
box: { top: -0.1, bottom: 0.4, left: 0.2, right: 0.5 },
confidence: null,
},
},
}),
),
).toThrow(/outside \[0, 1\]/);
});

test("parseJSONResponse rejects an inverted box", () => {
const adapter = createSystemOneAdapter();
const reject = (box: {
top: number;
bottom: number;
left: number;
right: number;
}) =>
adapter.parseJSONResponse(
JSON.stringify({
answers: {
region: { type: "box", box, confidence: null },
},
}),
);
expect(() =>
reject({ top: 0.8, bottom: 0.1, left: 0.2, right: 0.5 }),
).toThrow(/not below/);
expect(() =>
reject({ top: 0.2, bottom: 0.2, left: 0.1, right: 0.4 }),
).toThrow(/not below/);
expect(() =>
reject({ top: 0.1, bottom: 0.4, left: 0.9, right: 0.2 }),
).toThrow(/not left of/);
expect(() =>
reject({ top: 0.1, bottom: 0.4, left: 0.5, right: 0.5 }),
).toThrow(/not left of/);
});
});
Loading
Loading