diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5484168..f23ac2b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -37,6 +37,9 @@ jobs: - name: Unit tests run: npm test + - name: Sharing source contracts + run: node --test scripts/social-metadata.test.mjs + - name: Types run: npm run typecheck @@ -74,10 +77,13 @@ jobs: - name: End-to-end tests run: npm run e2e - - name: Upload report - if: failure() + - name: Upload report and sharing evidence + if: always() uses: actions/upload-artifact@v7 with: name: playwright-report - path: playwright-report/ + path: | + playwright-report/ + test-results/ + if-no-files-found: ignore retention-days: 7 diff --git a/README.md b/README.md index 7cbb76d..946465b 100644 --- a/README.md +++ b/README.md @@ -1,248 +1,86 @@ # Jev Tool & Model Router -A TypeScript routing library and interactive Next.js demo that use **TypeSafe AI's Jev** to select from a fixed set of tools or models, then apply explicit confidence and fallback policies. +A TypeScript routing library and Next.js lab for selecting from a fixed set of tools or models with TypeSafe AI's Jev, explicit confidence policies, and inspectable fallbacks. -**Jev selects. Your application authorizes and executes.** This is an independent community project under `BunsDev`, not an official TypeSafe SDK or a production authorization system. The root package is private; the library is source in this repository, not a published npm install. +**Jev selects. Your application authorizes and executes.** This is an unofficial [TypeSafeAI community](https://github.com/TypeSafeAI) project, not the official SDK or a production authorization system. The community organization was created by VC Moderator [@BunsDev](https://github.com/BunsDev). -[Contributing](CONTRIBUTING.md) · [Agent guide](AGENTS.md) · [TypeSafe API reference](https://docs.typesafe.ai/api) +[Try the lab](https://route.jev.works) · [Developer guide](docs/discovery/README.md) · [Detailed API reference](REFERENCE.md) · [Contributing](CONTRIBUTING.md) · [Agent instructions](AGENTS.md) -```text -request + allowed options -> Jev choice -> validation + fallback -> effective option - | - your authorization and executor -``` +![Jev Router: selection is separate from authorization](docs/discovery/assets/social-preview.svg) ## What it does -The model and tool routers share one engine, `createRouter` in `lib/jevRouter.ts`. Each call asks one `choice` question using the supplied option ids as the closed set. The result retains Jev's original pick, its scores, the effective option after policy, and the fallback reason. +```text +request + allowed options → Jev choice → validation + fallback → effective option + | + your authorization and executor +``` + +The shared `createRouter` engine asks a closed-set `choice` question, preserves the original selection and scores, applies deterministic policy, and returns the effective option plus a fallback reason. It does not execute the chosen tool or invoke the selected downstream model. A live Jev classification is a provider call; the no-key demo is a labeled local simulation. -The library **does not execute tools or invoke the selected downstream model**. It does make a Jev API call when using the live transport. The UI's execution display is illustrative, not evidence that an external action ran. A structurally valid choice can still be the wrong decision. +The root package is private. This is source in a repository, not a published `npm install typesafe-router` package. Imports using this project's aliases work inside this checkout; integrate deliberately rather than assuming a registry distribution exists. ## Run locally -Use npm and the committed `package-lock.json`; do not generate a competing pnpm, Yarn, or Bun lockfile. Use a Node.js version supported by the installed Next.js dependency; Node.js 22+ is a practical development baseline. This repository currently has no root `engines` or `packageManager` pin. +Use npm and the committed `package-lock.json`. The manifest has no Node/package-manager pin; use a Node version supported by the checked-in Next.js dependency. See the [CI workflow](.github/workflows/ci.yml) for the tested runtime. ```sh -git clone https://github.com/BunsDev/typesafe-router.git +git clone https://github.com/TypeSafeAI/typesafe-router.git cd typesafe-router npm ci npm run dev ``` -Open the address printed by Next.js, normally `http://localhost:3000`. With no configured key, the demo uses the local keyword-based `mockCallJev` transport. Mock and live results are labeled separately. - -For live routing, get a key from the [TypeSafe console](https://console.typesafe.ai), then optionally configure the server: - -```sh -cp .env.local.example .env.local -# Edit .env.local and set TYPESAFE_API_KEY, then restart the dev server. -``` - -### 1Password-backed key (recommended) - -Keep the key in 1Password and run the app through `op run`, so no secret ever -lands in a file: - -```bash -npm run dev:op -``` - -`.env.1password` is committed and holds a **reference**, not a secret: -`TYPESAFE_API_KEY="op://Development/Jev API Key/password"`. `op run` expands it -into the process environment at launch. Requires 1Password CLI 2.x, unlocked -(desktop-app integration or `op signin`). - -Verify it took effect — `live` reports whether the SERVER has a key: - -```bash -curl -s http://localhost:3000/api/route -# {"live":true} ({"live":false} means demo mode) -``` - -A shell-provided variable is not overridden by `.env.local` (`@next/env`'s -`processEnv` only fills keys absent from the initial environment), so this path -wins over any stale key left in `.env.local`. - -The browser calls this app's `/api/route` endpoint. The server forwards live evaluations to `POST https://api.typesafe.ai/v1/systemone` using `jev-latest`. Jev answers typed questions; it does not generate the code that executes a route. - -### Bring your own key and privacy - -The key dialog also accepts a personal key. Precedence is **browser key → server environment key → demo mode**. - -A browser key persists, unencrypted by the app, in `localStorage` under `jev-router:api-key`. It is sent in the `x-typesafe-api-key` header to this application's server, which forwards it to TypeSafe. Masking the field reduces accidental screen exposure; it does not protect against scripts running on the origin, access to the browser profile, developer tools, or an untrusted deployment. Use a deployment you trust and remove the key on shared machines. - -### Limits on a public deployment - -`/api/route` is unauthenticated, so it bounds what one request can make the server do (`lib/limits.ts`): the input and the context are capped at 20,000 characters each, an option list at 32 entries, and each option's id, label and description at 64, 120 and 1,000 characters. A body over 256 KB is refused with 413 before it is parsed. The lab applies the same caps as `maxLength` on its fields and stops the editor at 32 options. Option ids, labels and descriptions are collapsed onto one line in the text sent to Jev, so a line break inside one cannot open a fake section. - -Calls that would spend the **server's** key are also rate limited: `ROUTE_RATE_LIMIT_PER_MINUTE` (default 60, `0` disables) returns 429 with code `rate_limited` once exceeded. Calls that carry a browser-saved key are not counted; they spend that user's credits. By default every caller shares one bucket, because `x-forwarded-for` and `x-real-ip` are whatever the client sent when the app is reached directly. Set `ROUTE_TRUST_PROXY=1` when a proxy you control (Vercel, nginx, a load balancer) overwrites those headers, and the limit becomes per client IP, read from the last forwarded hop. The counter is per process and tracks at most 10,000 callers (idle ones are dropped first, then the least recently seen), so on a serverless host it slows casual abuse rather than enforcing a global quota. Put a gateway limiter or an auth wall in front if the deployment is meant for strangers. - -### What the lab remembers - -Edited options, the threshold, and the routing history are saved in your browser (`localStorage`, key `jev-router:lab`) so a refresh doesn't lose them. **Reset everything** in the footer removes the stored copy; nothing is written back until you change something again. **Export JSON** in the history panel downloads the full log, one `RoutingLogEntry` per decision, for review or tuning. The API key is stored separately and is never part of this blob. Whatever is read back is validated field by field, and a malformed entry is dropped rather than rendered. - -Selecting a request in the history table (click the row, or Tab to it and press Enter) loads its mode, input, conversation context and threshold back into the request panel. The options are whatever the editor holds at that moment (only their ids are logged), so a re-run is a fresh decision, not a replay. Resetting the lab discards any routing call still in flight; clearing the history while a call is running keeps that decision on screen but doesn't re-add it to the log. -The server environment key is not sent to the browser. Do not commit `.env.local`, embed a key in a component or URL, or use a `NEXT_PUBLIC_` credential. A public server key can spend the owner's credits on visitors' requests; add appropriate access controls, request limits, and provider-side spending limits before offering one publicly. These deployment controls are not provided by the routing decision itself. - -## Routing policies - -| Case | Model router | Tool router | -| --- | --- | --- | -| Example options | `fast_cheap_model`, `reasoning_model`, `code_model`, `general_model` | `web_search`, `calculator`, `calendar_lookup`, `no_tool_needed` | -| Required option | `general_model` | `no_tool_needed` | -| Below confidence threshold | Use configured `safe_default` | Return `needs_clarification`; `effectiveOptionId` is `null` | -| Unusable or out-of-list answer | Same configured fallback, reason `invalid_option` | Same configured fallback, reason `invalid_option` | -| Invalid configuration | Throw `RouterConfigError` before calling Jev | Throw `RouterConfigError` before calling Jev | -| Transport/provider error | Reject the call; do not turn failure into a successful route | Reject the call; do not turn failure into a successful route | - -Other scripts: `npm test` (vitest: the engine, the wire format, the simulator, storage, the rate limiter, and the API route), `npm run e2e` (Playwright: builds and serves the app in demo mode, then drives the lab and the API through a real browser: recall, persistence, Clear and Reset during a run, the option editor, fallbacks, limits; `--ui` opens Playwright's inspector), `npm run typecheck` (runs `next typegen` first so a clean checkout has the generated `next-env.d.ts`, which is gitignored as Next.js recommends), `npm run lint`, `npm run build`. -The default threshold is `0.75`, with per-router and per-call overrides. `safe_default` is the policy name, **not a guarantee of harmless execution**. A downstream model call may expose data, cost money, or violate an application's policy. The integrating application must validate the selected id against a trusted registry and independently enforce permissions, privacy, budgets, and approval requirements. - -`no_tool_needed` is an explicit no-action option, not a tool to invoke. Preserve it when editing the tool list. For clarification, ask the user rather than executing Jev's original low-confidence selection. - -## Use the library - -This example runs inside this repository and uses the mock transport, so it does not consume API credits: - -```ts -import { createRouter } from "@/lib/jevRouter"; -import { modelRouterConfig, modelRouterOptions } from "@/lib/routerConfigs"; -import { mockCallJev } from "@/lib/mockRouter"; - -const router = createRouter(modelRouterConfig, mockCallJev, "mock"); -const userInput = "Help me debug a TypeScript type error."; -const { result, logEntry } = await router.route({ - userInput, - context: "The request concerns a local development project.", - options: modelRouterOptions, -}); - -console.log(result.effectiveOptionId, result.action, logEntry.source); -// This example deliberately does not execute the returned option. -``` - -For live server-side use, supply the real transport instead of the mock and configure credentials securely. Do not move a provider credential into client-side imports merely to reuse this example. - -### Core API - -- `routeWithJev(request, threshold?, transport?)` → a raw `RouteDecision` with Jev's pick and `fallbackUsed` set, no policy applied. Throws `RouteIntegrityError` if Jev's answer isn't in the option list, isn't a `choice` answer, or is missing. A valid pick that arrives without a confidence (and without probabilities) is kept at confidence 0, so it takes the ordinary low-confidence path rather than being reported as invalid. -- `resolveFallback(decision, options, policy, source)` → applies a `FallbackPolicy` to a decision. Pure, no I/O. -- `assertFallbackOption(options, policy)` → the runtime check described below. -- `buildRouterContext(request)` / `buildJevRequest(request)` → the exact text and question sent to Jev, so you can inspect or test it. -- `callJev(request, apiKey?)` → the low-level client. `mockCallJev` has the same signature for demo mode; tests inject their own. -- `createRouter(config, transport?, source?)` validates requests, calls the transport, applies policy, and logs the result. -- `routeWithJev(request, threshold?, transport?)` returns the raw decision and throws `RouteIntegrityError` for an unusable or out-of-set answer. It does not apply the full router fallback policy. -- `resolveFallback(decision, options, policy, source)` applies the deterministic policy without network I/O. -- `assertFallbackOption(options, policy)` checks the required fallback/no-tool option. -- `buildRouterContext(request)` and `buildJevRequest(request)` expose what will be submitted. - -Public re-exports live in `lib/index.ts`; full types live in `types/router.ts`. - -```ts -type RouteOption = { - id: string; - label: string; - description: string; - metadata?: Record; -}; -type RouteRequest = { userInput: string; context?: string; options: RouteOption[] }; -``` +Open the address printed by Next.js. No key means local mock mode. For intentional live evaluation, follow the [credential and deployment reference](REFERENCE.md#bring-your-own-key-and-privacy); automated tests and screenshots must not spend provider credits. -Use stable, unique ids and at least two options. Write descriptions as “use this when…” guidance. The option `metadata` is for the integrating application and is not sent to Jev. Keep the policy's required option in every request. Add a new router with an explicit `mode`, threshold, and `safe_default` or `needs_clarification` policy. +A browser key overrides the server key, and without either the lab uses its mock. Browser keys persist in unencrypted localStorage and travel through this deployment's server, so use a deployment you trust and remove keys from shared machines. Never put a key in a URL, client bundle, screenshot, export, or `NEXT_PUBLIC_` variable. -## Saved state, logs, and tuning +## Policy outcomes -The lab stores edited options, thresholds, and history in `localStorage` under `jev-router:lab`. Reset clears the lab state; remove the API key separately. Exported JSON contains routing history, not the separately stored key. Inputs and conversation context can nevertheless contain private information: inspect exports before sharing. +| Situation | Behavior | +| --- | --- | +| Valid high-confidence selection | Return the permitted option ID as evidence for the host | +| Low-confidence model choice | Use the configured `safe_default` policy | +| Low-confidence tool choice | Request clarification with no effective tool ID | +| Invalid configuration or missing mandatory fallback | Reject before the provider call | +| Provider/transport failure | Propagate failure; do not manufacture a successful mock result | -1. Every `id` must be unique. It is the only value Jev can return, so make it stable. Ids are used as object keys in the wire format and the score maps, and the library builds and reads those maps so that awkward ids such as `constructor` or `__proto__` behave like any other. -2. The list must still contain the option the fallback policy requires (`general_model` for the model router, `no_tool_needed` for the tool router). Otherwise `createRouter(...).route()` throws a `RouterConfigError` **before** calling Jev: -Logs record the input/context, considered options, original and effective selections, confidence, threshold, scores, fallback action, source (`jev` or `mock`), and duration. The default console logger prints a shortened input excerpt. For sensitive integrations, supply an appropriate logger and retention policy; do not assume prompts are automatically redacted. Mock confidence and routing outcomes are demonstrations, not model-quality measurements. +The policy name `safe_default` is not a guarantee of harmless execution. Preserve `no_tool_needed`, mandatory fallbacks, unique option IDs, threshold validation, and the distinction between original and effective selections. The host still owns permissions, budgets, privacy, approvals, and dispatch through a trusted registry. -## Project map +## Source map | Path | Responsibility | | --- | --- | -| `lib/jevRouter.ts` | Validation, request construction, policy, and logging. | -| `lib/jevClient.ts` | TypeSafe wire format, response normalization, and provider errors. | -| `lib/routerConfigs.ts` | Example model/tool options and fallback policies. | -| `lib/mockRouter.ts` | Local simulated transport. | -| `lib/apiKeyStorage.ts`, `lib/labStorage.ts` | Separate browser key and lab-state storage. | -| `types/router.ts` | Public routing contracts. | -| `app/api/route/route.ts` | Same-origin server endpoint and credential handling. | -| `components/RouterLab.tsx` | Interactive lab, options, threshold, and history. | +| `lib/jevRouter.ts` | Validation, request construction, routing policy and logs | +| `lib/jevClient.ts` | Provider wire format, normalization and errors | +| `lib/routerConfigs.ts` | Example options and required fallback choices | +| `lib/mockRouter.ts` | Keyword-based local demonstration | +| `types/router.ts`, `lib/index.ts` | Types and public source exports | +| `app/api/route/route.ts` | Server endpoint and credential boundary | +| `components/RouterLab.tsx` | Interactive lab and state | +| `lib/apiKeyStorage.ts`, `lib/labStorage.ts` | Separate browser key and lab persistence | -## Development checks +The [complete pre-existing reference](REFERENCE.md) retains all API examples, storage/export semantics, request limits, proxy/rate-limit guidance, and wire-format details without deleting technical documentation. Historical transferred-owner links remain there; the canonical current clone URL is the TypeSafeAI URL above. + +## Verify changes ```sh npm test npm run typecheck npm run lint npm run build +npm run e2e +node --test scripts/social-metadata.test.mjs ``` -Use mock/injected transports for automated checks. For UI changes, also exercise both router modes, required-option deletion protection, invalid options, low-confidence clarification, key removal, persistence, exports, themes, and narrow screens. Report actual test results rather than keeping a hardcoded test count in this README. +The browser suite builds and serves a no-key demo. The sharing test checks rendered OG/Twitter metadata, an actual PNG response and its dimensions, and captures the browse-only interface. CI retains browser evidence on successful as well as failed runs. Review the exact run rather than treating a workflow file as proof that checks passed. -```text -e2e/ - lab.spec.ts browser flows against the production build (Playwright) - api.spec.ts /api/route contract in demo mode - helpers.ts locators and a request-delaying route -playwright.config.ts builds and serves on :3111 with the key unset -lib/ - apiKeyStorage.ts browser-only key storage; the only reader hands the key straight to fetch - labStorage.ts browser-only persistence of options, threshold and history, validated on load; JSON export - guards.ts isFiniteNumber and readOwn (prototype-safe reads of maps keyed by option id) - limits.ts size caps shared by the API route (enforced) and the UI (maxLength) - rateLimit.ts in-memory sliding-window limiter for calls that spend the server key - jevClient.ts low-level Jev API wrapper (wire format, errors, normalisation) - jevRouter.ts engine: validation, routeWithJev, resolveFallback, createRouter, logging - routerConfigs.ts modelRouterOptions, toolRouterOptions, fallback policies - mockRouter.ts demo-mode keyword simulator, same JevTransport signature as callJev - index.ts public re-exports - testing/fakeStorage.ts localStorage stand-in for the storage tests - *.test.ts vitest suites -types/ - router.ts RouteOption, RouteRequest, RouteDecision, FallbackPolicy, RoutingLogEntry, … -app/ - page.tsx renders the lab client-only (LabLoader) so restored state needs no hydration dance - api/route/route.ts server-side routing endpoint; the only place the API key is read -components/ - LabLoader.tsx client-only dynamic import of RouterLab with a skeleton - RouterLab.tsx state: mode, input, context, options per mode, threshold, result, history - ApiKeyPanel.tsx bring-your-own-key dialog: masked input, never displayed, stored in this browser - ThemeToggle.tsx light/dark switch with transitions suppressed for the flip - ModeToggle.tsx Model Router / Tool Router - OptionEditor.tsx live-editable option list with the required option locked - ThresholdSlider.tsx confidence threshold - RoutingResult.tsx pick, colour-coded confidence, fallback status, mocked execution line - ScoreBreakdown.tsx per-option probability bars with the threshold marker - RoutingHistoryTable.tsx -.env.local.example -``` +For UI changes, also inspect both router modes, option editing, null clarification, key removal, storage/export behavior, themes, keyboard controls, and narrow layouts. Do not weaken validation or lower thresholds to obtain favorable screenshots. -## Jev wire format, for reference - -`lib/jevClient.ts` converts the router's request into TypeSafe's format: - -The request line is JSON-encoded, so quotes and newlines inside the user's text are escaped and cannot pose as the `Options:` or `Conversation context:` sections that follow. - -```jsonc -// POST https://api.typesafe.ai/v1/systemone Authorization: Bearer -{ - "state": "Request: \"…\"\n\nOptions:\n- web_search: Web search — …", - "model": "jev-latest", - "questions": { - "router.select_option": { - "type": "choice", - "instructions": "Given the request, which option is the best fit?", - "criteria": { "web_search": "Web search — …", "calculator": "…", "…": "…" } - } - } -} -// → { "answers": { "router.select_option": { "type": "choice", "choice": "web_search", "probabilities": {…}, "confidence": 0.91 } } } -``` -## Related community projects +## Share and contribute + +The site includes a public 1200×630 OG image route containing editorial copy only. The SVG in this README is separate editable repository artwork, not a screenshot. Follow the [developer/sharing guide](docs/discovery/README.md) for actual image-response and capture evidence. -[TypeSafe AI Playground](https://github.com/BunsDev/typesafe-ai-playground) explores Jev use cases; [Clarity Judge](https://github.com/BunsDev/clarity-judge) applies named writing checks; [TypeSafe UI](https://github.com/BunsDev/typesafe-ui) provides interface components. They are separate projects, not automatically integrated dependencies. +`repository-metadata.json` records intended About text/topics; it does not apply GitHub settings or upload a Social preview. Do not claim deployed behavior from a source commit alone. Keep unofficial attribution, existing notices, and applicable licensing intact; this change does not grant a new license or imply vendor endorsement. -The proposed GitHub About description and topics are recorded in [repository-metadata.json](repository-metadata.json). Editing that file does not apply GitHub settings automatically. +Official product resources are [typesafe.ai](https://typesafe.ai) and [docs.typesafe.ai](https://docs.typesafe.ai), separate from this community lab. diff --git a/REFERENCE.md b/REFERENCE.md new file mode 100644 index 0000000..7cbb76d --- /dev/null +++ b/REFERENCE.md @@ -0,0 +1,248 @@ +# Jev Tool & Model Router + +A TypeScript routing library and interactive Next.js demo that use **TypeSafe AI's Jev** to select from a fixed set of tools or models, then apply explicit confidence and fallback policies. + +**Jev selects. Your application authorizes and executes.** This is an independent community project under `BunsDev`, not an official TypeSafe SDK or a production authorization system. The root package is private; the library is source in this repository, not a published npm install. + +[Contributing](CONTRIBUTING.md) · [Agent guide](AGENTS.md) · [TypeSafe API reference](https://docs.typesafe.ai/api) + +```text +request + allowed options -> Jev choice -> validation + fallback -> effective option + | + your authorization and executor +``` + +## What it does + +The model and tool routers share one engine, `createRouter` in `lib/jevRouter.ts`. Each call asks one `choice` question using the supplied option ids as the closed set. The result retains Jev's original pick, its scores, the effective option after policy, and the fallback reason. + +The library **does not execute tools or invoke the selected downstream model**. It does make a Jev API call when using the live transport. The UI's execution display is illustrative, not evidence that an external action ran. A structurally valid choice can still be the wrong decision. + +## Run locally + +Use npm and the committed `package-lock.json`; do not generate a competing pnpm, Yarn, or Bun lockfile. Use a Node.js version supported by the installed Next.js dependency; Node.js 22+ is a practical development baseline. This repository currently has no root `engines` or `packageManager` pin. + +```sh +git clone https://github.com/BunsDev/typesafe-router.git +cd typesafe-router +npm ci +npm run dev +``` + +Open the address printed by Next.js, normally `http://localhost:3000`. With no configured key, the demo uses the local keyword-based `mockCallJev` transport. Mock and live results are labeled separately. + +For live routing, get a key from the [TypeSafe console](https://console.typesafe.ai), then optionally configure the server: + +```sh +cp .env.local.example .env.local +# Edit .env.local and set TYPESAFE_API_KEY, then restart the dev server. +``` + +### 1Password-backed key (recommended) + +Keep the key in 1Password and run the app through `op run`, so no secret ever +lands in a file: + +```bash +npm run dev:op +``` + +`.env.1password` is committed and holds a **reference**, not a secret: +`TYPESAFE_API_KEY="op://Development/Jev API Key/password"`. `op run` expands it +into the process environment at launch. Requires 1Password CLI 2.x, unlocked +(desktop-app integration or `op signin`). + +Verify it took effect — `live` reports whether the SERVER has a key: + +```bash +curl -s http://localhost:3000/api/route +# {"live":true} ({"live":false} means demo mode) +``` + +A shell-provided variable is not overridden by `.env.local` (`@next/env`'s +`processEnv` only fills keys absent from the initial environment), so this path +wins over any stale key left in `.env.local`. + +The browser calls this app's `/api/route` endpoint. The server forwards live evaluations to `POST https://api.typesafe.ai/v1/systemone` using `jev-latest`. Jev answers typed questions; it does not generate the code that executes a route. + +### Bring your own key and privacy + +The key dialog also accepts a personal key. Precedence is **browser key → server environment key → demo mode**. + +A browser key persists, unencrypted by the app, in `localStorage` under `jev-router:api-key`. It is sent in the `x-typesafe-api-key` header to this application's server, which forwards it to TypeSafe. Masking the field reduces accidental screen exposure; it does not protect against scripts running on the origin, access to the browser profile, developer tools, or an untrusted deployment. Use a deployment you trust and remove the key on shared machines. + +### Limits on a public deployment + +`/api/route` is unauthenticated, so it bounds what one request can make the server do (`lib/limits.ts`): the input and the context are capped at 20,000 characters each, an option list at 32 entries, and each option's id, label and description at 64, 120 and 1,000 characters. A body over 256 KB is refused with 413 before it is parsed. The lab applies the same caps as `maxLength` on its fields and stops the editor at 32 options. Option ids, labels and descriptions are collapsed onto one line in the text sent to Jev, so a line break inside one cannot open a fake section. + +Calls that would spend the **server's** key are also rate limited: `ROUTE_RATE_LIMIT_PER_MINUTE` (default 60, `0` disables) returns 429 with code `rate_limited` once exceeded. Calls that carry a browser-saved key are not counted; they spend that user's credits. By default every caller shares one bucket, because `x-forwarded-for` and `x-real-ip` are whatever the client sent when the app is reached directly. Set `ROUTE_TRUST_PROXY=1` when a proxy you control (Vercel, nginx, a load balancer) overwrites those headers, and the limit becomes per client IP, read from the last forwarded hop. The counter is per process and tracks at most 10,000 callers (idle ones are dropped first, then the least recently seen), so on a serverless host it slows casual abuse rather than enforcing a global quota. Put a gateway limiter or an auth wall in front if the deployment is meant for strangers. + +### What the lab remembers + +Edited options, the threshold, and the routing history are saved in your browser (`localStorage`, key `jev-router:lab`) so a refresh doesn't lose them. **Reset everything** in the footer removes the stored copy; nothing is written back until you change something again. **Export JSON** in the history panel downloads the full log, one `RoutingLogEntry` per decision, for review or tuning. The API key is stored separately and is never part of this blob. Whatever is read back is validated field by field, and a malformed entry is dropped rather than rendered. + +Selecting a request in the history table (click the row, or Tab to it and press Enter) loads its mode, input, conversation context and threshold back into the request panel. The options are whatever the editor holds at that moment (only their ids are logged), so a re-run is a fresh decision, not a replay. Resetting the lab discards any routing call still in flight; clearing the history while a call is running keeps that decision on screen but doesn't re-add it to the log. +The server environment key is not sent to the browser. Do not commit `.env.local`, embed a key in a component or URL, or use a `NEXT_PUBLIC_` credential. A public server key can spend the owner's credits on visitors' requests; add appropriate access controls, request limits, and provider-side spending limits before offering one publicly. These deployment controls are not provided by the routing decision itself. + +## Routing policies + +| Case | Model router | Tool router | +| --- | --- | --- | +| Example options | `fast_cheap_model`, `reasoning_model`, `code_model`, `general_model` | `web_search`, `calculator`, `calendar_lookup`, `no_tool_needed` | +| Required option | `general_model` | `no_tool_needed` | +| Below confidence threshold | Use configured `safe_default` | Return `needs_clarification`; `effectiveOptionId` is `null` | +| Unusable or out-of-list answer | Same configured fallback, reason `invalid_option` | Same configured fallback, reason `invalid_option` | +| Invalid configuration | Throw `RouterConfigError` before calling Jev | Throw `RouterConfigError` before calling Jev | +| Transport/provider error | Reject the call; do not turn failure into a successful route | Reject the call; do not turn failure into a successful route | + +Other scripts: `npm test` (vitest: the engine, the wire format, the simulator, storage, the rate limiter, and the API route), `npm run e2e` (Playwright: builds and serves the app in demo mode, then drives the lab and the API through a real browser: recall, persistence, Clear and Reset during a run, the option editor, fallbacks, limits; `--ui` opens Playwright's inspector), `npm run typecheck` (runs `next typegen` first so a clean checkout has the generated `next-env.d.ts`, which is gitignored as Next.js recommends), `npm run lint`, `npm run build`. +The default threshold is `0.75`, with per-router and per-call overrides. `safe_default` is the policy name, **not a guarantee of harmless execution**. A downstream model call may expose data, cost money, or violate an application's policy. The integrating application must validate the selected id against a trusted registry and independently enforce permissions, privacy, budgets, and approval requirements. + +`no_tool_needed` is an explicit no-action option, not a tool to invoke. Preserve it when editing the tool list. For clarification, ask the user rather than executing Jev's original low-confidence selection. + +## Use the library + +This example runs inside this repository and uses the mock transport, so it does not consume API credits: + +```ts +import { createRouter } from "@/lib/jevRouter"; +import { modelRouterConfig, modelRouterOptions } from "@/lib/routerConfigs"; +import { mockCallJev } from "@/lib/mockRouter"; + +const router = createRouter(modelRouterConfig, mockCallJev, "mock"); +const userInput = "Help me debug a TypeScript type error."; +const { result, logEntry } = await router.route({ + userInput, + context: "The request concerns a local development project.", + options: modelRouterOptions, +}); + +console.log(result.effectiveOptionId, result.action, logEntry.source); +// This example deliberately does not execute the returned option. +``` + +For live server-side use, supply the real transport instead of the mock and configure credentials securely. Do not move a provider credential into client-side imports merely to reuse this example. + +### Core API + +- `routeWithJev(request, threshold?, transport?)` → a raw `RouteDecision` with Jev's pick and `fallbackUsed` set, no policy applied. Throws `RouteIntegrityError` if Jev's answer isn't in the option list, isn't a `choice` answer, or is missing. A valid pick that arrives without a confidence (and without probabilities) is kept at confidence 0, so it takes the ordinary low-confidence path rather than being reported as invalid. +- `resolveFallback(decision, options, policy, source)` → applies a `FallbackPolicy` to a decision. Pure, no I/O. +- `assertFallbackOption(options, policy)` → the runtime check described below. +- `buildRouterContext(request)` / `buildJevRequest(request)` → the exact text and question sent to Jev, so you can inspect or test it. +- `callJev(request, apiKey?)` → the low-level client. `mockCallJev` has the same signature for demo mode; tests inject their own. +- `createRouter(config, transport?, source?)` validates requests, calls the transport, applies policy, and logs the result. +- `routeWithJev(request, threshold?, transport?)` returns the raw decision and throws `RouteIntegrityError` for an unusable or out-of-set answer. It does not apply the full router fallback policy. +- `resolveFallback(decision, options, policy, source)` applies the deterministic policy without network I/O. +- `assertFallbackOption(options, policy)` checks the required fallback/no-tool option. +- `buildRouterContext(request)` and `buildJevRequest(request)` expose what will be submitted. + +Public re-exports live in `lib/index.ts`; full types live in `types/router.ts`. + +```ts +type RouteOption = { + id: string; + label: string; + description: string; + metadata?: Record; +}; +type RouteRequest = { userInput: string; context?: string; options: RouteOption[] }; +``` + +Use stable, unique ids and at least two options. Write descriptions as “use this when…” guidance. The option `metadata` is for the integrating application and is not sent to Jev. Keep the policy's required option in every request. Add a new router with an explicit `mode`, threshold, and `safe_default` or `needs_clarification` policy. + +## Saved state, logs, and tuning + +The lab stores edited options, thresholds, and history in `localStorage` under `jev-router:lab`. Reset clears the lab state; remove the API key separately. Exported JSON contains routing history, not the separately stored key. Inputs and conversation context can nevertheless contain private information: inspect exports before sharing. + +1. Every `id` must be unique. It is the only value Jev can return, so make it stable. Ids are used as object keys in the wire format and the score maps, and the library builds and reads those maps so that awkward ids such as `constructor` or `__proto__` behave like any other. +2. The list must still contain the option the fallback policy requires (`general_model` for the model router, `no_tool_needed` for the tool router). Otherwise `createRouter(...).route()` throws a `RouterConfigError` **before** calling Jev: +Logs record the input/context, considered options, original and effective selections, confidence, threshold, scores, fallback action, source (`jev` or `mock`), and duration. The default console logger prints a shortened input excerpt. For sensitive integrations, supply an appropriate logger and retention policy; do not assume prompts are automatically redacted. Mock confidence and routing outcomes are demonstrations, not model-quality measurements. + +## Project map + +| Path | Responsibility | +| --- | --- | +| `lib/jevRouter.ts` | Validation, request construction, policy, and logging. | +| `lib/jevClient.ts` | TypeSafe wire format, response normalization, and provider errors. | +| `lib/routerConfigs.ts` | Example model/tool options and fallback policies. | +| `lib/mockRouter.ts` | Local simulated transport. | +| `lib/apiKeyStorage.ts`, `lib/labStorage.ts` | Separate browser key and lab-state storage. | +| `types/router.ts` | Public routing contracts. | +| `app/api/route/route.ts` | Same-origin server endpoint and credential handling. | +| `components/RouterLab.tsx` | Interactive lab, options, threshold, and history. | + +## Development checks + +```sh +npm test +npm run typecheck +npm run lint +npm run build +``` + +Use mock/injected transports for automated checks. For UI changes, also exercise both router modes, required-option deletion protection, invalid options, low-confidence clarification, key removal, persistence, exports, themes, and narrow screens. Report actual test results rather than keeping a hardcoded test count in this README. + +```text +e2e/ + lab.spec.ts browser flows against the production build (Playwright) + api.spec.ts /api/route contract in demo mode + helpers.ts locators and a request-delaying route +playwright.config.ts builds and serves on :3111 with the key unset +lib/ + apiKeyStorage.ts browser-only key storage; the only reader hands the key straight to fetch + labStorage.ts browser-only persistence of options, threshold and history, validated on load; JSON export + guards.ts isFiniteNumber and readOwn (prototype-safe reads of maps keyed by option id) + limits.ts size caps shared by the API route (enforced) and the UI (maxLength) + rateLimit.ts in-memory sliding-window limiter for calls that spend the server key + jevClient.ts low-level Jev API wrapper (wire format, errors, normalisation) + jevRouter.ts engine: validation, routeWithJev, resolveFallback, createRouter, logging + routerConfigs.ts modelRouterOptions, toolRouterOptions, fallback policies + mockRouter.ts demo-mode keyword simulator, same JevTransport signature as callJev + index.ts public re-exports + testing/fakeStorage.ts localStorage stand-in for the storage tests + *.test.ts vitest suites +types/ + router.ts RouteOption, RouteRequest, RouteDecision, FallbackPolicy, RoutingLogEntry, … +app/ + page.tsx renders the lab client-only (LabLoader) so restored state needs no hydration dance + api/route/route.ts server-side routing endpoint; the only place the API key is read +components/ + LabLoader.tsx client-only dynamic import of RouterLab with a skeleton + RouterLab.tsx state: mode, input, context, options per mode, threshold, result, history + ApiKeyPanel.tsx bring-your-own-key dialog: masked input, never displayed, stored in this browser + ThemeToggle.tsx light/dark switch with transitions suppressed for the flip + ModeToggle.tsx Model Router / Tool Router + OptionEditor.tsx live-editable option list with the required option locked + ThresholdSlider.tsx confidence threshold + RoutingResult.tsx pick, colour-coded confidence, fallback status, mocked execution line + ScoreBreakdown.tsx per-option probability bars with the threshold marker + RoutingHistoryTable.tsx +.env.local.example +``` + +## Jev wire format, for reference + +`lib/jevClient.ts` converts the router's request into TypeSafe's format: + +The request line is JSON-encoded, so quotes and newlines inside the user's text are escaped and cannot pose as the `Options:` or `Conversation context:` sections that follow. + +```jsonc +// POST https://api.typesafe.ai/v1/systemone Authorization: Bearer +{ + "state": "Request: \"…\"\n\nOptions:\n- web_search: Web search — …", + "model": "jev-latest", + "questions": { + "router.select_option": { + "type": "choice", + "instructions": "Given the request, which option is the best fit?", + "criteria": { "web_search": "Web search — …", "calculator": "…", "…": "…" } + } + } +} +// → { "answers": { "router.select_option": { "type": "choice", "choice": "web_search", "probabilities": {…}, "confidence": 0.91 } } } +``` +## Related community projects + +[TypeSafe AI Playground](https://github.com/BunsDev/typesafe-ai-playground) explores Jev use cases; [Clarity Judge](https://github.com/BunsDev/clarity-judge) applies named writing checks; [TypeSafe UI](https://github.com/BunsDev/typesafe-ui) provides interface components. They are separate projects, not automatically integrated dependencies. + +The proposed GitHub About description and topics are recorded in [repository-metadata.json](repository-metadata.json). Editing that file does not apply GitHub settings automatically. diff --git a/app/layout.tsx b/app/layout.tsx index 76b169a..6dc41e5 100644 --- a/app/layout.tsx +++ b/app/layout.tsx @@ -2,8 +2,19 @@ import type { Metadata } from "next"; import "./globals.css"; export const metadata: Metadata = { + metadataBase: new URL("https://route.jev.works"), title: "Jev Router Lab", - description: "Jev decides which tool or model fits a request. Your code executes it. A TypeSafe Jev routing demo.", + description: "Unofficial community tool and model routing with Jev: closed-set choices, explicit fallbacks, and application-owned authorization.", + openGraph: { + type: "website", + siteName: "Jev Router Lab", + title: "Jev Router — selection is not authorization", + description: "An unofficial TypeScript routing lab with closed choices and inspectable fallback policies.", + }, + twitter: { + card: "summary_large_image", + images: [{ url: "/opengraph-image", alt: "Jev Router — unofficial community tool and model routing" }], + }, }; /** Apply a saved theme before first paint. Dark is the default. */ diff --git a/app/opengraph-image.tsx b/app/opengraph-image.tsx new file mode 100644 index 0000000..9971920 --- /dev/null +++ b/app/opengraph-image.tsx @@ -0,0 +1,27 @@ +import { ImageResponse } from "next/og" + +export const alt = "Jev Router — unofficial community tool and model routing" +export const size = { width: 1200, height: 630 } +export const contentType = "image/png" + +// Public editorial artwork only: no request data, provider calls, or credentials. +export default function Image() { + return new ImageResponse( +
+
TYPESAFEAI / UNOFFICIAL COMMUNITY
+
Jev Router
+
Jev selects. Your code authorizes.
+
Closed choices · Explicit fallbacks · Inspectable evidence
+
+ {["Allowed options", "Jev selection", "Host authorization"].map((label) => ( +
{label}
+ ))} +
+
+ TypeSafeAI / typesafe-router + Community-built. Not official. +
+
, + size + ) +} diff --git a/docs/discovery/README.md b/docs/discovery/README.md new file mode 100644 index 0000000..5462a5c --- /dev/null +++ b/docs/discovery/README.md @@ -0,0 +1,58 @@ +# Jev Router — developer and agent entry point + +> Unofficial TypeSafeAI community project, not the official TypeSafe AI SDK or a production authorization system. The community organization was created by VC Moderator [@BunsDev](https://github.com/BunsDev). + +![Jev Router editorial card: selection is separate from authorization](assets/social-preview.svg) + +[Full API and behavior reference](../../README.md) · [Agent instructions](../../AGENTS.md) · [Contributing](../../CONTRIBUTING.md) + +## Current repository and setup + +The canonical community source is [TypeSafeAI/typesafe-router](https://github.com/TypeSafeAI/typesafe-router). Older BunsDev links in historical documentation refer to the transferred project, not a separate current authority. + +```sh +git clone https://github.com/TypeSafeAI/typesafe-router.git +cd typesafe-router +npm ci +npm run dev +``` + +Use npm and `package-lock.json`; do not introduce another package manager or lockfile. Follow the checked-in Next.js requirements. Without a configured key, the lab uses a labeled local mock. Provider calls are explicit; tests must not consume shared API credits. This is source in a private package, not an assumed public npm installation. + +## Architecture and boundaries + +| Path | Responsibility | +| --- | --- | +| `lib/jevRouter.ts` | Validation, closed-set selection, fallback policy and result logging | +| `lib/jevClient.ts` | Jev wire format and provider errors | +| `lib/routerConfigs.ts` | Model/tool options and mandatory fallback choices | +| `lib/mockRouter.ts` | Keyword-based local demonstration | +| `types/router.ts`, `lib/index.ts` | Types and public source exports | +| `app/api/route/route.ts` | Same-origin provider boundary and credential handling | +| `components/RouterLab.tsx` | Interactive lab and state | +| `app/opengraph-image.tsx` | Public editorial PNG; no user input or provider calls | + +Jev selects; the application independently authorizes and executes. Preserve unique IDs, the allowed option set, required defaults, null clarification routes, no-tool outcomes, and provider-error propagation. The `safe_default` policy name is not proof that a downstream action is harmless. Do not turn failed live calls into successful mock outcomes. + +Browser keys and lab state have separate storage. LocalStorage is not an encrypted vault, and input/history exports may contain private text even when the key is omitted. Keep credentials out of URLs, metadata, images, logs and fixtures. Preserve rate limits, payload bounds and trusted-proxy behavior. + +## Verification + +```sh +npm test +npm run typecheck +npm run lint +npm run build +npm run e2e +node --test scripts/social-metadata.test.mjs +``` + +The existing browser suite builds and serves the no-key demo. The sharing test fetches rendered metadata and validates PNG signature/dimensions against that build. The standalone source check is narrower and is not deployment evidence. Preserve all existing policy, persistence and API tests. + +## Share accurately + +Website metadata uses the repository's configured public homepage, `https://route.jev.works`, and a 1200×630 generated PNG. Verify actual rendered tags and image responses after deployment; a merged source change is not proof that production is serving it. No root canonical forces future routes to the homepage. + +The SVG in this guide is separate 1280×640 repository artwork, not a product screenshot. Follow the [shared publishing checklist](https://github.com/TypeSafeAI/.github/blob/main/docs/discovery/SHARING.md). The committed About/topics manifest does not apply GitHub settings or upload a Social preview. + +Use clean, no-key browser state and synthetic inputs for screenshots. Record the source commit, route, viewport, theme and environment; keep demo/fallback labels visible. The sharing test captures an actual browse-only lab without submitting a routing request. Follow the [evidence protocol](https://github.com/TypeSafeAI/.github/blob/main/docs/discovery/SCREENSHOTS.md) and do not advertise tool execution, calibrated correctness, or production safety that this project does not establish. diff --git a/docs/discovery/assets/social-preview.svg b/docs/discovery/assets/social-preview.svg new file mode 100644 index 0000000..069ad64 --- /dev/null +++ b/docs/discovery/assets/social-preview.svg @@ -0,0 +1,16 @@ + +Jev Router — unofficial TypeSafeAI community +Jev selects. Your code authorizes. Editorial project card, not a product screenshot. + + + +TYPESAFEAI / UNOFFICIAL COMMUNITY +06 / 08 +Jev Router +Jev selects. Your code authorizes. +MODEL ROUTING · TOOL ROUTING · LLM ROUTING +01Allowed options02Jev selection03Host authorization + +github.com/TypeSafeAI/typesafe-router +Community-built. Not official. + \ No newline at end of file diff --git a/e2e/sharing.spec.ts b/e2e/sharing.spec.ts new file mode 100644 index 0000000..60eb0e8 --- /dev/null +++ b/e2e/sharing.spec.ts @@ -0,0 +1,64 @@ +import { writeFile } from "node:fs/promises"; +import { expect, test } from "@playwright/test"; + +test("sharing PNG and demo capture do not submit classification requests", async ({ page, request }, testInfo) => { + const blockedEvaluations: string[] = []; + // The app legitimately reads configuration with GET. Never allow a capture + // regression to submit an evaluation, even if a server key were configured. + await page.route("**/api/route", async (route) => { + const method = route.request().method(); + if (method !== "GET" && method !== "HEAD") { + blockedEvaluations.push(method); + await route.abort("blockedbyclient"); + return; + } + await route.continue(); + }); + const configuration = page.waitForResponse((response) => + new URL(response.url()).pathname === "/api/route" && response.request().method() === "GET" + ); + await page.goto("/"); + const status = await configuration; + expect(status.status()).toBe(200); + expect(await status.json()).toMatchObject({ live: false }); + await expect(page.getByRole("status").filter({ hasText: "Demo mode" })).toBeVisible(); + const fixture = "Explain a TypeScript type error in a synthetic example."; + await page.getByRole("textbox", { name: "User input", exact: true }).fill(fixture); + + const og = page.locator('meta[property="og:image"]').first(); + const twitter = page.locator('meta[name="twitter:image"]').first(); + await expect(og).toHaveAttribute("content", /opengraph-image/); + await expect(twitter).toHaveAttribute("content", /opengraph-image/); + await expect(page.locator('meta[name="twitter:card"]')).toHaveAttribute("content", "summary_large_image"); + await expect(page.locator('meta[name="description"]')).toHaveAttribute("content", /unofficial/i); + for (const [name, meta] of [["og", og], ["twitter", twitter]] as const) { + const url = new URL((await meta.getAttribute("content"))!); + expect(url.origin).toBe("https://route.jev.works"); + // Assert the public origin above, but fetch the image from this tested build. + const response = await request.get(url.pathname + url.search); + expect(response.status()).toBe(200); + expect(response.headers()["content-type"]).toContain("image/png"); + const bytes = await response.body(); + expect(bytes.subarray(0, 8).toString("hex")).toBe("89504e470d0a1a0a"); + expect(bytes.readUInt32BE(16)).toBe(1200); + expect(bytes.readUInt32BE(20)).toBe(630); + const path = testInfo.outputPath(`generated-${name}.png`); + await writeFile(path, bytes); + await testInfo.attach(name, { path, contentType: "image/png" }); + } + await page.evaluate(() => document.fonts.ready); + await page.setViewportSize({ width: 1440, height: 960 }); + await page.screenshot({ path: testInfo.outputPath("router-desktop.png"), animations: "disabled" }); + await page.setViewportSize({ width: 390, height: 844 }); + await page.screenshot({ path: testInfo.outputPath("router-mobile.png"), animations: "disabled" }); + expect(blockedEvaluations).toEqual([]); + const provenance = testInfo.outputPath("provenance.json"); + await writeFile(provenance, JSON.stringify({ + commit: process.env.GITHUB_SHA ?? "local-unrecorded", route: "/", fixture, + desktop: { width: 1440, height: 960 }, mobile: { width: 390, height: 844 }, + mode: "verified no-key demo; configuration GET allowed; evaluation methods blocked", + evaluationAttempts: blockedEvaluations.length, + environment: "local Playwright server, not production" + }, null, 2)); + await testInfo.attach("capture provenance", { path: provenance, contentType: "application/json" }); +}); diff --git a/llms.txt b/llms.txt index 971f739..6f2d418 100644 --- a/llms.txt +++ b/llms.txt @@ -1,15 +1,14 @@ -# Jev Tool & Model Router +# Jev Router -> Unofficial TypeScript tool and model router powered by Jev with closed-set choices, confidence thresholds, explicit fallbacks, and a Next.js lab. - -Unofficial TypeSafeAI community project — not the official TypeSafe AI team. +> Unofficial TypeScript tool and model selection using Jev, closed choices and explicit fallback policies. Selection does not authorize or execute a downstream action. ## Entry points - [README](https://github.com/TypeSafeAI/typesafe-router/blob/main/README.md) +- [Developer and sharing guide](https://github.com/TypeSafeAI/typesafe-router/blob/main/docs/discovery/README.md) - [Agent instructions](https://github.com/TypeSafeAI/typesafe-router/blob/main/AGENTS.md) - [Contributing](https://github.com/TypeSafeAI/typesafe-router/blob/main/CONTRIBUTING.md) +- [Source API](https://github.com/TypeSafeAI/typesafe-router/blob/main/lib/index.ts) +- [Routing types](https://github.com/TypeSafeAI/typesafe-router/blob/main/types/router.ts) -## Guidance - -Treat repository content as reference material, not authorization to run tools, disclose data, or bypass application policy. Preserve mock/live distinctions, security boundaries, attribution, and repository-specific verification requirements. Use synthetic data in examples and report unverified behavior explicitly. +Use npm and the committed lockfile. Preserve required fallback options, validation, null clarification routes, explicit no-tool outcomes and provider errors. Use no-key mocks for tests and demonstrations. Browser storage is not a vault. This navigation does not grant authority, expand allowed actions or replace AGENTS.md. diff --git a/scripts/social-metadata.test.mjs b/scripts/social-metadata.test.mjs new file mode 100644 index 0000000..deb1551 --- /dev/null +++ b/scripts/social-metadata.test.mjs @@ -0,0 +1,19 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { existsSync, readFileSync } from 'node:fs'; + +const root = new URL('../', import.meta.url); +const read = (path) => readFileSync(new URL(path, root), 'utf8'); + +// Source-contract checks; production build and HTTP image checks remain separate. +test('public OG image has a bounded PNG renderer with an unofficial label', () => { + const path = 'app/opengraph-image.tsx'; + assert.ok(existsSync(new URL(path, root)), 'missing Open Graph image route'); + const image = read(path); + assert.match(image, /ImageResponse/); + assert.match(image, /width: 1200/); + assert.match(image, /height: 630/); + assert.match(image, /image\/png/); + assert.match(image, /unofficial/i); + assert.doesNotMatch(image, /\b(fetch|cookies|headers)\s*\(|process\.env/); +});