Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 87 additions & 6 deletions site/src/docs/typescript/basic-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,24 +53,30 @@ export interface TypeChatLanguageModel {
retryPauseMs?: number;
/**
* Obtains a completion from the language model for the given prompt.
* @param prompt The prompt string.
* @param prompt The prompt string or role-tagged prompt sections.
*/
complete(prompt: string): Promise<Result<string>>;
complete(prompt: string | PromptSection[]): Promise<Result<string>>;
}
```

then you should be able to try TypeChat out with such a model.

The key thing here is that only `complete` is required.
`complete` is just a function that takes a `string` and eventually returns a `string` if all goes well.
`complete` takes a string or an array of prompt sections and eventually returns text wrapped in a
`Result` if all goes well.

For convenience, TypeChat provides two functions out of the box to connect to the OpenAI API and Azure's OpenAI Services.
You can call these directly.

```ts
export function createOpenAILanguageModel(apiKey: string, model: string, endPoint?: string): TypeChatLanguageModel;

export function createAzureOpenAILanguageModel(apiKey: string, endPoint: string): TypeChatLanguageModel;
export function createOpenAILanguageModel(
apiKey: string | undefined, model: string, endPoint?: string,
org?: string, options?: LanguageModelOptions
): TypeChatLanguageModel;

export function createAzureOpenAILanguageModel(
apiKey: string | undefined, endPoint: string, options?: LanguageModelOptions
): TypeChatLanguageModel;
```

For even more convenience, TypeChat also provides a function to infer whether you're using OpenAI or Azure OpenAI.
Expand All @@ -82,6 +88,81 @@ export function createLanguageModel(env: Record<string, string | undefined>): Ty
With `createLanguageModel`, you can populate your environment variables and pass them in.
Based on whether `OPENAI_API_KEY` or `AZURE_OPENAI_API_KEY` is set, you'll get a model of the appropriate type.

#### Using a bearer-token provider

Pass `undefined` for the API key and supply your existing synchronous or asynchronous
token provider in the options:

```ts
const model = createAzureOpenAILanguageModel(undefined, endpoint, {
tokenProvider
});
```

`TokenProvider` is `() => string | Promise<string>`. Return the raw access-token string,
without a `Bearer ` prefix. The provider owns credential acquisition, caching, renewal,
and cleanup; TypeChat adds no Azure Identity dependency. It invokes the provider for each
HTTP attempt, including retries, and sends bearer authorization without an `api-key`
header. API keys and providers are mutually exclusive. Non-string/empty tokens or tokens
containing whitespace, control characters, or non-ASCII characters return an error before
dispatch. Provider exceptions follow the HTTP model's existing
retry/error policy.

Providers execute on the caller's JavaScript event loop. Synchronous providers should
return cached tokens quickly; use asynchronous SDK/network APIs for acquisition or renewal.
Adding `async` or calling `Promise.resolve` around synchronous I/O does not make that work
nonblocking. If a legacy provider requires blocking I/O, offload it using application-managed
workers (for example Node.js `worker_threads`) with an appropriate thread-safety and timeout
strategy. TypeChat invokes providers directly and does not automatically move them to workers.

The same option works with `createOpenAILanguageModel`, including its Responses mode:

```ts
const model = createOpenAILanguageModel(undefined, deployment, responsesEndpoint, "", {
tokenProvider,
useResponsesApi: true
});
```

The HTTP factories retain their proxy, timeout, response-size, and retry controls.
The HTTP request timeout does not bound credential acquisition inside your provider.
Existing string-key calls, environment handling, defaults, and legacy headers are unchanged;
`AZURE_OPENAI_API_KEY=identity` does not activate a provider.

#### Reusing a configured client

Adapt an existing OpenAI/AzureOpenAI SDK client, or a structurally compatible asynchronous
client, without importing an SDK into TypeChat:

```ts
const model = createOpenAILanguageModelFromClient(client, modelOrDeployment);
const responsesModel = createOpenAILanguageModelFromClient(client, modelOrDeployment, {
useResponsesApi: true
});
```

Chat Completions is the default and calls `client.chat.completions.create`. Responses mode
is explicit and calls `client.responses.create`; a Chat-only client does not need a
`responses` property. TypeChat requests non-streaming output at temperature zero, with
`n=1` only for Chat Completions.

The adapter exposes only `complete`, retaining the supplied client's endpoint, API version,
authentication, retry/timeout settings, and ownership. It does not configure or close the
client or add another network-control layer. Configure proxies, timeouts, retries, and
response-size safeguards on that client, not on the adapter. Client exceptions and
cancellation propagate unchanged; missing/malformed output and filtered/refused content
return TypeChat errors.

String prompts and text/image user messages are supported. Bare strings in content arrays
become text parts; images retain URLs/data URLs and detail levels. Responses mode maps them
to `input_text`/`input_image` parts, defaulting omitted image detail to `"auto"`. System and
assistant messages support text only; unsupported content is reported rather than dropped.
Neither path fetches image references separately.

Pass either model to `createJsonTranslator` as usual. Schema validation, repair, and
reported response metadata/filtering behavior are preserved. SDK metadata covers the
resolved responses visible to TypeChat, not HTTP attempts hidden inside the client's retries.

#### Using OpenAI-compatible gateways

TypeChat works with any service that exposes the OpenAI Chat Completions API.
Expand Down
12 changes: 12 additions & 0 deletions typescript/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,18 @@ To see TypeChat in action, we recommend exploring the [TypeChat example projects

To learn more about TypeChat, visit the [documentation](https://microsoft.github.io/TypeChat) which includes more information on TypeChat and how to get started.

## Reusing model authentication

The HTTP model factories accept a synchronous or asynchronous `tokenProvider` option
with `undefined` for the API key. Existing string-key and environment-based calls are
unchanged. For an already configured OpenAI/AzureOpenAI or compatible client, use
`createOpenAILanguageModelFromClient`, optionally with `{ useResponsesApi: true }`.

These paths add no SDK dependency to TypeChat. See the
[token-provider example](../site/src/docs/typescript/basic-usage.md#using-a-bearer-token-provider)
and [configured-client example](../site/src/docs/typescript/basic-usage.md#reusing-a-configured-client)
for multimodal prompts, network-control ownership, and error behavior.

## Accessing failed output

Failed results include a `message` and may also include `data: unknown` and `responseText: string`.
Expand Down
6 changes: 6 additions & 0 deletions typescript/metadata.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ response that is filtered or refused returns a concise error rather than putting
provider body or refusal text in the error message. Filtering details remain in metadata, and
available text remains in `responseText`.

Models adapted by `createOpenAILanguageModelFromClient` use the same reported-usage and
filtering parser for Chat Completions and Responses output. Their records cover resolved
SDK responses visible to TypeChat, including translator repair, not requests/retries hidden
inside the supplied client. Client exceptions propagate instead of creating fabricated
response records. HTTP token-provider models retain the existing per-HTTP-response accounting.

`metadata.usage` sums **reported counts only**, once per record: input/output/total 13/7/20 plus
5/3/8 yields 18/10/28. Each corresponding `metadata.usageReporting` field is `"complete"` if every
record reports that count, otherwise `"partial"`. If no record reports a count, its total and
Expand Down
Loading
Loading