Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ All you need to do is change your import statement:
+ import AnyLanguageModel
```

Anything AnyLanguageModel adds beyond Apple's API
is marked as an extension in its documentation.
[Differences from Foundation Models](#differences-from-foundation-models)
summarizes these extensions
and lists the few Foundation Models APIs it doesn't implement yet.

```swift
struct WeatherTool: Tool {
let name = "getWeather"
Expand Down Expand Up @@ -373,6 +379,10 @@ use a vision-capable model
For llama.cpp,
pass the model's multimodal projector with `mmprojPath:`.

> [!NOTE]
> Image inputs are an AnyLanguageModel extension.
> See [Differences from Foundation Models](#differences-from-foundation-models).

### Tool Calling

Tool calling is supported by all providers.
Expand Down Expand Up @@ -432,6 +442,10 @@ actor ToolExecutionObserver: ToolExecutionDelegate {
session.toolExecutionDelegate = ToolExecutionObserver()
```

> [!NOTE]
> Tool execution delegates are an AnyLanguageModel extension.
> See [Differences from Foundation Models](#differences-from-foundation-models).

### Token Usage

Inspect token counts with `response.usage`
Expand Down Expand Up @@ -492,6 +506,69 @@ the latest value is kept for each metadata key.
> [Foundation Models 27 usage API](https://developer.apple.com/documentation/foundationmodels/languagemodelsession/usage-swift.struct).
> `Codable` and `Equatable` support are AnyLanguageModel extensions.

## Differences from Foundation Models

Code that uses an extension doesn't compile with Foundation Models on OS 26.
Each extension's documentation comment says so,
and extensions that follow a Foundation Models 27 API,
such as token usage,
say which API they follow.

### Extensions

- `LanguageModel` and `Availability`:
the protocol that lets a session use any model provider.
Foundation Models 27 adds its own `LanguageModel` protocol
with different requirements.
Session initializers require a `model:` argument
instead of defaulting to `SystemLanguageModel.default`.
- `GenerationOptions[custom:]` and `CustomGenerationOptions`:
options for one provider,
described in that provider's section under [Providers](#providers).
- `respond(to:images:)`, `Transcript.ImageSegment`, and `Transcript.Segment.image`:
[image inputs](#image-inputs).
Foundation Models 27 adds prompt attachments,
and AnyLanguageModel 2.0 will change to match them.
- `ToolExecutionDelegate`, `ToolExecutionDecision`, and `toolExecutionDelegate`:
[observing and controlling tool calls](#tool-calling).
- `transcriptErrorHandlingPolicy` and `waitForResponseCompletion()`:
what a transcript keeps when a request fails or is cancelled.
- `LanguageModelSession.tools` and `instructions`:
the session's tools and instructions,
for language models defined outside AnyLanguageModel.
- `Usage` and the `usage` properties:
[token usage](#token-usage),
which follows the Foundation Models 27 API.
- Public initializers for `Response`, `ResponseStream`, `ResponseStream.Snapshot`,
`GenerationGuide`, and `LanguageModelFeedback`,
for language models defined outside AnyLanguageModel.
- `Codable` conformance for `GeneratedContent`, `GenerationID`, `Usage`,
and the types nested in `Transcript`.
- `GeneratedContentError` and each provider's error type.
- `JSONValue`:
JSON values for provider options such as `extraBody`.
Comment on lines +548 to +549
- Smaller additions to existing types,
such as `LanguageModel.isAvailable`, `SystemLanguageModel.supportsImageInput`,
`Transcript.ToolCall.providerMetadata`, and `GenerationError.Refusal.transcriptEntries`.

### Behavior differences

- A session adds the prompt to its transcript before the model responds,
as a single text segment.
- A structured response is recorded in the transcript as JSON text,
not as a structured segment.
- A session created from a transcript restores `instructions`
only when the first entry is instructions with a single text segment.

### Not yet implemented

- `@Generable(description:representNilExplicitlyInGeneratedContent:)`
and the matching `GenerationSchema` and `DynamicGenerationSchema` initializers
- `DynamicGenerationSchema.null`
- `logFeedbackAttachment(sentiment:issues:desiredResponseText:)`
and `logFeedbackAttachment(sentiment:issues:desiredResponseContent:)`
- `SystemLanguageModel.supportedLanguages`, `supportsLocale(_:)`, and `tokenCount(for:)`

## Providers

### Apple Foundation Models
Expand Down
9 changes: 9 additions & 0 deletions Sources/AnyLanguageModel/Availability.swift
Original file line number Diff line number Diff line change
@@ -1,4 +1,13 @@
/// The availability status for a specific language model.
///
/// This generic type replaces Foundation Models' `SystemLanguageModel.Availability`
/// so that each model can report its own reasons for being unavailable.
/// The cases have the same names,
/// so a `switch` over a model's `availability` works with both frameworks.
///
/// - Note: This API is exclusive to AnyLanguageModel
/// and using it means your code is no longer drop-in compatible
/// with the Foundation Models framework.
public enum Availability<UnavailableReason> {
/// The model is ready for making requests.
case available
Expand Down
13 changes: 13 additions & 0 deletions Sources/AnyLanguageModel/GeneratedContent.swift
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ import CoreFoundation
/// A type that represents structured, generated content.
///
/// Generated content may contain a single value, an array, or key-value pairs with unique keys.
///
/// - Note: The `Codable` conformance is exclusive to AnyLanguageModel
/// and using it means your code is no longer drop-in compatible
/// with the Foundation Models framework.
public struct GeneratedContent: Sendable, Equatable, Generable, CustomDebugStringConvertible, Codable {
/// An instance of the generation schema.
public static var generationSchema: GenerationSchema {
Expand Down Expand Up @@ -338,6 +342,10 @@ extension GeneratedContent {
///
/// `Kind` represents the various types of JSON-compatible data that can be held within
/// a `GeneratedContent` instance, including primitive types, arrays, and structured objects.
///
/// - Note: The `Codable` conformance is exclusive to AnyLanguageModel
/// and using it means your code is no longer drop-in compatible
/// with the Foundation Models framework.
public enum Kind: Equatable, Sendable {

/// Represents a null value.
Expand Down Expand Up @@ -393,6 +401,11 @@ extension GeneratedContent {

// MARK: - GeneratedContentError

/// Errors that can occur when converting generated content to a value.
///
/// - Note: This API is exclusive to AnyLanguageModel
/// and using it means your code is no longer drop-in compatible
/// with the Foundation Models framework.
public enum GeneratedContentError: Error, Hashable {
case propertyNotFound(String)
case typeMismatch
Expand Down
5 changes: 5 additions & 0 deletions Sources/AnyLanguageModel/GenerationGuide.swift
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ public struct GenerationGuide<Value>: Sendable {
var minimum: Double?
var maximum: Double?

/// Creates a guide with no constraints.
///
/// - Note: This initializer is exclusive to AnyLanguageModel.
/// It's public for language models outside this module;
/// Foundation Models doesn't make it public.
public init() {}

init(minimumCount: Int?, maximumCount: Int?) {
Expand Down
6 changes: 5 additions & 1 deletion Sources/AnyLanguageModel/GenerationID.swift
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import struct Foundation.UUID
/// }
///
/// struct PeopleView: View {
/// @State private var session = LanguageModelSession()
/// @State private var session = LanguageModelSession(model: SystemLanguageModel.default)
/// @State private var people = [Person.PartiallyGenerated]()
///
/// var body: some View {
Expand Down Expand Up @@ -41,6 +41,10 @@ import struct Foundation.UUID
/// }
/// }
/// ```
///
/// - Note: The `Codable` conformance is exclusive to AnyLanguageModel
/// and using it means your code is no longer drop-in compatible
/// with the Foundation Models framework.
public struct GenerationID: Sendable, Hashable, Codable {
private let uuid: UUID

Expand Down
8 changes: 8 additions & 0 deletions Sources/AnyLanguageModel/GenerationOptions.swift
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,10 @@ public struct GenerationOptions: Sendable, Equatable {
///
/// - Parameter modelType: The language model type to get or set custom options for.
/// - Returns: The custom options for the specified model type, or `nil` if none are set.
///
/// - Note: This API is exclusive to AnyLanguageModel
/// and using it means your code is no longer drop-in compatible
/// with the Foundation Models framework.
public subscript<Model: LanguageModel>(
custom modelType: Model.Type
) -> Model.CustomGenerationOptions? {
Expand Down Expand Up @@ -219,6 +223,10 @@ extension GenerationOptions {
/// }
/// }
/// ```
///
/// - Note: This API is exclusive to AnyLanguageModel
/// and using it means your code is no longer drop-in compatible
/// with the Foundation Models framework.
public protocol CustomGenerationOptions: Equatable, Sendable {}

extension Never: CustomGenerationOptions {}
Expand Down
21 changes: 21 additions & 0 deletions Sources/AnyLanguageModel/LanguageModel.swift
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
import Foundation

/// A type that generates responses for a language model session.
///
/// Model providers conform to this protocol
/// so that a ``LanguageModelSession`` can use them.
/// A conforming type reports its availability,
/// generates complete and streamed responses,
/// and can define its own custom generation options.
///
/// - Note: This API is exclusive to AnyLanguageModel.
/// On OS 26, a Foundation Models session always uses `SystemLanguageModel`;
/// this protocol is what lets a session use any provider.
/// Foundation Models 27 adds its own `LanguageModel` protocol
/// with different requirements,
/// so using this protocol means your code is no longer drop-in compatible
/// with the Foundation Models framework.
public protocol LanguageModel: Sendable {
associatedtype UnavailableReason

Expand Down Expand Up @@ -100,6 +115,12 @@ extension LanguageModel {
)
}

/// A Boolean value that indicates whether the model is ready for requests.
///
/// - Note: This property is exclusive to AnyLanguageModel
/// and using it means your code is no longer drop-in compatible
/// with the Foundation Models framework.
/// Foundation Models provides `isAvailable` only on `SystemLanguageModel`.
public var isAvailable: Bool {
if case .available = availability {
return true
Expand Down
10 changes: 10 additions & 0 deletions Sources/AnyLanguageModel/LanguageModelFeedback.swift
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
/// Feedback about a model's response.
///
/// - Note: The ``sentiment`` and ``issues`` properties are exclusive to AnyLanguageModel.
/// They're public for language models outside this module;
/// Foundation Models doesn't make them public.
public struct LanguageModelFeedback {
/// A sentiment regarding the model's response.
public enum Sentiment: Sendable, CaseIterable, Equatable, Hashable {
Expand Down Expand Up @@ -92,6 +97,11 @@ public struct LanguageModelFeedback {
/// - Parameters:
/// - sentiment: A sentiment for the model's response.
/// - issues: Issues with the model's response.
///
/// - Note: This initializer is exclusive to AnyLanguageModel.
/// It's public so that language models outside this module
/// can create feedback;
/// Foundation Models doesn't make it public.
public init(sentiment: Sentiment, issues: [Issue]) {
self.sentiment = sentiment
self.issues = issues
Expand Down
Loading
Loading