Add tool calling to the Llama provider

[ActuallyTaylor]

LlamaLanguageModel ignored session.tools entirely — no tool calling in respond or streamResponse.

What the binding actually supports: nothing. LlamaSwift re-exports the raw llama.cpp C API. llama_chat_message is just { role, content }, and llama_chat_apply_template has no tools parameter and explicitly does not use a Jinja parser. llama_model_chat_template() returns the GGUF's template text, but nothing in the binding can render it. llama.cpp's tool-aware path lives in its common library, which isn't in the xcframework. So tools cannot be injected through the model's own chat template.

Tools are therefore advertised through a synthesized system message carrying each tool's name, description and JSON Schema. Parsing is deliberately more permissive than emission, covering Hermes/Qwen <tool_call>, Llama 3.x <|python_tag|> (including ;-separated calls), Mistral [TOOL_CALLS], <function=name>, and bare leading JSON — the last only when every parsed name matches a registered tool, so ordinary JSON answers aren't swallowed. Brace matching is string- and escape-aware.

Both paths honor the full delegate contract and the transcript ordering, with an 8-iteration cap and repeated-signature detection. Because tool markup arrives inline with prose, streaming withholds text that has begun or could still grow into a marker.

Also fixes a pre-existing break: the Llama trait did not compile at all. use_mmap/use_mlock were replaced upstream by a load_mode enum, and Package.resolved had no llama.swift entry, so the trait had never been resolved. This is the same fix already pushed to #3.

22 ungated parser unit tests run with no model present — the only new logic in this series that CI actually exercises. Verified live against Qwen2.5-3B-Instruct Q4_K_M: 54/54 pass including withTools and streamWithTools.

Tool calling applies to String generation only, matching MLX.

Based on #4; merge that first.

🤖 Generated with Claude Code

[ActuallyTaylor]

Both review points addressed, in two commits.

KV cache should be trimmed back, not dropped

585f25c. Each tool iteration cleared the cache and re-decoded the entire conversation from position 0, so the work grew with the conversation even though an iteration only appends a message or two. The cache now keeps the prefix it shares with the next prompt and trims only the divergent tail.

Per the request to check this against llama-server, I read the llama.h contracts rather than assuming, and the first draft was unsound in three ways that the guards now cover:

• llama_memory_seq_rm returns false when a partial removal isn't supported — the header says so explicitly. My first version ignored the return value, which would have left the cache inconsistent with the assumed prefix on recurrent models. It now falls back to discarding the sequence.
• Sliding-window caches evict the head, so llama_memory_seq_pos_min is non-zero and position 0 is already gone; a prefix anchored at 0 can't be trusted.
• Recurrent models (llama_model_is_recurrent — Mamba, RWKV) carry a rolling state with no per-token prefix, and encoder models hold state these tokens don't describe.

Any of those falls back to a full clear and a decode from scratch. At least one token is always re-decoded so the batch still produces logits to sample from.

Use a tool role where the template supports one

d22b9a2. Tool results were always replayed as user messages; this also brings the provider in line with MLX and Core ML, which both append tool-role messages.

llama_chat_apply_template implements a fixed set of built-in templates — ChatML, Qwen and Hermes render a tool role, while Llama 2 and Mistral fold unknown roles into the user turn where the role name would leak into the prompt. Rather than hardcode which templates qualify, the same content is rendered under both roles and compared: a template that distinguishes them produces different output. The result is memoized, and templates without tool support keep the previous behavior.

Verification, including something worth flagging

Build and lint clean; the 22 ungated parser tests and withTools pass against Qwen2.5-3B.

streamWithTools is flaky against this model, and was already flaky before these changes — measured, not assumed: 1/3 on the pre-change baseline vs 4/5 with these commits. The earlier "54/54 passing" run was luckier than the code deserved, and I shouldn't have reported it without repeat runs. The change looks neutral-to-better on reliability, but the sample is small and the underlying flakiness is a separate problem: the 3B model doesn't always emit parseable tool markup in the streaming path even though the non-streaming path is consistent.

🤖 Generated with Claude Code