Repository navigation
Build ManagedCode.LlmTestServer with isolated deterministic API testing - #1
Merged
Merged
Conversation
KSemenenko
had a problem deploying
to
github-pages
October 20, 2026 21:57 — with
GitHub Actions
Failure
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build ManagedCode.LlmTestServer as a deterministic LLM API test backend with one identity across the runtime, 19 packages, provider adapters, control client, Aspire integration, samples and documentation.
The branch adds isolated sessions, conversation graphs and correlated tool results; bounded request query/export and workflow verification; seeded wire metadata and fixture clocks; and twelve typed function-tool presets. Cache keys preserve provider/session ownership, and capacity failures are rejected before cache admission.
OpenAI Responses supports bounded completed native web-search, file-search and code-interpreter fixtures, with ordered outputs, citations, SSE lifecycle, mixed function rounds and continuation. Anthropic Messages adds bounded native web search, citations, six documented in-band search errors and strict official SDK JSON/SSE parsing. Native tools return scripted data; they do not execute searches or code.
Official SDK integration now covers Anthropic, Google Gemini, AWS Bedrock and OpenAI-compatible chat routes, in addition to existing OpenAI/Azure/Foundry clients. Fixtures use current documented model IDs, including the original Anthropic SDK Sonnet 5.5 identifier. Gemini's SDK batch-embedding route validates requested dimensions atomically. Gemini function rounds preserve opaque thought signatures in JSON/SSE, history, cache identity and retention; first-call presence and last locally issued signature checks are explicit, with broader text/multimodal signature support outside this slice.
Tool-failure integrations distinguish literal null, empty and absent results; controlled invocation exceptions and cancellation; provider 500/delays; filtered tool results; and explicit recovery with cache/history assertions. Text SSE on OpenAI Chat Completions, OpenAI Responses, Anthropic Messages and Gemini supports bounded early clean EOF, a cancellable post-frame delay and actual connection abort after a selected nonempty text frame. Official SDK tests over real Kestrel/TCP distinguish committed inference/cache effects from delivery, verify missing terminals even after all text arrives, replay and explicit recovery, and reject unsupported request modes atomically. Gemini replay excludes transport capability metadata from inference identity while preserving credential/provider/session ownership and checking the current delivery mode before replay. Tests synchronize real server startup and cancellation, drain HTTP hosts before disposal, and retain strict cache/queue rollback checks. Other E6 delivery families and malformed streams remain unfinished.
Native text output-policy fixtures now distinguish successful output refusal/filtering from input rejection and broken transport. OpenAI Chat maps typed refusal and content_filter; Responses maps refusal parts/events or incomplete content_filter; strict Anthropic parses refusal stop details; Gemini emits a terminal SAFETY candidate without duplicated content; DeepSeek Chat returns native content_filter with its own mandatory final-choice usage layout. Microsoft.Extensions.AI preserves refusal as ErrorContent. Real SDK tests cover terminal evidence, explicit recovery, graph/cache/session ownership, canonical stored/manual history and bounded diagnostics; unsupported surfaces fail before consuming the fixture. Raw JSON configuration accepts the documented Filtered/Refused names, and invalid configuration tests prove their intended runtime rejection after a successful baseline. DeepSeek rejects non-streaming stream_options before consumption. Raw empty-header TCP tests synchronize actual endpoint startup and drain requests before disposal. This slice does not qualify policy behavior for the remaining provider roots.
Launch preparation includes GitHub Pages and .NET SDK-based container distribution with management configuration. Owner visual/analytics changes remain outside this code checkpoint.
Validation for code checkpoint
3ce0742:Exact-head GitHub CI and website checks for this new checkpoint are pending. Previous checkpoint
0aea3dcpassed CI and website; those results do not qualify the new head.This remains a draft for the complete v1.0.0 launch. Other E5 native contracts and E6–E13 remain unfinished; public NuGet/GHCR/Pages launch has not occurred. The owner merges when the full batch is ready. Release acceptance requires actual public package availability, anonymous GHCR pull and the public installed consumer. No intermediate release or version bump is included.