SPT-VoiceLib (aowl.api)

One shared backend for voice, brain and world that any SPT 4.1.5 mod can talk to -- packaged as a BepInEx 5 plugin, Aowl.Api.dll, that other plugins depend on.

The heavy parts (speech-to-text, text-to-speech, the LLM "brain" behind each NPC, the persistent world of people, factions, caches and loot) live in one sidecar process, aowlspt-backend.exe, next to the game. This library owns everything that talks to it: discovery and start, the HTTP transport, the event stream with its cursor and hole detection, push-to-talk streaming, wav playback with a clip cache, the brain call and the typed world queries. A consumer never sends HTTP.

[BepInDependency(AowlApi.Guid)]          // "aowl.api", hard dependency
public sealed class MyPlugin : BaseUnityPlugin { ... }

Status: pre-release, tested in one live raid. The API is 0.1.0 and will change; see "What is not verified" at the end of docs/API.md.

Features

class	what it gives you
AowlBackend	sidecar start/stop, /status, State (Starting / Ready / Failed / Disabled), WaitReady, the version-compatibility verdict
AowlEvents	the long-polled event stream, a persisted cursor, hole and backend-restart detection, and a directive registry: any plugin owns any event kind (AowlEvents.Directives.Register("npc.goto", e => ...)); every directive is acked exactly once, by a handler or by the library with the reason
AowlBrain	Ask(personId, text) -- an NPC's reply, streamed sentence by sentence with a wav per sentence while the model is still writing the rest; Observe(kind, ...) -- report a fact the backend cannot see
AowlSpeech	Say(...) a wav at a Transform, a position or 2-D, with a clip cache; Listen(personId) -- push-to-talk: feed microphone samples at any rate, the library resamples to 16 kHz PCM16 and streams chunks, you get Partial and Final transcripts
AowlWorld	typed People / Person / Scene / Loot / Caches / Spawn queries; every refusal is a string in Result<T>.Error, never an exception
AowlHttp	the escape hatch for a route the library does not wrap; three outcomes (transport / status / JSON)

Every event, callback and handler runs on the Unity main thread in stream order; the methods that block are documented as such and have Async twins. The full surface, one example per class, is in docs/API.md; the wire protocol the library speaks is in docs/CLIENT-CONTRACT.md.

Install

Three things go into an SPT 4.1.5 install. Only the first is in this repo.

1. Aowl.Api.dll (+ Aowl.Api.xml for IntelliSense) -> BepInEx\plugins\. Build it (below) or take it from a release of this repo.
2. The sidecar -> BepInEx\plugins\aowlspt-sidecar\. It is not in this repo: the backend (aowlspt-backend.exe, the basement mod and its data) is built from the separate, private aowlspt project and ships as a release zip whose top level is the sidecar root below. Unzip it so that BepInEx\plugins\aowlspt-sidecar\aowlspt-backend.exe exists.
3. Engines (optional, for speech): a whisper.cpp binary + model for speech-to-text and a piper binary + voice for text-to-speech. Without them the backend still answers -- transcripts come back empty with a note naming the missing path, and say segments arrive as text with wav == "". The sidecar release notes say where the backend looks for them. The LLM key (ANTHROPIC_API_KEY) is read by the backend from the environment of the process that starts it -- the game, when the library auto-starts the sidecar. Nothing in this library holds or reads a key.

Then set [Backend] Enabled = true in BepInEx\config\aowl.api.cfg (it is off by default: the library loads, logs one line, and every consumer sees AowlBackend.State == Disabled).

The sidecar root

[Sidecar] SidecarRoot (default BepInEx\plugins\aowlspt-sidecar\) is passed to the backend as --root and must look like this -- which is what the release zip unpacks to:

aowlspt-sidecar\
  aowlspt-backend.exe
  mods\
    aowlspt-selection.json           {"schema":"aowlspt.selection/1","side":"server",
                                      "registry":"<abs path>\\registry\\mods.json","load":["aowl.basement"]}
    basement\
      basement.dll
      config.json                    "enabled": true, engine paths, hearing ranges, chatter switches
      data\                          ontology, names, archetypes, presets
  registry\
    mods.json                        {"schema":"aowlspt.registry/1", ..., "mods":[{"id":"aowl.basement", ...}]}

The backend serves http://127.0.0.1:6970/aowlspt/basement/*; GET .../status answering {"ok":true,"enabled":true,...} is the whole health check.

Config (F12 / BepInEx\config\aowl.api.cfg)

Every key is a BepInEx ConfigEntry, so the in-game F12 Configuration Manager shows and edits them under Aowl API.

section.key	default	meaning
Backend.Url	http://127.0.0.1:6970	sidecar base URL; every route is under /aowlspt/basement/
Backend.Enabled	false	master switch
Backend.PollWaitMs	20000	/events long-poll hold time (the backend clamps at 25000)
Backend.StatusEveryS	60	period of the status line (every counter, one log line); 0 = never
Sidecar.AutoStart	true	if GET /status does not answer within 2 s, start the sidecar
Sidecar.SidecarExe	<plugins>\aowlspt-sidecar\aowlspt-backend.exe	the backend executable
Sidecar.SidecarRoot	<plugins>\aowlspt-sidecar	passed as --root (layout above)
Sidecar.SidecarPort	6970	passed as --port; must match Backend.Url

The stream cursor persists in BepInEx\config\aowl.api.cursor, so a restart resumes where it left off instead of replaying old directives.

Your first plugin

Twenty lines: F9 asks the nearest NPC its name and plays the reply where it is spoken. The complete, commented version is examples/HelloAowl.

using Aowl.Api; using BepInEx; using UnityEngine; using System.Threading.Tasks;

[BepInPlugin("me.hello", "Hello", "0.1.0")]
[BepInDependency(AowlApi.Guid)]
public sealed class Plugin : BaseUnityPlugin
{
    void Awake() => AowlEvents.Directives.Register("hud.note", e => Logger.LogInfo("HUD: " + e.Str("text")));

    void Update()
    {
        if (!Input.GetKeyDown(KeyCode.F9) || !AowlBackend.IsReady) return;
        var here = Camera.main.transform.position;
        Task.Run(async () =>
        {
            var people = await AowlWorld.PeopleAsync();                     // Result<List<PersonInfo>>, never throws
            AowlApi.OnMain(() =>
            {
                if (!people.Ok) { Logger.LogWarning(people.Error); return; }
                var who = AowlWorld.Nearest(people.Value, here.x, here.y, here.z);
                var ask = AowlBrain.Ask(who.Id, "what is your name");
                ask.Sentence += s => AowlSpeech.Say(s.Text, s.Wav, here + Vector3.forward * 2f, who.Id);
                ask.Failed   += why => Logger.LogWarning(why);
            });
        });
    }
}

Reference Aowl.Api.dll with <Private>false</Private> and ship your DLL next to it in BepInEx\plugins.

The compatibility rule

GET /status carries the backend's version and schema. The library declares BackendMinVersion and BackendMaxKnownVersion (both 0.1.0 today) and requires schema to start with aowlspt.basement.status/. The verdict -- compatible, backend OLDER than the minimum X, backend NEWER than the newest checked Y, schema mismatch (...) -- is logged once at startup and kept in AowlBackend.Compatibility. The library keeps running either way: each call reports its own refusal, so an older backend degrades route by route with a named reason instead of dying at startup. A plugin that needs a route added after 0.1.0 should read AowlBackend.BackendVersion and refuse with its own line. Unknown fact kinds sent to the backend are accepted and ignored; unknown directive kinds arriving from it are acked ok:false "unsupported kind", never dropped silently.

Build

dotnet build SPT-VoiceLib.sln -c Release -p:OutDir=<dir>\
    -> <dir>\Aowl.Api.dll  Aowl.Api.xml  HelloAowl.dll

The game references (UnityEngine*.dll, Newtonsoft.Json.dll) come from an SPT 4.1.5 install: D:\SPT415 by default, overridden with -p:SptInstallDir=<path> or the SPT_INSTALL_DIR environment variable. Without -p:OutDir the build writes straight into <SptInstallDir>\BepInEx\plugins\. BepInEx.Core / BepInEx.PluginInfoProps restore from the BepInEx feed in NuGet.Config (they are not on nuget.org). Needs the .NET SDK (any recent one; 8 and 10 are known to work).

CI (.github/workflows/build.yml) has no game, so it builds with -p:GameRefsFromNuGet=true, which swaps the install's DLLs for the UnityEngine.Modules and Newtonsoft.Json packages from nuget.org. That is a real compile of the same sources, and it uploads the DLLs as an artifact, but the DLL you ship should be the one built against the game.

Layout

src/Aowl.Api/            the library (BepInEx plugin "aowl.api"); XML docs on every public member
examples/HelloAowl/      the smallest useful consumer
docs/API.md              the surface, class by class, with the threading rules
docs/CLIENT-CONTRACT.md  the wire protocol (routes, event kinds, acks, speech format)

License

MIT, see LICENSE.