feat(tts): MOSS-TTS-Nano backend (beta) — streaming zero-shot voice cloning, 48 kHz stereo

Documentation/Models.md

@@ -58,6 +58,7 @@ TDT/CTC and the non-autoregressive models above are wrapped by `SlidingWindowAsr
 | **Kokoro ANE (7-stage)** | Kokoro 82M weights split into 7 CoreML stages so the ANE-friendly layers (Albert / Prosody / Vocoder) stay resident on the Neural Engine while PostAlbert / Alignment / Noise / Tail run on CPU. 3-11× RTFx. English (`af_heart`) and Mandarin (`ANE-zh`) variants. ≤510 IPA phonemes per call, no chunker / SSML / custom lexicon. Managed by `KokoroAneManager`. | ANE-optimized variant derived (with permission) from [laishere/kokoro-coreml](https://github.com/laishere/kokoro-coreml). The original single-graph (mono) Kokoro backend was removed in favor of this ANE pipeline; the kokoro repo root is now retained only for shared G2P assets. |
 | **PocketTTS** | TTS backend (~155M params). Autoregressive frame-by-frame generation with dynamic audio chunking. No phoneme stage, works directly on text tokens. Managed by `PocketTtsManager`. | Supports streaming, minimal RAM usage, excellent quality |
 | **Supertonic-3** | Multilingual TTS, 31 languages (`en`, `ko`, `ja`, `ar`, `bg`, `cs`, `da`, `de`, `el`, `es`, `et`, `fi`, `fr`, `hi`, `hr`, `hu`, `id`, `it`, `lt`, `lv`, `nl`, `pl`, `pt`, `ro`, `ru`, `sk`, `sl`, `sv`, `tr`, `uk`, `vi`, plus `na` for numeric/language-agnostic input). 4-stage CoreML pipeline (text_encoder → duration_predictor → vector_estimator → vocoder, ~398 MB). Caller-supplied voice styles loaded from Supertonic preset JSON. 44.1 kHz mono fp32 output. Managed by `Supertonic3Manager`. | CoreML conversion of upstream `Supertone/supertonic-3`; see `Scripts/convert_supertonic3_to_coreml.py`. |
+| **MOSS-TTS-Nano (beta)** | 0.1B multilingual streaming TTS with zero-shot voice cloning (20 languages incl. `zh`, `en`, `de`, `es`, `fr`, `ja`, `ko`, `ru`, `ar`). GPT-2 global LM + local transformer over MOSS-Audio-Tokenizer-Nano codes, decoded per 80 ms frame by a streaming codec (`Prefill → Frame → CodecStep → Step`). Voices are codec-token sequences of a reference clip (two presets, or `cloneVoice(audioURL:)`). Native 48 kHz **stereo** fp32 output. Managed by `MossTtsNanoManager`. | CoreML conversion of upstream `OpenMOSS-Team/MOSS-TTS-Nano-100M`; see mobius `models/tts/moss-tts-nano/coreml` and `Documentation/TTS/MossTtsNano.md`. |
 | **StyleTTS2 (LibriTTS, iteration_3)** | Reference-audio–driven zero-shot English TTS. 8-stage CoreML pipeline (`text_encoder → bert → ref_encoder → fused_diffusion_sampler → duration_predictor → fused_f0n_har_source → decoder_pre → decoder_upsample`) with 3 lazily-loaded T = 64 / 128 / 256 bucket variants of `bert` / `fused_diffusion_sampler`. 5-step ADPM2 Karras-σ diffusion sampler with α/β style blending against a speaker reference clip. 24 kHz mono fp32 output. Phonemizer reuses Kokoro's Misaki lexicon cache + BART G2P CoreML model with Misaki uppercase diphthong shorthand (`A O I Y W` → `eɪ oʊ aɪ ɔɪ aʊ`) expanded before encoding so the output matches the espeak IPA the model was trained on. Callers with a higher-quality phonemizer can bypass the stack via `StyleTTS2Manager.synthesize(ipa:...)`. See [StyleTTS2.md](TTS/StyleTTS2.md). | Zero-shot voice cloning from a single reference WAV; English only |
 
 ## Evaluated Models (Not Supported)
@@ -93,6 +94,7 @@ Models we converted and tested but are not supported: too large for on-device de
 | PocketTTS | [FluidInference/pocket-tts-coreml](https://huggingface.co/FluidInference/pocket-tts-coreml) |
 | StyleTTS2 (LibriTTS, iteration_3) | [FluidInference/StyleTTS-2-coreml/iteration_3/compiled](https://huggingface.co/FluidInference/StyleTTS-2-coreml/tree/main/iteration_3/compiled) (shared phonemizer assets pulled from [`FluidInference/kokoro-82m-coreml`](https://huggingface.co/FluidInference/kokoro-82m-coreml): `G2PEncoder.mlmodelc`, `G2PDecoder.mlmodelc`, `g2p_vocab.json`, `us_lexicon_cache.json`) |
 | Supertonic-3 | [FluidInference/supertonic-3-coreml](https://huggingface.co/FluidInference/supertonic-3-coreml) |
+| MOSS-TTS-Nano (beta) | [FluidInference/moss-tts-nano-coreml](https://huggingface.co/FluidInference/moss-tts-nano-coreml) (`voices/*.json` presets, fp32 `MossNano-CodecEncoder-fp32.mlmodelc` fetched only for custom cloning) |
 | Nemotron Streaming (English) | [FluidInference/nemotron-speech-streaming-en-0.6b-coreml](https://huggingface.co/FluidInference/nemotron-speech-streaming-en-0.6b-coreml) (subdirs: `/560ms`, `/1120ms`, `/2240ms`) |
 | Nemotron Streaming (Multilingual) | [FluidInference/Nemotron-3.5-ASR-Streaming-Multilingual-0.6b-CoreML](https://huggingface.co/FluidInference/Nemotron-3.5-ASR-Streaming-Multilingual-0.6b-CoreML) (`<lang>/<tier>ms/` subfolders) |
 | Multilingual G2P (Charsiu ByT5) | [FluidInference/charsiu-g2p-byt5-coreml](https://huggingface.co/FluidInference/charsiu-g2p-byt5-coreml) |

Documentation/TTS/MossTtsNano.md

@@ -0,0 +1,141 @@
+# MOSS-TTS-Nano Swift Inference
+
+> **Beta.** This is a beta model conversion: the Swift API, the published model
+> artifacts and the output quality may change between releases.
+
+0.1B multilingual streaming TTS with zero-shot voice cloning. 20 languages, native
+48 kHz **stereo** Float32 output, 80 ms frames streamed as they are decoded.
+Six CoreML bundles (≈ 275 MB fp16 on disk; the fp32 voice encoder is fetched only
+for custom cloning).
+
+## Overview
+
+MOSS-TTS-Nano (OpenMOSS, Apache-2.0) is a pure autoregressive "audio tokenizer +
+LLM" model: a 12-layer GPT-2 (768-d, RoPE) reads rows of `[text token, 16 codec
+codes]`, a 1-layer local transformer emits the 16 RVQ codebooks of each 12.5 Hz
+frame, and MOSS-Audio-Tokenizer-Nano (22M, causal transformer codec) turns frames
+into 48 kHz stereo audio. The conversion lives in
+[mobius `models/tts/moss-tts-nano/coreml`](https://github.com/FluidInference/mobius/tree/main/models/tts/moss-tts-nano/coreml);
+weights are published at
+[FluidInference/moss-tts-nano-coreml](https://huggingface.co/FluidInference/moss-tts-nano-coreml).
+
+```
+reference clip ─CodecEncoder (fp32)─► 16×T codes ─┐
+text ─SentencePiece BPE─► ids ────────────────────┴─► rows [T,17] ─Prefill─► hidden + KV
+                                                                 │  per 80 ms frame
+                Frame (local transformer + sampler) ─► 16 codes ─► CodecStep ─► 3840 stereo samples
+                Step (global GPT-2, KV update)      ◄── [assistant_slot, 16 codes]
+```
+
+Sampling (top-k / top-p / temperature / repetition penalty) is inside the `Frame`
+graph; the host only supplies uniform randoms, so a `seed` makes a run
+reproducible on the same machine.
+
+## Quick Start
+
+### CLI
+
+```bash
+# Preset voice (en_2), stereo 48 kHz WAV
+swift run fluidaudiocli tts "Hello from the neural engine." \
+    --backend moss-tts-nano --output hello.wav
+
+# Clone a voice from a clip (≤ ~25 s), save the codes, reuse them later
+swift run fluidaudiocli tts "Same voice, new words." --backend moss \
+    --clone-voice speaker.wav --save-voice speaker.json --output clone.wav
+swift run fluidaudiocli tts "Again." --backend moss --voice-file speaker.json
+
+# Deterministic sampling; tokenizer parity check
+swift run fluidaudiocli tts "Hello" --backend moss --seed 7 --metrics moss.json
+swift run fluidaudiocli tts "Hello" --backend moss --tokens-only
+```
+
+| Flag | Default | Notes |
+|---|---|---|
+| `--voice <name>` | `en_2` | Preset: `en_2` (English), `zh_1` (Mandarin) |
+| `--clone-voice <clip>` | – | Any AVFoundation-readable file; resampled to 48 kHz stereo |
+| `--save-voice` / `--voice-file <json>` | – | Persist / reuse cloned voice codes |
+| `--seed N` | system RNG | Seed for the in-graph sampler |
+| `--greedy` | off | Argmax decoding (parity oracle; upstream greedy never stops) |
+| `--cpu-only` | off | Everything on CPU |
+
+### Swift
+
+```swift
+import FluidAudio
+
+let manager = try await MossTtsNanoManager.downloadAndCreate()
+let voice = try await manager.loadVoice(.en2)              // or manager.cloneVoice(audioURL:)
+
+// Batch
+let audio = try await manager.synthesize(text: "Local speech synthesis.", voice: voice)
+play(left: audio.left, right: audio.right, sampleRate: audio.sampleRate)   // 48 kHz
+// audio.mono / audio.interleaved for single-channel or interleaved sinks
+
+// Streaming — 80 ms stereo frames as they are decoded
+var options = MossTtsNanoSamplingOptions()
+options.seed = 42
+for try await frame in try await manager.synthesizeStreaming(text: text, voice: voice, options: options) {
+    schedule(left: frame.left, right: frame.right)         // frame.isPause marks inter-chunk silence
+}
+```
+
+`MossTtsNanoSamplingOptions` exposes the upstream defaults (`textTemperature` 1.5,
+`audioTemperature` 1.7, `audioTopP` 0.8, `repetitionPenalty` 1.0, `maxNewFrames`
+375 = 30 s per chunk, `maxTextTokens` 50).
+
+## Text handling
+
+- Tokenizer: SentencePiece **BPE** (`MossTtsNanoTokenizer`) with `nmt_nfkc`
+  normalization, dummy prefix and byte fallback — verified id-for-id against the
+  upstream `sentencepiece` processor. Upstream additionally runs WeTextProcessing
+  (number/date verbalization); that step is not ported, so spell out numbers or
+  run `NemoTextNormalizer` first for best results.
+- Chunking: upstream's sentence → clause → token-budget splitter
+  (`MossTtsNanoTextChunker`), 50 tokens per chunk, first letter capitalized and a
+  terminal period added when missing; chunks are joined with 0.40 s (≤ 4 words) or
+  0.24 s pauses.
+- Prompt: `<im_start>user … Reference: <audio_start>[reference rows]<audio_end> …
+  Text: {ids} </user_inst><im_end><im_start>assistant <audio_start>`; the template
+  ids and special tokens come from the repo's `config.json`.
+
+## Voices
+
+A voice is the codec token sequence of a reference clip (`MossTtsNanoVoice`,
+`[frames][16]`). Two presets are published; `cloneVoice(audioURL:)` encodes any
+clip with the fp32 codec encoder (fp16 loses a third of the codes through the
+residual quantizer). The prefill graph holds 512 rows, so reference frames + text
+tokens + 78 template rows must fit: keep clips under ~25 s.
+
+## Performance (M5 Pro, macOS 26.7, warm)
+
+| Stage | ms / call | Unit |
+|---|---|---|
+| Prefill (512 rows) | 11 | GPU |
+| Step (KV M=1024) | 7.6 | GPU |
+| Frame | 5.3 | any |
+| CodecStep | 5.2 | GPU / ANE |
+| CodecEncoder fp32 (8 s clip) | 22 | GPU |
+
+≈ 18 ms of model compute per 80 ms frame. End to end through the Swift host
+(release CLI, `en_2` voice, 9–10 s utterances): **3.1–3.8× real time**, first audio
+0.22–0.40 s after the call (includes prefill and the first frame); a 3 s Mandarin
+utterance with `zh_1` runs at 2.5×. Prefill and Step fail ANE compilation (`ANECCompile FAILED`) and are
+pinned to CPU+GPU by `MossTtsNanoModelStore`; `computeUnits` applies to Frame and
+CodecStep. The Step graph round-trips a 38 MB KV cache per frame; a `StateType`
+(iOS 18) variant is a planned follow-up.
+
+## Parity
+
+- Wrappers vs upstream fp32: hidden states 7e-6, codec 234 dB SNR, encoder codes exact.
+- CoreML fp16 greedy replay of a 375-frame upstream reference: 370/375 frames token-exact.
+- Streaming codec step vs full decode: 56.6 dB SNR (GPU).
+- Parakeet ASR, two English phrases with the `en_2` voice: 8.3 % WER (CoreML) vs
+  10.1 % (upstream PyTorch fp32).
+
+## Known limitations
+
+- No text normalization (numbers, dates) — see above.
+- `nq < 16` low-bitrate decoding and the batch `CodecDecoder` are not exposed in Swift.
+- Sentence chunks are independent generations (upstream behaviour); prosody is not
+  carried across chunk boundaries.

Sources/FluidAudio/ModelNames.swift

@@ -77,6 +77,12 @@ public enum Repo: String, CaseIterable, Sendable {
     /// recipe. Ships four `.mlmodelc` bundles + `tts.json` +
     /// `unicode_indexer.json` at the repo root.
     case supertonic3 = "FluidInference/supertonic-3-coreml"
+    /// MOSS-TTS-Nano (OpenMOSS) — 0.1B multilingual streaming TTS with zero-shot
+    /// voice cloning, 48 kHz stereo. Four streaming-path `.mlmodelc` bundles
+    /// (Prefill / Step / Frame / CodecStep) + `config.json` + `tokenizer.model` at
+    /// the repo root, fp32 `CodecEncoder` for voice cloning, `voices/*.json` presets.
+    /// Conversion: mobius `models/tts/moss-tts-nano/coreml`.
+    case mossTtsNano = "FluidInference/moss-tts-nano-coreml"
     /// NeuTTS-2E emotional English TTS (Qwen3 236M backbone + NeuCodec
     /// decoder). Compiled `.mlmodelc` bundles + `tokenizer.json` +
     /// `samples/<speaker>.json` reference codes at the repo root; the
@@ -178,6 +184,8 @@ public enum Repo: String, CaseIterable, Sendable {
             return "inflect-v2-coreml/micro"
         case .inflectNano:
             return "inflect-v2-coreml/nano"
+        case .mossTtsNano:
+            return "moss-tts-nano-coreml"
         }
     }
 
@@ -1204,6 +1212,34 @@ public enum ModelNames {
     /// Supertonic-3 multilingual TTS — 4 `.mlmodelc` bundles + 2 companion
     /// JSON files. File names match the HuggingFace tree at
     /// `FluidInference/supertonic-3-coreml/`.
+    public enum MossTtsNano {
+        public static let prefill = "MossNano-Prefill-T512-M1024-fp16"
+        public static let step = "MossNano-Step-M1024-fp16"
+        public static let frame = "MossNano-Frame-fp16"
+        public static let codecStep = "MossNano-CodecStep-fp16"
+        public static let codecDecoder = "MossNano-CodecDecoder-fp16"
+        public static let codecEncoder = "MossNano-CodecEncoder-fp32"
+
+        public static let prefillFile = prefill + ".mlmodelc"
+        public static let stepFile = step + ".mlmodelc"
+        public static let frameFile = frame + ".mlmodelc"
+        public static let codecStepFile = codecStep + ".mlmodelc"
+        /// Flexible-length batch decoder (≤ 125 frames); not used by the Swift streaming path.
+        public static let codecDecoderFile = codecDecoder + ".mlmodelc"
+        /// fp32 prompt encoder for custom voice cloning; downloaded on demand.
+        public static let codecEncoderFile = codecEncoder + ".mlmodelc"
+
+        public static let configFile = "config.json"
+        public static let tokenizerFile = "tokenizer.model"
+        public static let voicesSubdir = "voices"
+
+        /// Streaming-path bundles loaded by `MossTtsNanoModelStore`.
+        public static let requiredModels: Set<String> = [prefillFile, stepFile, frameFile, codecStepFile]
+
+        /// Models + companion files fetched on first use.
+        public static let requiredFiles: Set<String> = requiredModels.union([configFile, tokenizerFile])
+    }
+
     public enum Supertonic3 {
         public static let textEncoder = "TextEncoder"
         public static let durationPredictor = "DurationPredictor"
@@ -1630,6 +1666,8 @@ public enum ModelNames {
             return ModelNames.LuxTts.requiredFiles(variant: variant)
         case .inflectMicro, .inflectNano:
             return ModelNames.Inflect.requiredModels
+        case .mossTtsNano:
+            return ModelNames.MossTtsNano.requiredFiles
         }
     }
 }

Sources/FluidAudio/Shared/Download/ModelHub.swift

@@ -770,7 +770,9 @@ public enum ModelHub {
                     itemPath.hasSuffix(".json") || itemPath.hasSuffix(".model") || itemPath.hasSuffix(".bin")
                 return isInSubPath && (matchesPattern || isMetadata)
             }
-            return patterns.isEmpty || patterns.contains { itemPath.hasPrefix($0) }
+            // Patterns carry a trailing "/" (bundle directories); a required root-level
+            // *file* (e.g. `tokenizer.model`) matches when the pattern is exactly its path.
+            return patterns.isEmpty || patterns.contains { itemPath.hasPrefix($0) || $0 == itemPath + "/" }
                 || itemPath.hasSuffix(".json") || itemPath.hasSuffix(".txt")
         }
     }

Sources/FluidAudio/TTS/MossTtsNano/Assets/MossTtsNanoAudioLoader.swift

@@ -0,0 +1,63 @@
+@preconcurrency import AVFoundation
+import Foundation
+import os
+
+/// Decodes any AVFoundation-readable file to 48 kHz stereo Float32 (mono is duplicated).
+enum MossTtsNanoAudioLoader {
+
+    static func loadStereo48k(url: URL) throws -> (left: [Float], right: [Float]) {
+        do {
+            let file = try AVAudioFile(forReading: url)
+            guard
+                let target = AVAudioFormat(
+                    commonFormat: .pcmFormatFloat32, sampleRate: Double(MossTtsNanoConstants.sampleRate),
+                    channels: 2, interleaved: false)
+            else { throw MossTtsNanoError.audioLoadFailed(path: url.path, underlying: "cannot build 48 kHz format") }
+            let frameCount = AVAudioFrameCount(file.length)
+            guard let input = AVAudioPCMBuffer(pcmFormat: file.processingFormat, frameCapacity: max(frameCount, 1))
+            else { throw MossTtsNanoError.audioLoadFailed(path: url.path, underlying: "cannot allocate buffer") }
+            try file.read(into: input)
+
+            guard let converter = AVAudioConverter(from: file.processingFormat, to: target) else {
+                throw MossTtsNanoError.audioLoadFailed(
+                    path: url.path, underlying: "no converter for \(file.processingFormat)")
+            }
+            let ratio = target.sampleRate / file.processingFormat.sampleRate
+            let capacity = AVAudioFrameCount(Double(input.frameLength) * ratio) + 4096
+            guard let output = AVAudioPCMBuffer(pcmFormat: target, frameCapacity: capacity) else {
+                throw MossTtsNanoError.audioLoadFailed(path: url.path, underlying: "cannot allocate output buffer")
+            }
+            let provided = OSAllocatedUnfairLock(initialState: false)
+            let inputBlock: AVAudioConverterInputBlock = { _, outStatus in
+                let wasProvided = provided.withLock { state -> Bool in
+                    if state { return true }
+                    state = true
+                    return false
+                }
+                if wasProvided {
+                    outStatus.pointee = .endOfStream
+                    return nil
+                }
+                outStatus.pointee = .haveData
+                return input
+            }
+            var conversionError: NSError?
+            let status = converter.convert(to: output, error: &conversionError, withInputFrom: inputBlock)
+            if status == .error || conversionError != nil {
+                throw MossTtsNanoError.audioLoadFailed(
+                    path: url.path, underlying: conversionError?.localizedDescription ?? "conversion failed")
+            }
+            let n = Int(output.frameLength)
+            guard let channels = output.floatChannelData, n > 0 else {
+                throw MossTtsNanoError.audioLoadFailed(path: url.path, underlying: "empty audio")
+            }
+            let left = Array(UnsafeBufferPointer(start: channels[0], count: n))
+            let right = Array(UnsafeBufferPointer(start: channels[1], count: n))
+            return (left, right)
+        } catch let error as MossTtsNanoError {
+            throw error
+        } catch {
+            throw MossTtsNanoError.audioLoadFailed(path: url.path, underlying: "\(error)")
+        }
+    }
+}