Stream live audio input from stdin

[patrickvonplaten]

Adds --audio - so the streaming CLI can transcribe audio as it is produced, instead of only replaying something already in memory. Carved out of #112.

 ffmpeg -f avfoundation -i ":0" -ar 16000 -ac 1 -f s16le - 2>/dev/null \
  | ./build/macos-metal-release/bin/audiocpp_cli \
      --task asr --family voxtral_realtime \
      --model /Users/patrickprivate/models/Voxtral-Mini-4B-Realtime-2602-GGUF/voxtral-mini-4b-realtime-2602-q8_0.gguf \
      --backend metal --mode streaming --audio -

Why

run_streaming_task required a fully materialized request.audio_input and sliced it into chunks. On main, process_audio_chunk has exactly one call site — inside feed_audio_chunks — and it cannot be reached without the complete audio in hand, so no C++ path accepted audio incrementally.

That also applies to the two existing streaming paths, which is worth stating since both look like they already cover this:

• tools/streaming/measure_asr_streaming_client.py paces a multipart upload across several socket writes, but computes Content-Length from a complete file and, as its own manifest records, "current server reads full HTTP body before invoking ASR streaming handler". It measures response latency; a microphone cannot produce a Content-Length.
• webui/realtime_pipeline.py does transcribe a live mic, but via Python silero-VAD segmentation with each completed utterance sent as an ordinary /v1/audio/transcriptions request. Nothing surfaces until the speaker pauses (min_silence_ms=450), and _run_turn runs STT → LLM → TTS with no early exit — it is a speech-to-speech assistant, not a transcriber.

Approach

AudioChunkStream — a format declared up front plus a pull-based reader:

using AudioChunkReader = std::function<bool(int64_t max_samples, std::vector<float> & samples)>;

Both input routes go through it, and the file path is expressed as a reader over the buffer (buffer_audio_stream), so live and buffered input run the same code and cannot drift apart — that is what makes the byte-identical result below a property of the design rather than a coincidence. The three-argument run_streaming_task stays as a wrapper, so app/server/runtime.cpp is untouched.

--input-format (s16le/f32le), --input-rate and --input-channels supply the format a headerless stream cannot carry. They double as the audio contract prepare() requires — VoxtralRealtimeSession::prepare throws without one — carried as an audio_input with rate and channels but no samples. Decoding is explicitly little-endian and reuses the WAV reader's /32768 scaling, so raw PCM and WAV of the same audio decode identically. partial_text= lines are flushed as produced, otherwise a piped consumer sees nothing until the stdio buffer fills.

app/cli/main.cpp changes by +50/−6, with no preprocessor work; the in-place terminal rendering that made the original diff larger stays in #112, as it is orthogonal and improves file streaming on main today.

Verification

• File input, s16le stdin and f32le stdin all transcribe assets/resources/sample_16k.wav to byte-identical stdout apart from the audio_input=stdin banner — 34 partials each, growing one token at a time.
• --audio - without --mode streaming fails with --audio - reads live PCM and requires --mode streaming; --input-format s24le fails with unsupported raw PCM sample format 's24le' (expected s16le or f32le).
• streaming_audio_input_test (new) covers stdin/buffer equivalence, both sample formats, chunk sizing and drain, a trailing partial frame, the empty stream, format parsing, and the two cases from the second commit.
• ctest 29/32, matching main. gguf_tensor_source_test and scaled_dot_product_attention_test fail on main too; asr_standalone_gguf_test does not compile there (test_asr_standalone_gguf.cpp calls load_hviske_assets, the header declares load_hviske_asr_assets), as do the three tests/moss_tts_local/codec_*_parity.cpp targets. audio_dsp_test and supertonic_vector_convnext_exp_test are flaky rather than broken — test_audio_dsp.cpp:38 seeds its RNG from the wall clock. None of these are touched by this branch. CI is green on all four platforms.

[patrickvonplaten]

Curious what thoughts you have on this. If such changes to main.cpp are too intrusive it could also make sense to "only" allow true live realtime transcription from a script for now?

[0xShug0]

It's a good extension. The current streaming in the cli is naive and is mostly for testing the correctness of streaming path implementations. In streaming tests I only used the server and the python streaming client.

[patrickvonplaten]

Ok great, the PR is ready for review then I think