End capture when the speaker stops, and think until speech actually starts#216
Conversation
…tarts Two things reported from live use: replies were cut off at 4.8 s, and the face did not look like it was thinking while a reply was being prepared. Capture length: - The voice-activity endpoint's maximumCaptureMs was 4800, so any sentence past roughly four seconds was truncated mid-word regardless of whether the speaker had finished. It is now 12000, which is a backstop rather than the normal path: capture still ends on 550 ms of trailing silence as soon as the speaker stops. 12 s of 16 kHz mono PCM is 384 KB, inside the 512 KB uplink limit. - The dedicated capture chunk ceiling moves from 96 (9.6 s) to 130 (13 s) so it sits behind the endpoint rather than in front of it. - The endpoint now runs for the initial wake-gated utterance too, not only for conversation replies. Previously the first capture after a wake phrase always ran its full fixed length, so a two-second question still cost the whole window before anything reached the bridge. Measured, capture length against how long the speaker actually talks: spoke 1.0s before 1.6s after 1.6s spoke 4.0s before 4.6s after 4.6s spoke 6.0s before 4.8s after 6.6s spoke 8.0s before 4.8s after 8.6s spoke 11.0s before 4.8s after 11.6s Before, anything longer than about four seconds ended with reason max_duration and lost the remainder. After, every case ends with trailing_silence and a constant 0.6 s tail. Thinking face: - response_start only means the reply text is ready; TTS still has to render. That gap was spent in Speak, with a speaking mouth and no sound. He now holds the thinking face through it and moves to Speak on the first audio of the response, whether that arrives as an audio stream or a mouth frame. - Adds IntentEngine::setMode for state changes that are a continuation of something already accounted for, so the emotional event is not applied twice. It refreshes the event clock so the mode-decay timers do not unwind it, and it deliberately cannot wake a sleeping robot; waking stays event-driven. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Post-merge hardware/bridge handoff from PR #199: the longer endpoint is present and active (12,000 ms maximum, 550 ms trailing silence, 13-second dedicated capture ceiling). A boundary race remains because BridgeWakeGate also expires at 12,000 ms. At the exact ceiling it can close audio uplink before the final capture chunk submit. Two live captures reached the host with their exact declared totals (118 + 113 = 231 chunks; 369,600 bytes), but robot counters increased by 80 bridge_uplink_errors with bridge_uplink_last_error=audio_uplink_not_active and by 2 mww_uplink_submit_failed; queue failures stayed zero. This points to local firmware gate ordering, not Cloudflare/host transport loss. PR #199 keeps firmware byte-for-byte aligned with main and will not patch it; zero new uplink errors remains a physical-promotion gate for the firmware owner. |
|
Qualification provenance follow-up: PR #199 now refuses the historical pre-216 ce66f8a / 69d3... acceptance pair. Before the supervised bridge session, firmware ownership needs to record the exact private paired firmware.bin SHA-256 together with source commit 6d39af7 (or a later main descendant) in docs/FIRST_DEPLOY_STATUS.md, after closing the 12-second gate race. The live robot currently reports the old configured ota_expected_sha256 even though the longer capture behavior is present, so that field alone is not sufficient source-to-binary provenance. |
Two things reported from live use: replies cut off at 4.8 s, and no thinking face while a reply is being prepared.
Capture length
Three things stacked up:
VoiceActivityEndpointConfig::maximumCaptureMswas 4800, so any sentence past ~4 s was truncated mid-word regardless of whether the speaker had finished. Now 12000 — a backstop, not the normal path. Capture still ends on 550 ms of trailing silence as soon as you stop.kWakeMwwDedicatedCaptureChunkswas 96 (9.6 s). Now 130 (13 s), so the chunk ceiling sits behind the endpoint rather than in front of it.Measured, capture length vs how long the speaker actually talks:
Before, anything past ~4 s ended with
reason=max_durationand lost the remainder — 11 s of speech lost 6.2 s. After, every case ends withtrailing_silenceand a constant 0.6 s tail.Byte budget: 12 s of 16 kHz mono PCM is 384 KB, inside the 512 KB limit on both firmware (
kBridgeAudioUplinkMaxBytes) and bridge (DEFAULT_MAX_AUDIO_BYTES).Thinking face
response_startonly means the reply text is ready — TTS still has to render. That gap was spent inSpeak, with a speaking mouth and no sound.He now holds the thinking face through it and moves to
Speakon the first audio of the response, whether that arrives as anaudio_stream_startor a mouthaudioframe.Thinkalready reads clearly: eyes up and to the side (FaceAnimatorThink saccade targets), amber body light with a chase pulse.Adds
IntentEngine::setModefor state changes that continue something already accounted for, so the emotional event isn't applied twice. It refreshes the event clock so the mode-decay timers don't unwind it, and it deliberately cannot wake a sleeping robot — waking stays event-driven.The awaiting-first-speech flag is cleared on
response_end,error, and the downlink abort path, so it can't leak into the next turn. If audio never arrives at all, the 90 s conversation-stall recovery from #214 still applies.Verification
setModemoves without an event and is ignored while asleepExisting VAD tests all set explicit configs, so the default change doesn't affect them.
Note
The
main.cppchanges (endpoint gating, thinking-face deferral) are only exercised on target — they sit in the bridge output handler, which isn't in the native test build. The endpoint timing itself is covered natively.🤖 Generated with Claude Code