Add grounded conversation, memory, research, and host awareness

[RobVanProd]

Summary

• add bridge-only one-wake, back-and-forth Conversation v2 with deterministic reply windows, cancellation, response-wire closure, and a constant 10-second follow-up lease
• add persistent local STT, in-process Ollama, and in-process DirectML RVC for the warm low-latency path
• add bounded local-first web research through loopback-only SearXNG and a restricted public-HTTPS fetcher
• add truthful Memory v4 callbacks, restrained Spark character, emotional speech metadata, justified initiative, privacy-filtered room awareness, and local face detection
• add a responsive loopback bridge dashboard with firmware-authoritative motion and awareness controls
• route visual questions through a fresh privacy-filtered camera observation and report the grayscale limitation instead of guessing colours
• make production host startup fail closed until firmware verifies motion, servo rail, and torque off
• defer/cancel background room observation while a foreground conversation turn is active
• require exact PR Keep the wake gate open while the microphone is still hearing speech #217-or-later firmware provenance and complete physical behavior evidence before promotion
• keep the accepted main firmware unchanged

Scope and authority

This branch contains current main, including merged PR #217, and is 55 commits ahead and 0 behind. The aggregate branch diff is empty for src/, platformio.ini, partitions.csv, partitions_esp_sr_16.csv, boards/, personas/, and the other guarded firmware inputs. This bridge PR does not flash or replace robot firmware.

The model and research broker have no actuator, power, pairing, camera-authentication, filesystem, shell, or OTA authority. Web text is untrusted context and cannot write durable memory. Raw camera frames remain in memory only; prompts receive an allowlisted typed room summary.

Exact candidate

• source commit: 60edca4642d5950969e66e2e7f4fcb45b74533c8
• branch: agent/natural-research-voice
• package: stackchan_alive_v0.2.0-139-g60edca46.zip
• package bytes: 189434808
• package SHA-256: A782B81E9868089A1C269EC9339B6E81FE50793F4810B02D8BB376F85635D1CB

The public ZIP is credential-clean and intentionally excludes private paired firmware, pairing material, and private runtime evidence.

Validation

• worktree clean; exact source commit pushed; 0 commits behind origin/main
• guarded firmware-input diff against origin/main: empty
• bridge reference suite: 477/477 pass
• firmware native logic suite: 289/289 pass
• Python compileall, diff check, and supervised-qualification PowerShell contract: pass
• full public release packaging/build gate: pass using a single PlatformIO build job
• exact ZIP independently reverified against full commit 60edca4642d5950969e66e2e7f4fcb45b74533c8
• package manifest reports dirty=false and the exact source commit
• all 11 exact-head GitHub Actions checks pass, including native tests, embedded build, bridge tests, Android emulator smoke, release evidence, and Android/Linux/macOS/Windows artifacts

Supervised evidence gate

The qualification can no longer report ready using the historical pre-PR217 firmware record. It requires an accepted private firmware SHA/source pair from merged PR #217 (10b0cc5404e072bb5784d9cfd2fabb0babd8a02e) or a later main commit, recorded together in docs/FIRST_DEPLOY_STATUS.md. This pins PR #216's 12-second endpoint and PR #217's wake-gate renewal plus strict 12 s endpoint < 13 s capture < 15 s privacy guard ordering.

The checker also requires:

• every required robot transport counter to exist
• zero new uplink, MWW submit/drop, capture, writer-drop, reply-window, playback, audio-stop, raw-speaker, or forced-stop events
• redacted route metadata proving cited research, a fresh visual observation, the deterministic grayscale guard, and deterministic memory recall
• operator confirmation of natural conversation, accurate recall without unrelated memory hijack, grounded research/vision/person noticing, complete audio, initiative behavior, privacy, and recovery

The gate remains passive: it does not flash, reboot, restart, stop processes, or control motion.

Reset-safe motion policy

A live host restart exposed that accepted firmware can return from its own boot with motion, servo rail, and torque enabled. The bridge-only production launcher sends the existing firmware-owned motion-stop command through the loopback dashboard immediately after reconnect and refuses to report ready unless both dashboard verification and a fresh /debug read confirm all three actuator states off. Resume remains a separate operator-confirmed dashboard action.

Physical restart proof is intentionally pending current operator presence and body-clear confirmation; the robot was not restarted merely to obtain evidence.

Research and visual-context evidence

Natural check, verify, fact-check, freshness-sensitive, and explicit-search questions enter one bounded research round without wasting an initial model pass. If Gemma claims it cannot access the web, the bridge recovers through the same policy-limited search path. Verification may fetch one selected public HTTPS result. Fetched text remains untrusted, citations stay bounded companion metadata, and research cannot write memory.

A live local probe answered that Python 3.13.0 was released on October 7, 2024, cited Python.org, and completed in 3.9 s. The pinned SearXNG service is loopback-only and passes JSON search, engine allowlist, broker search, restricted fetch, and privacy-safe audit checks.

A visual question performs one fresh authenticated grayscale observation before generation. A live robot-camera probe completed in 2.6 s, recorded one successful observation with zero failures, and produced a grounded door, shelf, and bright-lighting answer. Deictic colour questions take a deterministic local path: the camera feed is grayscale, so Stackchan says it cannot determine the colour instead of inventing one.

The owner physically exercised the research and visual paths and reported that the interaction worked well. That is useful informal evidence, not a substitute for the uninterrupted exact-head qualification.

Conversation quality and latency

A predecessor regression forcibly replaced valid Gemma answers with the newest stored episode, causing repeated bridge or sleep topics. This branch injects episodes only when query terms match or the user explicitly asks to recall prior context. Due callbacks remain one-shot.

Ordinary turns now use a compact typed prompt while retaining trusted embodiment, relevant memory, bounded conversation history, research routing, actuator rules, and Character Lock. Explicit memory write/forget requests retain the full memory contract. Eight warm local generations completed with a 1126.33 ms median, 1872.31 ms maximum, and zero fallbacks. Background room observation defers or cancels during foreground conversation instead of competing with the reply path.

Three-plus physical warm turns under the end-to-end three-second first-audio limit are still required before promotion.

PR #217 firmware baseline and handoff

Merged PR #216 raises the voice-activity endpoint maximum from 4800 ms to 12000 ms, gives the dedicated capture path a 13-second chunk ceiling, applies endpointing to both initial and follow-up captures, and ends speech after 550 ms of trailing silence. Merged PR #217 renews the existing wake gate only while device VAD observes real speech, places its hard privacy limit at 15 seconds, and adds compile-time plus native long-utterance boundary coverage. The bridge branch contains both baselines unchanged.

Before PR #217, two long captures delivered all declared audio (118 + 113 = 231 chunks, 369600 bytes) but recorded 80 audio_uplink_not_active errors and two mww_uplink_submit_failed events. PR #217 closes that source mechanism. Formal qualification still requires an exact PR #217-or-later image and zero new uplink or submit-failure deltas; source tests are not substituted for physical evidence.

Current live state

The host bridge and dashboard are currently stopped. The robot remains reachable in local/offline mode. The firmware handoff initially reported motion, servo rail, and torque enabled while the bridge was offline, so this bridge task issued the existing safety /motion-stop command and verified all three off; audio is also off. No flash, reboot, motion-enable, or firmware edit was performed.

Remaining merge gates

This PR remains draft. Do not merge until one uninterrupted exact-head session produces bridge-ai-supervised-ready and the owner explicitly approves promotion.

Still required:

• an accepted PR217-or-later private firmware binary SHA and source commit recorded together on main
• zero new uplink errors on the exact PR Keep the wake gate open while the microphone is still hearing speech #217-or-later image
• current operator/body-clear confirmation and exact-head startup proof with motion, rail, and torque off
• at least three physical warm local turns with first audio under three seconds
• cited research, fresh visual grounding, truthful grayscale limitation, memory recall/isolation, and person-noticing evidence
• natural multi-turn, exit, silence-close, physical barge-in, and complete continuous audio
• at least 100 reply windows with zero accepted echo
• two natural initiative openers at least ten minutes apart, ignored-opener backoff, and night suppression
• room observation grounding, disable/clear, and no-frame-persistence evidence
• bridge-loss local recovery
• exact firmware SHA before and after the uninterrupted session
• zero new required error deltas
• explicit owner approval

No merge is part of this update.

[RobVanProd]

Bridge transport follow-up after syncing current main:

• Root cause: bridge_uplink_errors=80 corresponds exactly to two failed dedicated-capture chunks retried 40 times each (mww_uplink_submit_failed=2), with zero binary queue failures.
• Fixed cross-type queue ordering so a final PCM frame cannot be overtaken by a later utterance_end text frame.
• Remote response_start/response_end events no longer close an active local microphone turn; local SpeechEnded, errors, and bounded gate timeouts retain ownership of capture completion.
• response_end now retains its sequence for diagnostics.

Verification:

• native logic: 272/272 pass
• bridge Python suite: 337/337 pass
• stackchan_release_full: build pass
• current live robot: exact downlink accounting, 0 playback errors, 0 playRaw failures; host log has not observed PCM after utterance_end

Physical gate remains intentionally pending: flash an exact private candidate only with the operator present, keep motion stopped, capture a pre-test /debug baseline, run at least ten wake/speech turns including one interruption near response completion, and require zero deltas in uplink errors/failed chunks plus clean heard audio and exact downlink/playRaw accounting. The existing live process and robot were not restarted or flashed for this software-only update.

[RobVanProd]

Final post-push verification for e9b7cba4:

• GitHub Actions bridge-tests: passed (1m33s)
• GitHub Actions native-tests: passed (41s)
• GitHub Actions Windows firmware build: passed (11m54s)
• path-filtered companion jobs: skipped as expected
• branch is 0 behind origin/main and 0 ahead/behind its remote head after push
• final local release ZIP: 189,300,476 bytes, SHA-256 7e41f8b64e4bff4ee4ab12703e468a0893bc90312df15e9894a2c9082e6f046a
• strengthened release verifier passed and archive contains zero Python cache/bytecode entries

The PR remains draft. No live bridge restart, robot flash, motion command, or merge was performed. The remaining gates are the exact-image supervised physical qualification and normal soak documented in docs/BRIDGE_AI_QUALIFICATION.md.

[RobVanProd]

Exact candidate update for 5767ec958e1ba528fe3e477d7df4eeec4add3dc7:

• branch clean, pushed, 0 behind origin/main; guarded firmware-input diff remains empty
• bridge suite 448/448 pass; Character Lock red team 26/26 pass; credential hygiene 11/11 pass
• all 11 GitHub Actions checks pass, including bridge/native tests, firmware build, Android emulator smoke, companion tests/release evidence, and Android/Linux/macOS/Windows artifacts
• exact package stackchan_alive_v0.2.0-124-g5767ec95.zip, 189400522 bytes, SHA-256 BF42EAB100A7D8E9EB9D09E1A111F5FB602623D4A8A80582F8C809747DE57C4A, independently reverified
• exact clean commit is live; robot/bridge ready; motion, servo rail, and torque off; uplink errors remain at the established baseline of 80 and playback errors remain 0
• live follow-up VAD now records successful speech and no-speech evidence together; a quiet 93-chunk capture was classified no-speech and closed silently without STT/model/TTS
• host-only face/room stack is enabled; authenticated frame/target/face counters advance with zero new frame/auth failures
• privacy control exercised: room observation produced grounded person-present and empty-room summaries with zero observation failures, then disable cleared state immediately and no image file was persisted; observation was re-enabled at the 120-second cadence

The PR remains draft. The exact supervised evidence session still requires the operator-present attestation, multi-turn/exit/barge-in checks, 100 echo-free reply windows, initiative timing/backoff/night checks, bridge-loss recovery, exact pre/post firmware SHA, and explicit owner approval. No firmware operation or merge was performed.

[RobVanProd]

PR #216 baseline is now included unchanged. Exact bridge candidate: 04064bf; package stackchan_alive_v0.2.0-135-g04064bf7.zip; SHA-256 A6ED79002713C288CAF70A5D812D8C60F8166A88B40D174DD36A5A2BB51E632B. Local validation: bridge 471/471, native 287/287, package verifier pass; warm local generation median 1126.33 ms, max 1872.31 ms. Firmware guarded diff against main remains empty. PR #216's 12-second capture improves long utterances, but live evidence found a same-threshold BridgeWakeGate/final-chunk race producing audio_uplink_not_active at the ceiling; this is documented as a firmware-owner handoff and must be closed before supervised promotion. PR remains draft and unmerged.

[RobVanProd]

Final pre-physical candidate update: head 5d6c6c9; package stackchan_alive_v0.2.0-136-g5d6c6c91.zip; 189,432,449 bytes; SHA-256 A3DDDD3A4FA4D2493FD2F747BAB8BA2D6056B8D9D94289FE559889F6F0741B30. Local bridge suite 477/477; independent package verifier pass; all 11 exact-head CI checks pass; guarded firmware diff remains empty. The supervised checker now requires PR #216-or-later accepted firmware provenance, complete required transport telemetry with zero deltas, redacted route proof for research/visual/grayscale/memory, and operator-confirmed natural conversation, recall isolation, grounded person noticing, initiative, privacy, and recovery. The live bridge remains stopped and no firmware, reboot, or motion action was performed.