diff --git a/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/_index.md b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/_index.md new file mode 100644 index 0000000000..49b1635d5e --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/_index.md @@ -0,0 +1,64 @@ +--- +title: Deploy Silero VAD on Alif Ensemble E8 with ExecuTorch +description: Export a stateful Silero voice activity detector, build its Zephyr firmware for the Alif E8, and compare recorded-audio probabilities with a host reference. +minutes_to_complete: 120 +draft: true +cascade: + draft: true +who_is_this_for: This Learning Path is for embedded ML developers porting streaming audio inference to the Cortex-M55 HP core and Ethos-U85-256 on an Alif Ensemble E8. +learning_objectives: + - Explain the waveform context and recurrent state retained between Silero VAD invocations + - Export a quantized Ethos-U85 program and matching speech-probability reference using released Python packages + - Build E8 Zephyr firmware with the model, recorded audio, and required CPU operators + - Program the E8 and compare all 79 frame probabilities with the host reference +prerequisites: + - An [Alif Ensemble E8 Development Kit](https://alifsemi.com/support/kits/ensemble-e8devkit/) and a data-capable USB cable + - An Apple silicon Mac with Python 3.12, Git, CMake, Ninja, ripgrep, and a C++ build toolchain + - Access to the E8 application UART4 console through PRG USB and the board’s SW4 selector; no separate USB-to-UART adapter is required + - Zephyr SDK 0.17.0, Alif Security Toolkit for your silicon revision, and SEGGER J-Link + - Experience with PyTorch model export, C++, and Zephyr builds +author: Usamah Zaheer +generate_summary_faq: false +rerun_summary: false +rerun_faqs: false +skilllevels: Advanced +subjects: ML +armips: + - Ethos-U +tools_software_languages: + - ExecuTorch + - PyTorch + - Zephyr + - Python + - CMake +operatingsystems: + - macOS + - RTOS +further_reading: + - resource: + title: Deploy Silero VAD on Arm Ethos-U with ExecuTorch + link: /learning-paths/embedded-and-microcontrollers/silero-vad-ethos-u/ + type: documentation + - resource: + title: Set up the Alif Ensemble E8 board + link: /learning-paths/embedded-and-microcontrollers/alif-image-classification/board-setup/ + type: documentation + - resource: + title: Alif Ensemble E8 DevKit support + link: https://alifsemi.com/support/kits/ensemble-e8devkit/ + type: website + - resource: + title: ExecuTorch Silero VAD export and runtime source + link: https://github.com/pytorch/executorch/tree/12dac4aa042b66a7229e8d7d3af2bdde8c9f262f/examples/arm/silero_vad_example_ethos_u + type: repository + - resource: + title: Silero VAD project + link: https://github.com/snakers4/silero-vad + type: repository + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 +layout: learningpathall +learning_path_main_page: "yes" +--- diff --git a/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/_next-steps.md b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/_next-steps.md new file mode 100644 index 0000000000..727b395ddd --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # The weight controls the order of the pages. _index.md always has weight 1. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/build-the-firmware.md b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/build-the-firmware.md new file mode 100644 index 0000000000..2172099ee8 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/build-the-firmware.md @@ -0,0 +1,85 @@ +--- +title: Build the E8 Zephyr firmware +description: Embed the exported model and recording, link the CPU operators and Ethos-U delegate, and inspect the HP firmware memory use. +weight: 5 +layout: learningpathall +--- + +## Inspect the application configuration + +The application in `zephyr/samples/silero-vad-ethosu` runs once over the embedded recording. It loads `forward` once and preserves that method's recurrent state for every window. Resetting the board starts an independent stream. + +The board-specific files select the HP Ethos-U85-256, enable its clock, and assign application SRAM beginning at `0x020C0000` with size `0x740000` bytes. This is a 7.25 MiB application region for the pinned E8 memory map. It is not portable to another board or a concurrently running application that owns the same SRAM. + +The starting allocation capacities are: + +| Allocation | Capacity or source | +| --- | --- | +| Method and planned buffers | 2 MiB static allocator pool | +| Temporary delegate allocations | 1 MiB static allocator pool | +| PTE copy | Exported PTE size, aligned to 16 bytes in SRAM | +| C/C++ heap | 64 KiB | +| Main stack | 16 KiB | +| Audio | Up to 40,000 normalized float32 samples generated from the PCM16 WAV | + +These are configured capacities, not measurements of model demand. The generated audio array occupies 160,000 bytes for this clip. The PTE and audio begin in the firmware image; startup copies the PTE once to SRAM for NPU access. The linker report checks the static allocation footprint. A completed board run is still needed to verify allocator demand and NPU access. + +## Build for the HP core + +Keep the released Python environment active. Build with absolute paths to the exact PTE and WAV from the previous lesson: + +```bash +cd "$EXECUTORCH_ROOT" +west build \ + -b alif_e8_dk/ae822fa0e5597xx0/rtss_hp \ + zephyr/samples/silero-vad-ethosu \ + -d "$SILERO_BUILD" -- \ + -DZEPHYR_EXTRA_MODULES="$EXECUTORCH_ROOT;$SDK_ALIF" \ + -DPython3_EXECUTABLE="$(command -v python)" \ + -DET_PTE_FILE_PATH="$SILERO_OUTPUT/silero_vad_ethos_u.pte" \ + -DAUDIO_PATH="$SILERO_OUTPUT/validation.wav" +``` + +The CMake application generates the model/audio headers and selects portable CPU operators from the PTE. It links those registrations, the quantized kernels, and the Ethos-U backend into the firmware. The app also rejects inputs that do not match the single float32 `[1,576]` audio tensor and single float32 probability output. + +If you change the board, PTE, WAV, or allocation configuration, rebuild in a new directory or use `west build -p always` with the complete command. This avoids keeping headers or operator selections from an earlier model. + +## Check the firmware and memory placement + +Require the ELF and binary, then inspect the generated target and allocator configuration: + +```bash +test -s "$SILERO_BUILD/zephyr/zephyr.elf" +test -s "$SILERO_BUILD/zephyr/zephyr.bin" +rg 'CONFIG_(RTSS_HP|ARM_ETHOS_U85_256|EXECUTORCH_.*ALLOCATOR_POOL_SIZE)=' \ + "$SILERO_BUILD/zephyr/.config" +rg -n 'app_sram|020c0000|20c0000|ethosu' \ + "$SILERO_BUILD/zephyr/zephyr.dts" +"$ZEPHYR_SDK_INSTALL_DIR/arm-zephyr-eabi/bin/arm-zephyr-eabi-size" \ + "$SILERO_BUILD/zephyr/zephyr.elf" +``` + +Require the HP target and `CONFIG_ARM_ETHOS_U85_256=y`. Keep the memory-usage report printed by the linker and inspect `zephyr.map` if a region is close to its limit. The `.bss` includes allocator capacities and the PTE copy; unused space inside a pool is still reserved in the linked firmware. + +Successful linking proves that the firmware fits the configured static regions. It does not prove that `load_method()` or the largest delegate allocation fits at runtime. The application prints its planned buffer sizes during startup and reports allocation or execution failures; retain those messages during validation. + +The local cross-build with the pinned source, vendor modules, Zephyr SDK 0.17.0, and released Python environment passed. Its linker report is: + +| Region | Used | Available | Utilization | +| --- | --- | --- | --- | +| Flash application partition | 830,084 bytes | 3,584 KiB | 22.62% | +| Application RAM | 3,597,440 bytes | 7,424 KiB | 47.32% | + +The binary is linked at `0x80200000`, including the vendor flash load offset. These are cross-build results, not measurements from a running E8. Physical execution and probability agreement remain to be checked. + +Record the firmware identity alongside the model inputs: + +```bash +shasum -a 256 "$SILERO_BUILD/zephyr/zephyr.elf" \ + "$SILERO_BUILD/zephyr/zephyr.bin" \ + > "$SILERO_OUTPUT/firmware.sha256" +``` + +## What you've accomplished + +You have built one firmware image containing the recorded clip, exported program, CPU kernels, and U85 delegate, and checked its configured memory placement. Next, program this matching image onto the E8 HP core. diff --git a/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/export-the-streaming-model.md b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/export-the-streaming-model.md new file mode 100644 index 0000000000..84f430eccf --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/export-the-streaming-model.md @@ -0,0 +1,99 @@ +--- +title: Export the streaming VAD model +description: Quantize Silero VAD for Ethos-U85 and generate the recorded-audio reference used to check the E8 output. +weight: 4 +layout: learningpathall +--- + +## Download the model and calibration audio + +Use the model and recording from one pinned Silero VAD revision. Keep the upstream project's license with any redistributed model or audio. The application archive contains neither file. + +Download both inputs into your output directory: + +```bash +cd "$EXECUTORCH_ROOT" +curl --fail --location \ + --output "$SILERO_OUTPUT/silero_vad.jit" \ + https://raw.githubusercontent.com/snakers4/silero-vad/dbacf536adadf42210f37ae50fbaf75f6235b3cf/src/silero_vad/data/silero_vad.jit +curl --fail --location \ + --output "$SILERO_OUTPUT/test.wav" \ + https://raw.githubusercontent.com/snakers4/silero-vad/dbacf536adadf42210f37ae50fbaf75f6235b3cf/tests/data/test.wav +shasum -a 256 "$SILERO_OUTPUT/silero_vad.jit" "$SILERO_OUTPUT/test.wav" +``` + +The expected hashes are: + +| File | SHA256 | +| --- | --- | +| `silero_vad.jit` | `e1122837f4154c511485fe0b9c64455f7b929c96fbb8d79fbdb336383ebd3720` | +| `test.wav` | `89f17d9c94c4b31eb320f424628bcbc920abaddbee6e2760fd868bfb1d9a2e47` | + +Trim the first 2.5 seconds for a short firmware validation workload: + +```bash +python examples/arm/silero_vad_example_ethos_u/model_export/generate_smoke_audio.py \ + --source-wav "$SILERO_OUTPUT/test.wav" \ + --output "$SILERO_OUTPUT/validation.wav" +``` + +This produces 40,000 samples. The file remains mono 16-bit PCM at 16 kHz. Using a real speech recording exercises speech and silence probabilities; the helper's optional synthetic tone is a different workload. + +## Export the quantized program and reference + +The exporter calibrates int8 activations and weights, estimates the hidden/cell state quantization parameters, and serializes initialized recurrent state. It lowers supported computation to Ethos-U85-256 while retaining CPU operators needed for state updates and tensor conversion. + +Use `test.wav` as the calibration source and the trimmed clip for validation. The exporter caps calibration at 256 windows by default and processes all 79 validation windows. Enable pipeline failure propagation so an export error remains visible even when saving the log: + +```bash +set -o pipefail +python examples/arm/silero_vad_example_ethos_u/model_export/export_silero_vad_ethos_u.py \ + --jit-model "$SILERO_OUTPUT/silero_vad.jit" \ + --calibration-audio "$SILERO_OUTPUT/test.wav" \ + --validation-audio "$SILERO_OUTPUT/validation.wav" \ + --output-path "$SILERO_OUTPUT/silero_vad_ethos_u.pte" \ + --expected-output-path "$SILERO_OUTPUT/expected_probs.bin" \ + 2>&1 | tee "$SILERO_OUTPUT/export.log" +``` + +The compile specification in this pinned example selects `ethos-u85-256`, `Shared_Sram`, and the default `Ethos_U85_SYS_DRAM_Mid` system configuration. The latter is a compiler performance model, not a claim that the E8 has the FVP memory system. The application copies the PTE into aligned SRAM and supplies SRAM for delegated scratch. Keep the exported memory mode and firmware placement together when changing either. + +The export generates two artifacts that must stay paired: + +| Artifact | Purpose | +| --- | --- | +| `silero_vad_ethos_u.pte` | Quantized program, delegated command streams, weights, and initial recurrent state | +| `expected_probs.bin` | Host quantized-graph probabilities for the 79 windows in `validation.wav` | + +The host reference checks agreement with the quantized model. It is not the floating-point model's accuracy against labeled speech. Calibration and validation overlap in this example; use separate labeled recordings for an accuracy study. + +The pinned source and released environment were checked together: export produced a 363,392-byte PTE and 316-byte reference. Both matched the retained FVP-tested release artifacts byte for byte. This checks host export reproducibility; it does not establish physical E8 execution. + +## Inspect the exported operators + +Require both nonempty artifacts, then generate the CPU operator selection that the firmware build will use: + +```bash +test -s "$SILERO_OUTPUT/silero_vad_ethos_u.pte" +test -s "$SILERO_OUTPUT/expected_probs.bin" +python codegen/tools/gen_oplist.py \ + --model_file_path "$SILERO_OUTPUT/silero_vad_ethos_u.pte" \ + --output_path "$SILERO_OUTPUT/selected_operators.yaml" +cat "$SILERO_OUTPUT/selected_operators.yaml" +``` + +CPU operators are expected. A Vela subgraph's zero CPU-operator count describes that compiled subgraph, not the entire ExecuTorch program. Do not apply a fully delegated MobileSAM check to this stateful model. + +Record hashes so you can identify the exact firmware inputs later: + +```bash +shasum -a 256 \ + "$SILERO_OUTPUT/silero_vad_ethos_u.pte" \ + "$SILERO_OUTPUT/expected_probs.bin" \ + "$SILERO_OUTPUT/validation.wav" \ + > "$SILERO_OUTPUT/model-inputs.sha256" +``` + +## What you've accomplished + +You have exported the U85 program, preserved its initialized recurrent state, and generated a 79-window host reference from the same audio. Next, embed the program and recording in E8 firmware with the required CPU and NPU components. diff --git a/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/prepare-the-environment.md b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/prepare-the-environment.md new file mode 100644 index 0000000000..6c5c71b9e1 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/prepare-the-environment.md @@ -0,0 +1,123 @@ +--- +title: Prepare ExecuTorch and the E8 tools +description: Pin the source and vendor SDKs, install released export dependencies, and prepare the board connection. +weight: 3 +layout: learningpathall +--- + +## Select the host and workspace + +Use an Apple silicon Mac with Python 3.12, CMake, Ninja, Git, and ripgrep installed. Install [Zephyr SDK 0.17.0](https://github.com/zephyrproject-rtos/sdk-ng/releases/tag/v0.17.0) for macOS on Arm, including the Arm toolchain. Set its location below. + +The upstream Silero example supplies the exporter and audio helpers. The downloadable application adds the E8 Zephyr target. It is a local port, so use the exact source and vendor revisions in this lesson when reproducing it. + +Choose a new workspace and define the paths used throughout this Learning Path: + +```bash +export ALIF_WORK="$HOME/silero-alif-e8" +export EXECUTORCH_ROOT="$ALIF_WORK/executorch" +export ZEPHYR_BASE="$ALIF_WORK/zephyr_alif" +export SDK_ALIF="$ALIF_WORK/sdk-alif" +export SILERO_OUTPUT="$ALIF_WORK/model-and-builds" +export SILERO_BUILD="$SILERO_OUTPUT/build-alif" +export ZEPHYR_SDK_INSTALL_DIR=/absolute/path/to/zephyr-sdk-0.17.0 +export ZEPHYR_TOOLCHAIN_VARIANT=zephyr +mkdir -p "$ALIF_WORK" "$SILERO_OUTPUT" +``` + +## Get the model source and E8 application + +Clone the pinned ExecuTorch source, including its build dependencies: + +```bash +git clone https://github.com/pytorch/executorch.git "$EXECUTORCH_ROOT" +git -C "$EXECUTORCH_ROOT" checkout --detach \ + 12dac4aa042b66a7229e8d7d3af2bdde8c9f262f +git -C "$EXECUTORCH_ROOT" submodule update --init --recursive +``` + +Download the [E8 Silero application](../silero-alif-e8.zip) and save it as `$ALIF_WORK/silero-alif-e8.zip`. The archive contains source code for `zephyr/samples/silero-vad-ethosu`, its license notices, and a README. It contains no model weights or audio. + +Check the archive hash, then extract it into the new checkout: + +```bash +shasum -a 256 "$ALIF_WORK/silero-alif-e8.zip" +unzip "$ALIF_WORK/silero-alif-e8.zip" -d "$EXECUTORCH_ROOT" +test -f "$EXECUTORCH_ROOT/zephyr/samples/silero-vad-ethosu/src/main.cpp" +``` + +The expected SHA256 is `7c671b6665bdf11d087cc4574fd8def27a75b2c5f8ec3b3db0a7ef83eac32f78`. + +## Pin the vendor board support + +This application uses Alif's Zephyr fork and SDK. The board name and NPU configuration below belong to these revisions: + +```bash +git clone https://github.com/alifsemi/zephyr_alif.git "$ZEPHYR_BASE" +git -C "$ZEPHYR_BASE" checkout --detach \ + f002a4d8499c35fc70ec4d5324faa7b01c152c72 +git clone https://github.com/alifsemi/sdk-alif.git "$SDK_ALIF" +git -C "$SDK_ALIF" checkout --detach \ + 1db82c9d0cc594e2b548a13699e8a4157e6dcd4c +``` + +The selected target is `alif_e8_dk/ae822fa0e5597xx0/rtss_hp`: the Cortex-M55 HP core with an Ethos-U85-256. Do not substitute the HE target or an older sample's U55 configuration. + +## Install released Python dependencies + +Create an isolated environment for export and firmware code generation: + +```bash +python3.12 -m venv "$ALIF_WORK/.venv" +source "$ALIF_WORK/.venv/bin/activate" +python -m pip install "torch==2.14.0" \ + --index-url https://download.pytorch.org/whl/cpu +python -m pip install "executorch[ethos_u]==1.5.1" \ + "torchao==0.18.0" "cmake<4" "west==1.5.0" fdt pytest +python -m pip install -r "$ZEPHYR_BASE/scripts/requirements.txt" +``` + +These are released packages. The wheel supplies host export and code-generation components; the pinned source supplies the example scripts and the C++ runtime built for Zephyr. Keep this environment active for both operations. The source revision and wheel version are separate pins, so retain the model/firmware validation results for this combination. + +The `ethos_u` extra installs Vela and TOSA tooling. This board workflow uses the Zephyr SDK compiler and does not require an FVP installation. Do not run the source tree's general nightly dependency installer over this environment. + +## Initialize the vendor west workspace + +Initialize west once in the new workspace, resolve the vendor modules, and save the resolved versions: + +```bash +cd "$ALIF_WORK" +west init -l "$ZEPHYR_BASE" +west update +west zephyr-export +west manifest --freeze > "$SILERO_OUTPUT/west-manifest.yml" +python -m pip freeze > "$SILERO_OUTPUT/python-requirements.txt" +cd "$EXECUTORCH_ROOT" +``` + +If resuming an existing workspace, verify its manifest instead of initializing it again. The pinned vendor manifest includes moving module references. Save the frozen manifest above with each build; the Zephyr and SDK commit pins alone do not lock every west project. + +## Prepare the board tools + +Install Alif Security Toolkit for your E8 silicon revision and SEGGER J-Link. Follow the Security Toolkit and debug-probe portions of the [E8 board setup guide](https://learn.arm.com/learning-paths/embedded-and-microcontrollers/alif-image-classification/board-setup/). Its CMSIS application build is a separate example. + +Connect the board's **PRG USB** port with a data-capable cable, retain the vendor jumper settings, and leave **SW4 on SEUART** for programming. Confirm that the host discovers the serial port and J-Link probe. Close applications holding the programming serial port. + +The E8 DevKit has an onboard USB-to-UART bridge. SW4 routes it to SEUART for programming or UART4 for the HP application console at 115200 baud, 8N1. No separate adapter is required. The programming lesson shows the single switch change to UART4 after flashing and the return to SEUART before any later programming. + +Set the installed tool locations and HP device: + +```bash +export ALIF_SE_TOOLS_DIR=/absolute/path/to/alif-security-toolkit +export JLINK_COMMANDER=/absolute/path/to/JLinkExe +export ALIF_DEVICE=AE822FA0E5597LS0_M55_HP +test -d "$ALIF_SE_TOOLS_DIR/build/config" +test -x "$JLINK_COMMANDER" +command -v west +``` + +Match the toolkit's A0/A1 configuration to your actual board. The device above selects the HP core; it does not detect the silicon revision for you. + +## What you've accomplished + +You have selected the source, E8 vendor revisions, released Python packages, and board tools. Next, generate the quantized streaming model and its matching probability reference. diff --git a/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/program-the-board.md b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/program-the-board.md new file mode 100644 index 0000000000..1ab9342226 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/program-the-board.md @@ -0,0 +1,115 @@ +--- +title: Program the Silero firmware on the E8 HP core +description: Generate an application image configuration and program the firmware, embedded model, and audio into E8 MRAM. +weight: 6 +layout: learningpathall +--- + +## Check the firmware and programming connection + +The firmware contains both the PTE and validation audio. Program the single `zephyr.bin` produced by the previous lesson into MRAM. Rebuild and reprogram that firmware whenever you change the model or embedded recording. + +Connect the board's **PRG USB** port with a data-capable cable. For programming, leave **SW4 on SEUART** and close any terminal using the Security Toolkit's serial port. Use the Security Toolkit configured for your E8 silicon revision, with the device configuration and boot images already established in the setup lesson. + +Check the paths from your active shell: + +```bash +test -s "$SILERO_BUILD/zephyr/zephyr.bin" +test -s "$SILERO_BUILD/zephyr/zephyr.elf" +test -x "$ALIF_SE_TOOLS_DIR/app-gen-toc" +test -x "$ALIF_SE_TOOLS_DIR/app-write-mram" +``` + +The pinned HP board configuration places the Zephyr application at **`0x80200000`**: MRAM base `0x80000000` plus flash load offset `0x200000`. This is the Zephyr HP partition used here. The address must agree with your generated build configuration and linker layout. + +## Preserve the existing boot configuration + +The Security Toolkit generates an application table of contents (AToC) from a JSON configuration. Start with the configuration already used by your working E8 boot setup. Keep its device configuration, AP images, HE images, signing options, and clock settings. This lesson changes only the HP application entry. + +The standard Zephyr runner uses `build/config/app-cpu-stubs.json` as its template. Use that file only after confirming that its device configuration and companion images match your E8. If your working setup uses a different application configuration, set `ALIF_APP_CONFIG` to that file instead. + +Save a copy of the toolkit configuration and stage the new binary under a distinct name: + +```bash +export ALIF_APP_CONFIG="$ALIF_SE_TOOLS_DIR/build/config/app-cpu-stubs.json" + +test -s "$ALIF_APP_CONFIG" +tar -czf "$SILERO_OUTPUT/setools-config-before-silero.tar.gz" \ + -C "$ALIF_SE_TOOLS_DIR" build/config +cp "$SILERO_BUILD/zephyr/zephyr.bin" \ + "$ALIF_SE_TOOLS_DIR/build/images/silero-vad-hp.bin" +``` + +The next command reads that template, checks the build's HP flash address, and writes a separate application configuration. It leaves the original template unchanged: + +```bash +python - "$SILERO_BUILD/zephyr/.config" "$ALIF_APP_CONFIG" \ + "$SILERO_OUTPUT/silero-vad-hp.json" <<'PY' +from pathlib import Path +import json +import sys + +build_config, template, destination = map(Path, sys.argv[1:]) +values = dict( + line.split("=", 1) + for line in build_config.read_text().splitlines() + if line.startswith("CONFIG_") and "=" in line +) +assert values.get("CONFIG_RTSS_HP") == "y", "Build must target the HP core" +address = int(values["CONFIG_FLASH_BASE_ADDRESS"], 0) + int( + values["CONFIG_FLASH_LOAD_OFFSET"], 0 +) +assert address == 0x80200000, f"Unexpected HP flash address: {address:#x}" + +configuration = json.loads(template.read_text()) +hp = configuration["HP_APP"] +assert hp["cpu_id"] == "M55_HP", "Template must contain the HP application" +hp["disabled"] = False +hp["binary"] = "silero-vad-hp.bin" +hp["mramAddress"] = hex(address) +hp["flags"] = ["boot"] +hp.pop("loadAddress", None) +destination.write_text(json.dumps(configuration, indent=4) + "\n") +print(json.dumps(hp, indent=2)) +PY +``` + +Review the printed HP entry. It must select `M55_HP`, name `silero-vad-hp.bin`, use `mramAddress` of `0x80200000`, and contain the `boot` flag. MRAM execution uses `mramAddress`; the template's SRAM `loadAddress` and `load` flag must be absent from this entry. + +Review the complete new JSON before generating images. Its other entries should match the known working template, and every referenced binary must remain available in the toolkit's `build/images` directory. Keep the existing Security Enclave table of contents (SETOC) and device clock configuration; this application update does not require replacing them. + +## Generate and program the application image + +Run the toolkit from its installation directory so it finds its configuration and image files: + +```bash +cd "$ALIF_SE_TOOLS_DIR" +./app-gen-toc -f "$SILERO_OUTPUT/silero-vad-hp.json" +./app-write-mram -p +cd "$EXECUTORCH_ROOT" +``` + +Select the E8's SEUART programming port when prompted. Require successful image generation and MRAM programming with no device-revision, image, authentication, or serial-port errors. Programming success establishes that the image was transferred; the next lesson checks model execution and output agreement. + +## Capture the application console + +The pinned E8 HP device tree selects **UART4 at 115200 baud, 8N1** for the Zephyr console. The E8 DevKit routes its onboard USB-to-UART bridge between SEUART, UART2, and UART4 using SW4. It uses the same PRG USB connection for programming and application logging; no separate serial adapter is needed. + +After programming completes, disconnect board power, move **SW4 from SEUART to UART4**, and reconnect PRG USB. Rediscover the host serial device, then set `ALIF_CONSOLE` to that device. PySerial is installed by the Zephyr requirements; use its terminal to capture one run: + +```bash +python -m serial.tools.list_ports +export ALIF_CONSOLE=/dev/cu.replace-with-your-application-uart +python -u -m serial.tools.miniterm "$ALIF_CONSOLE" 115200 --raw \ + 2>&1 | tee "$SILERO_OUTPUT/alif-silero.log" +``` + +Start capture first, then press the board’s **RESET** button to begin a fresh run. Wait for all probability lines and `Model executed successfully.` before exiting the terminal with **Ctrl+]**. Keep one run per log file. If the application reports an error or the console remains silent, preserve that log and diagnose the failed stage before retrying the probability comparison. + +Before any later programming operation, close the terminal, disconnect board power, return **SW4 to SEUART**, and reconnect PRG USB. SETOOLS cannot communicate with the Secure Enclave while the bridge is routed to UART4. + +## What you've accomplished + +After completing these steps, you have programmed the HP firmware with its embedded model and audio while retaining the E8's other boot-image and device settings. You have also captured a complete application run for validation. + +Next, compare its 79 speech probabilities with the matching quantized host reference. diff --git a/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/silero-alif-e8.zip b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/silero-alif-e8.zip new file mode 100644 index 0000000000..3b89bf75af Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/silero-alif-e8.zip differ diff --git a/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/troubleshoot.md b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/troubleshoot.md new file mode 100644 index 0000000000..ec5bedf10c --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/troubleshoot.md @@ -0,0 +1,61 @@ +--- +title: Diagnose export and E8 execution failures +description: Locate failures at the package, export, firmware, programming, and streaming-output checkpoints. +weight: 8 +layout: learningpathall +--- + +## Start with the first failed checkpoint + +Keep the exporter log, frozen west manifest, resolved Python requirements, model hashes, ELF/map, and complete UART capture. Work from the earliest failing step before changing the model or numerical tolerances. + +| Symptom | Likely checkpoint and next action | +| --- | --- | +| Export import or native-library error | Activate the Python 3.12 environment from preparation. Check the released package versions and avoid mixing another checkout's editable installation with this environment. | +| WAV rejected | Require uncompressed mono PCM16 at 16 kHz. Generate `validation.wav` from the pinned `test.wav` using the supplied helper. | +| Export fails during state-buffer handling | Use the exact pinned exporter and ExecuTorch package. The serialized quantized initial state is part of the model contract. | +| Board qualifier or devicetree property missing | Check both vendor commits and the full `rtss_hp` target. This sample depends on the vendor fork's E8 bindings. | +| U55 selected or driver configuration conflict | Use the supplied E8 board configuration and U85-256 target. Clean the build directory after changing an NPU selection. | +| CPU operator missing at load or execution | Rebuild registrations from the exact PTE, including portable and quantized kernels. CPU state updates are required. | +| Linker RAM overflow | Inspect `zephyr.map`, the PTE-copy size, and static allocator pools. Check the intended `APP_RAM` region before changing capacities. | +| `load_method` or temporary allocation failure | Retain the reported planned sizes and allocator error. Linking alone does not establish runtime pool sufficiency. Increase only the relevant pool within the verified SRAM budget, then rebuild. | +| Toolkit cannot find the E8 | Check PRG USB data connectivity, SEUART switch position, serial-port ownership, and the toolkit's silicon revision. | +| Boot succeeds but inference faults or hangs | Check NPU power/clock, U85 driver selection, SRAM placement/access, and model/runtime compatibility. Retain the complete fault and delegate logs. | +| Only silence or no `SEGMENT` lines | First require all 79 `PROB` values and compare them. A segment is conditional on the probabilities; its absence alone is not a capture failure. | + +## Check the Python environment + +Print the host package versions using the active interpreter: + +```bash +python - <<'PY' +from importlib.metadata import version + +for package in ("torch", "torchao", "executorch", "ethos-u-vela", "west"): + print(f"{package}=={version(package)}") +PY +``` + +Require PyTorch `2.14.0`, TorchAO `0.18.0`, ExecuTorch `1.5.1`, and west `1.5.0`. Record the Vela version resolved by the released Ethos-U extra. If these differ from the saved requirements, recreate the isolated environment using the preparation lesson before debugging the export. + +## Check state and framing before tolerances + +If probabilities differ, verify the exact PTE, reference, and audio hashes first. Then check that the firmware: + +1. Loads the method once for the recording. +2. Starts waveform history at zero and retains the last 64 actual samples between calls. +3. Preserves the model's initialized hidden/cell state between invocations. +4. Divides PCM16 samples by 32,768 and zero-pads the final incomplete window. +5. Captures exactly one 79-window run in the log. + +The final frame contains only 64 new samples. A tail-only difference often points to padding or history handling. Divergence that grows over the recording can indicate recurrent-state resets or incorrect initialization. Replacing serialized int8 initial values with literal zero bytes changes the quantized state when its zero point is nonzero. + +## Separate validation from performance work + +Finish the probability comparison before measuring latency or extending input capture. The application reports an inference-loop duration that includes frame preparation, method execution, and output retrieval, while probability printing happens afterward. That duration is not an NPU-only measurement, and 32 ms is the audio-window step rather than measured execution time. + +The local port has no retained physical-board probability or timing result yet. Record the first successful E8 run with its firmware/model hashes and clocks before adding hardware claims to this Learning Path. The existing FVP evidence remains useful for the export and reference flow, but does not substitute for the E8 checks. + +## What you've learned + +You can locate a failure in the host environment, export, firmware, board connection, or streaming contract. After the E8 comparison passes, the next extension is to replace the prerecorded input with a microphone pipeline that supplies the same 512-sample step and preserves both forms of state. diff --git a/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/understand-the-system.md b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/understand-the-system.md new file mode 100644 index 0000000000..0f65831eca --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/understand-the-system.md @@ -0,0 +1,56 @@ +--- +title: Understand streaming Silero VAD on the Alif E8 +description: Trace audio frames, recurrent state, and speech probabilities through the E8 CPU and NPU. +weight: 2 +layout: learningpathall +--- + +{{% notice Important %}} +This Learning Path is a local draft. Host export and the E8 firmware cross-build have passed with the pinned dependencies. Physical E8 execution has not been validated. Complete programming and the probability comparison before treating this port as a working board deployment. +{{% /notice %}} + +## Detect speech in a recorded audio stream + +You'll export Silero voice activity detection (VAD), build a Zephyr application for the Alif Ensemble E8 Development Kit, and compare its speech probabilities with a host reference. VAD estimates whether each audio window contains speech. It does not transcribe speech or identify the speaker. + +The application targets the board's Cortex-M55 High-Performance (HP) core and Ethos-U85-256 NPU. It embeds a recorded WAV clip and the exported ExecuTorch program in the firmware. It processes the recording sequentially, using the same state and framing rules that a streaming source would require. Microphone capture is outside this application. + +## Follow one audio frame + +The source recording must be uncompressed, mono, 16-bit PCM WAV at 16 kHz. The preparation step converts its signed samples to floating-point values by dividing by 32,768. Each model invocation receives **512 new samples plus 64 samples of history**, giving one `[1, 576]` floating-point input tensor. + +The runtime follows this sequence: + +1. **Build the input window.** Copy the previous 64 samples into the start of the input, then append up to 512 new samples. The first window starts with zero history. Pad the final window with zeros if fewer than 512 new samples remain. +2. **Execute the method.** Reuse the loaded ExecuTorch method for the entire recording. Delegated `EthosUBackend` subgraphs execute on the NPU. CPU operators handle work between delegates, including quantization, tensor copies, and recurrent-state updates. +3. **Retain streaming state.** The exported model stores its hidden and cell state in a persistent int8 buffer. Export initializes that buffer to its quantized representation of zero. Each invocation reads the previous state and writes the next state. Keep it across calls; resetting it for every window changes the model's behavior. +4. **Read the probability.** Each call returns one floating-point speech probability. The application labels a frame `speech` when its probability is strictly greater than `0.5`; otherwise it labels it `silence`. Consecutive speech frames form a reported segment. + +The two forms of history have different owners: + +| Information retained between calls | Owner | Initialization | +| --- | --- | --- | +| Previous 64 waveform samples | Application input-framing code | Floating-point zeros before the first window | +| Quantized hidden and cell state | Exported model's mutable int8 buffer | Serialized initial values when the method is loaded | + +Reloading the method or clearing either history between windows breaks the streaming contract. For a new, independent recording, start with a fresh application run so both histories return to their initial values. + +## Set the validation workload + +The supplied validation clip contains 40,000 samples: **2.5 seconds at 16 kHz**. The runtime makes `ceil(40000 / 512) = 79` calls. The last call contains 64 new samples and 448 padding samples, in addition to the 64-sample history. + +Each call advances by 512 samples, or 32 ms of audio. Log timestamps describe positions in the recording; they do not measure how long the board takes to execute inference. With padding, the final frame boundary is 2.528 seconds even though the original clip lasts 2.5 seconds. + +The host export produces `expected_probs.bin` from the quantized graph using the same clip, framing, and initial state. The board's probabilities should agree with that reference within the validation tolerances. This checks deployment behavior. It does not establish speech-detection accuracy on unseen recordings, because the example's calibration and validation audio overlap. + +## Keep CPU execution and NPU execution together + +This model intentionally retains CPU operators around delegated subgraphs. Build the portable and quantized operators selected from the PTE as well as the Ethos-U backend. Requiring the entire program to be one NPU partition would reject the state updates needed by this application. + +The separate Corstone-320 example provides the exporter and streaming logic used by this port. Its simulator results do not establish E8 execution time, memory fit, or completed NPU execution on the physical board. Validate those on the E8 using the firmware and model built in this Learning Path. + +## What you've learned and what's next + +You now know why each invocation accepts 576 samples, how waveform history differs from recurrent state, and why CPU operators remain in an NPU application. You can also distinguish the 32 ms audio-window step from inference latency. + +Next, prepare the matching source and E8 tools, then export the model and host reference before building the Zephyr application. diff --git a/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/validate-speech-probabilities.md b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/validate-speech-probabilities.md new file mode 100644 index 0000000000..d92f646d65 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/deploy-silero-vad-on-alif-e8/validate-speech-probabilities.md @@ -0,0 +1,92 @@ +--- +title: Validate the E8 speech probabilities +description: Capture one board run and compare every speech probability with the quantized host reference. +weight: 7 +layout: learningpathall +--- + +{{% notice Important %}} +Physical E8 execution remains unverified in this local draft. The steps below define the checks to perform on your board; they are not a record of a completed hardware run. +{{% /notice %}} + +## Capture one complete application run + +Use the UART4 connection and capture command from the programming lesson, with SW4 set to UART4 after programming. Save the output as `$SILERO_OUTPUT/alif-silero.log`. + +Start capture before resetting or starting the application. Retain startup messages, every probability line, segment lines, and the completion message `Model executed successfully.`. Capture one run per log file. Concatenating multiple runs duplicates probabilities and invalidates the comparison. + +The application emits one `PROB speech|silence` entry per input window. It also reports `SEGMENT speech` for consecutive speech frames. Zephyr or ExecuTorch prefixes are acceptable: the comparison script searches within each line for `PROB`. + +For the 40,000-sample validation clip, require all **79 probability entries** and a successful completion message. A detected NPU device, successful model load, or first probability alone does not establish completion. Preserve any allocation, missing-operator, or delegate errors instead of filtering them out of the captured log. + +## Check the matching host reference + +Use `expected_probs.bin` from the same export and validation clip used to build this firmware. It contains one little-endian float32 probability per window: 79 values occupy 316 bytes. Check that the reference is complete before comparing the board log: + +```bash +python - "$SILERO_OUTPUT/expected_probs.bin" <<'PY' +from pathlib import Path +import math +import struct +import sys + +path = Path(sys.argv[1]) +data = path.read_bytes() +assert len(data) == 79 * 4, f"Expected 316 bytes, got {len(data)}" +values = struct.unpack("<79f", data) +assert all(math.isfinite(value) for value in values), "Non-finite reference value" +print(f"Reference contains {len(values)} finite probabilities") +PY +``` + +The expected output is: + +```output +Reference contains 79 finite probabilities +``` + +Keep the PTE, validation WAV, host reference, firmware ELF, and board log together. Re-exporting a model while retaining an older reference or firmware creates a different test. + +## Compare the probability sequence + +Run the existing comparison script from the matching ExecuTorch checkout. These options require a maximum absolute probability error of `0.25`, a mean absolute error of `0.02`, and no changed speech/silence decisions at threshold `0.5`: + +```bash +python "$EXECUTORCH_ROOT/examples/arm/silero_vad_example_ethos_u/runtime/compare_vad_probs.py" \ + --expected "$SILERO_OUTPUT/expected_probs.bin" \ + --actual-log "$SILERO_OUTPUT/alif-silero.log" \ + --threshold 0.5 \ + --atol 0.25 \ + --mean-atol 0.02 \ + --max-threshold-mismatches 0 +``` + +A successful comparison reports `Compared 79 probability values`, zero threshold mismatches, and `Validation passed`. It also prints the maximum and mean errors measured from your run. Missing values, extra values, non-finite values, exceeded tolerances, or changed threshold decisions cause the command to fail. + +The decision test uses `probability > 0.5`. A value equal to `0.5` is silence. The comparator checks probabilities and decisions; it does not compare `SEGMENT` lines or validate segment timestamps. + +## Interpret failures before changing tolerances + +| Failure | Check first | +| --- | --- | +| No `PROB` entries | Correct application UART, complete boot, successful model load, and first execution | +| Fewer than 79 entries | Truncated capture, reset during execution, inference error, or a shorter embedded clip | +| More than 79 entries | Repeated application runs in one log or a different embedded clip | +| Errors grow across successive windows | Method reloading, recurrent-state reinitialization, or incorrect 64-sample history | +| Only the tail differs | Final-window zero padding and the exact number of embedded samples | +| Small errors change speech labels | Values near the threshold; inspect the differing windows without relaxing the zero-mismatch check | +| Missing CPU operator | Selective portable or quantized operator registration from the exact PTE | + +If the comparison fails, retain the failing log and verify the model, clip, initial state, and firmware inputs before trying a larger tolerance. A passing comparison establishes agreement with this quantized host reference on this clip. It does not establish accuracy against labeled speech, performance on other audio, or microphone behavior. + +## Record the physical-board evidence + +Record the E8 silicon revision, Cortex-M55 HP target, NPU configuration, vendor revisions, dependency versions, model and firmware hashes, linker memory report, and complete UART log. If you measure execution time, state whether it covers one `method.execute()` call or the complete application loop, and retain the clock configuration and measurement method. + +Audio timestamps are not wall-clock measurements. Corstone-320 counter values are simulator measurements. Neither can be reported as E8 hardware latency. Confirm the delegated backend is linked and completes on the board before attributing results to NPU execution. + +## What you've accomplished + +After completing these checks on the E8, you have captured a complete 79-window run and compared every board probability and speech decision with the matching quantized host reference. You have also separated output agreement from accuracy and performance claims. + +Next, diagnose any failed checkpoint using the troubleshooting lesson. For a later extension, use separate labeled validation recordings before evaluating a microphone input pipeline.