Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,19 @@ Arm ML Inference Advisor (MLIA) helps you evaluate a model for an Arm target bef

You'll use the `mlia-neural-technology` package to evaluate two pretrained, quantized MobileNet V1 image classifiers from the TensorFlow public model catalog. You'll compare a baseline with a smaller-input variant under the same Neural Technology configuration. You don't need a model stored in this repository or a device to run the estimator.

The tested example reduces estimated total cycles from `136,962` to `79,824`. These are compiler estimates, not device measurements or an accuracy result. You'll learn about the model and configuration assumptions behind this comparison in [Compare a smaller model under the same target profile](/learning-paths/mobile-graphics-and-gaming/mlia-nx-performance-estimator/6-iterate-and-compare/).
The reference example, recorded with Neural Technology `0.2.0` and LiteRT converter `0.1.2`, reduces estimated total cycles from `136,962` to `79,824`. These are compiler estimates, not device measurements or an accuracy result. You'll learn about the model and configuration assumptions behind this comparison in [Compare a smaller model under the same target profile](/learning-paths/mobile-graphics-and-gaming/mlia-nx-performance-estimator/6-iterate-and-compare/).

The setup installs Neural Technology `0.3.0` and LiteRT converter `0.1.3`, so your generated estimates can differ from this reference. Neural Technology `0.3.0` is required for the optional ETRecord and ETDump VGF profiling workflow.

If you have a neural accelerator (NX)-capable Android device and a compatible native Vulkan driver, you can execute the model on the device. You can then bring a measured capture back to MLIA. For more information, see [(Optional) Run and profile the model on an NX device](/learning-paths/mobile-graphics-and-gaming/mlia-nx-performance-estimator/7-run-on-nx-device/).

## How the NX and Ethos-U workflows are connected

The [Analyze Ethos-U models with MLIA Learning Path](/learning-paths/embedded-and-microcontrollers/analyze-ethos-u-models-with-mlia/) applies the same workflow to Arm Ethos-U: select a target profile, check compatibility, inspect performance, and use the advice to guide your next change. It uses Vela for compiler estimates and Corstone for whole-model NPU performance counters. In this Learning Path, you'll use the Neural Technology plugin and NX Performance Estimator, with optional measurements from an Android device.

The report-reading approach carries across targets, but their backends, supported operators, and performance metrics differ. Don't compare cycle counts across targets assuming that they describe the same hardware configuration.
The report-reading approach carries across targets, but their backends, supported operators, and performance metrics differ. Don't compare cycle counts across targets assuming that they describe the same hardware configuration. For an editor-based visual workflow, see the [Ethos-U VS Code and Model Explorer walkthrough](/learning-paths/embedded-and-microcontrollers/analyze-ethos-u-models-with-mlia/7-vscode-plugin/).

If you already have an ExecuTorch ETRecord and matching ETDump from a deployed program, you can also [analyze recorded runtime timings and VGF profiling](/learning-paths/mobile-graphics-and-gaming/mlia-nx-performance-estimator/8-executorch-profiling/). This optional workflow uses different artifacts from the Scenario Runner capture directory.

## Understand the conversion and analysis flow

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ layout: learningpathall

## Perform an environment check

Use an x86-64 Ubuntu 24.04 LTS development host with CPython 3.12. The `mlia-neural-technology` version `0.2.0` wheel is published for CPython 3.12 on x86-64 Linux with glibc version 2.39 or later. Ubuntu 22.04, other Python versions, native macOS, and native Linux on Arm aren't supported by this wheel.
Use an x86-64 Ubuntu 24.04 LTS development host with CPython 3.12. The `mlia-neural-technology` version `0.3.0` wheel is published for CPython 3.12 on x86-64 Linux with glibc version 2.39 or later. Ubuntu 22.04, other Python versions, native macOS, and native Linux on Arm aren't supported by this wheel.

Check your architecture, Python version, and glibc version:

Expand Down Expand Up @@ -40,7 +40,7 @@ cd mlia-demo
python3.12 -m venv mlia-venv
source mlia-venv/bin/activate
python -m pip install --upgrade pip
python -m pip install mlia-neural-technology==0.2.0 mlia-converters-litert==0.1.2
python -m pip install mlia==0.13.0 mlia-neural-technology==0.3.0 mlia-converters-litert==0.1.3
python -m pip check
```

Expand All @@ -52,6 +52,8 @@ These are public PyPI packages. The Neural Technology plugin installs the follow

The LiteRT converter plugin lets MLIA analyze `.tflite` files directly. You don't need TensorFlow, a GPU driver, or a separate model-training environment for this example.

Neural Technology `0.3.0` requires MLIA core `0.13.x`. LiteRT converter `0.1.3` supports this core version. Upgrade these packages together when reusing an existing environment.

The expected output is:

```output
Expand All @@ -76,6 +78,19 @@ The estimator executable is bundled with the public wheel. MLIA offers to instal
If `pip` reports that no matching distribution exists for `mlia-neural-technology`, check the three host requirements again and upgrade `pip` inside the environment. A virtual environment doesn't change the host architecture or glibc version. Don't substitute an older Python version or Ubuntu 22.04 for this release.
{{% /notice %}}

## (Optional) Prepare Model Explorer visualizations

The [MLIA Model Explorer plugin](https://pypi.org/project/mlia-model-explorer/) adds graph visualizations with performance overlays. Version `0.2.0` supports MLIA core `0.13.x`. The `vgf` extra includes the adapter for the compiled model you'll inspect later. The plugin generates a standalone HTML page that you can open directly in your browser. You don't need to install Model Explorer separately or run additional services.

Install the plugin in the same environment:

```bash
python -m pip install "mlia-model-explorer[vgf]==0.2.0"
python -m pip check
```

The expected output from `pip check` is `No broken requirements found.` The [ExecuTorch profiling steps](/learning-paths/mobile-graphics-and-gaming/mlia-nx-performance-estimator/8-executorch-profiling/) install an additional extra for ETRecord and PTE adapters.

## What you've accomplished and what's next

You've installed MLIA with the Neural Technology and LiteRT converter plugins.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ assert not failed_checks, "Review the non-passing operator checks."
PY
```

The expected output for the pinned packages and baseline is:
The reference output for the baseline is:

```output
Compatibility status: ok
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Run a performance check with the same model and target profile:
mlia check models/baseline.tflite --target-profile neural-technology --performance --output-dir runs/baseline
```

With the tested packages and `neural-technology` profile, expect an estimated `136,962` total cycles and about `0.09131 ms` per inference.
The reference run with Neural Technology `0.2.0`, LiteRT converter `0.1.2`, and the `neural-technology` profile produced an estimated `136,962` total cycles and about `0.09131 ms` per inference. The setup now uses Neural Technology `0.3.0` and LiteRT converter `0.1.3`; use your generated report for the current result. These figures are compiler estimates, rather than timings measured on a device.

## Locate and read the reports

Expand Down Expand Up @@ -48,7 +48,7 @@ for name in ("total_cycles", "compute_cycles", "dram_cycles", "inference_time",
PY
```

The expected output is:
The reference output is:

```output
total_cycles: 136962 cycles
Expand All @@ -64,6 +64,30 @@ For the baseline, `dram_cycles` is `94,396`, compared with `34,976` compute cycl

Use `results[].breakdowns` and their `entity_id` links to `results[].entities` to inspect individual chains and cascades. These are compiler scheduling groups. They're not necessarily one framework layer each. Source-operator provenance helps you relate those groups to the original graph. Some metrics overlap or use non-additive aggregation policies, so don't sum every breakdown to recreate the model total.

## (Optional) Explore results in Model Explorer

After installing the compatible Model Explorer plugin in [Set up your MLIA environment](/learning-paths/mobile-graphics-and-gaming/mlia-nx-performance-estimator/2-devenv/#optional-prepare-model-explorer-visualizations), rerun the check into a separate directory with visualization enabled:

```bash
mlia check models/baseline.tflite \
--target-profile neural-technology \
--performance \
--output-dir runs/baseline-visualized \
--model-explorer open
```

`--model-explorer open` creates a self-contained HTML visualization and opens it in your browser. Use `--model-explorer files` to generate the files without opening a browser. If the browser doesn't open, open `runs/baseline-visualized/mlia-output/model-explorer/bundle/index.html` manually.

Inspect the generated MLIA hierarchy alongside **Source Model**:

1. Select a cycle metric in the node-data controls to color the hierarchy by cost.
2. Expand the hierarchy and select a chain or cascade to read its reported values.
3. Follow its mapping to **Source Model**, where available, to find the related model operations.

Use these overlays to find scheduling groups worth investigating. A group can contain several operations, so its cost isn't a separate measurement for each operation. Keep using the model-level report for whole-model totals.

For the editor workflow, follow the [Ethos-U VS Code and Model Explorer walkthrough](/learning-paths/embedded-and-microcontrollers/analyze-ethos-u-models-with-mlia/7-vscode-plugin/). Its example uses Ethos-U and Vela; use the NX metrics from this analysis when exploring your results.

## (Optional) Compare a different built-in profile

Keep the model unchanged when exploring a different target configuration. For example, run the peak profile into its own directory:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ for name in ("total_cycles", "inference_time", "inferences_per_second", "dram_cy
print(f"{name}: {before:.5f} -> {after:.5f} ({change:+.2f}%)")
PY
```
The following table includes example results:
The following table contains reference results recorded with Neural Technology `0.2.0` and LiteRT converter `0.1.2`. Compare your own reports after installing the updated packages:

| Metric | Baseline, 224 × 224 | Smaller, 128 × 128 | Change |
| --- | ---: | ---: | ---: |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,47 @@ Your cycle counts will depend on your device and its configuration.

To inspect measured NX performance in Model Explorer and trace costs to Python source, follow [Capture and analyze NX profiling data with MLIA for VS Code](/learning-paths/mobile-graphics-and-gaming/profile-nx-with-mlia-vscode/).

## (Optional) Explore measured performance and TOSA operations

After installing the compatible Model Explorer plugin in [Set up your MLIA environment](/learning-paths/mobile-graphics-and-gaming/mlia-nx-performance-estimator/2-devenv/#optional-prepare-model-explorer-visualizations), rerun the measured analysis with the same VGF and complete capture directory:

```bash
mlia check nx-demo/baseline.vgf \
--target-profile neural-technology \
--performance \
--profiling-data "nx-demo/captures/$CAPTURE_NAME" \
--output-dir runs/baseline-measured-visualized \
--model-explorer open
```

### View measured cycle counts

The command opens Model Explorer with the capture's results already loaded. In the generated MLIA hierarchy, select a cycle metric using the node-data controls. Expand the hierarchy and select an entity to inspect its reported cycle count. Follow the source mapping, where available, to see related operations in **Source Model**.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might add some clarifying that a source mapping associates operations with measured execution groups; it does not necessarily provide an independent cycle measurement for each operation.


A source mapping associates operations with measured execution groups. It doesn't necessarily provide an independent cycle measurement for each operation. Several operations can share the same group measurement, so don't sum their displayed cycle counts as separate costs.

These overlays show the capture's measurements. Only entities with reported metrics have values; a missing value doesn't mean zero cycles. The `951,154` cycles in the reference CLI output describe the whole-model result, rather than the selected convolution.

MLIA saves the visualization in `runs/baseline-measured-visualized/mlia-output/model-explorer/bundle/index.html`. Open this self-contained file to revisit or share the results without rerunning the capture or starting a server.

### Inspect a TOSA convolution

In **Source Model**, inspect `nx-demo/baseline.vgf`. The reference graph contains 103 nodes. Expand `graph_segment_0` and select the first `CONV2D` node, between two `RESCALE` nodes. Your graph can differ with compiler or package changes.

This view exposes the TOSA operations in the VGF graph. You can see how the original MobileNet layer is represented after MLIA's LiteRT-to-TOSA conversion and VGF generation. The screenshot shows the operation's attributes.

![Model Explorer shows a TOSA CONV2D operation between two RESCALE nodes in baseline.vgf. The Attributes panel links to the TOSA CONV2D specification and shows the MobileNet source location, connecting the converted operation to its definition and original model.#center](model-explorer-conv2d.png "Inspect a TOSA convolution in the VGF graph")

In **ATTRIBUTES**, **opcode** identifies the TOSA operation as `CONV2D`. Select **TOSA spec: CONV2D** in the **spec** field to open the operator definition. Compare it with the values shown for this node:

| Field | Value in the reference graph | What it tells you |
| --- | --- | --- |
| **result type** | `uint32[1, 112, 112, 8]` | The intermediate tensor has a batch size of one, 112 × 112 spatial dimensions, and eight channels. |
| **arg1 stride** | `int32[2, 2]` | A stride of two reduces the 224 × 224 input to 112 × 112. |
| **debug info** | `MobilenetV1/MobilenetV1/Conv2d_0/Relu6` | Relates the TOSA operation to its source location in MobileNet. Expand the field to read the full name. |

Select the following `RESCALE` node to inspect how TOSA scales the convolution's intermediate integer result into the output's quantized range. Search for `DEPTHWISE_CONV2D` and follow its specification link to compare the two convolution operators.


## Use the measurements

Expand All @@ -249,3 +290,5 @@ Check the predictions on real images as well. A faster model isn't useful if it
You've run the VGF file on an NX-capable device, checked that Scenario Runner completed, and confirmed that the neural statistics capture contains data. You've also returned the capture to MLIA for measured analysis.

Use this evidence alongside model accuracy tests and end-to-end application profiling before making deployment decisions for your own use cases.

If your deployment uses ExecuTorch, continue with [Analyze ExecuTorch ETRecord and ETDump profiling](/learning-paths/mobile-graphics-and-gaming/mlia-nx-performance-estimator/8-executorch-profiling/) using artifacts from that deployed program.
Loading
Loading