Repository navigation
Add ETRecord/ETDump MLIA Model Explorer section - #4015
Conversation
narumol-arm
left a comment
narumol-arm
left a comment
There was a problem hiding this comment.
I have added some comments. Thank you very much.
| You'll use the `mlia-neural-technology` package to evaluate two pretrained, quantized MobileNet V1 image classifiers from the TensorFlow public model catalog. You'll compare a baseline with a smaller-input variant under the same Neural Technology configuration. You don't need a model stored in this repository or a device to run the estimator. | ||
|
|
||
| The tested example reduces estimated total cycles from `136,962` to `79,824`. These are compiler estimates, not device measurements or an accuracy result. You'll learn about the model and configuration assumptions behind this comparison in [Compare a smaller model under the same target profile](/learning-paths/mobile-graphics-and-gaming/mlia-nx-performance-estimator/6-iterate-and-compare/). | ||
| The reference example, recorded with Neural Technology `0.2.0` and LiteRT converter `0.1.2`, reduces estimated total cycles from `136,962` to `79,824`. The setup uses updated packages; compare the reports you generate with those versions. These are compiler estimates, not device measurements or an accuracy result. You'll learn about the model and configuration assumptions behind this comparison in [Compare a smaller model under the same target profile](/learning-paths/mobile-graphics-and-gaming/mlia-nx-performance-estimator/6-iterate-and-compare/). |
There was a problem hiding this comment.
This part might not be relevant to the ETRecord/ETDump but I just would like to confirm to avoid confusion.
For the new VGF Profiling for ETRecord/ETDump, the Neural Technology should be v0.3.0.
mlia-neural-technology: https://github.com/Arm-Debug/mlia-neural-technology/releases/tag/v0.3.0
|
|
||
| ## (Optional) Prepare Model Explorer visualizations | ||
|
|
||
| The [MLIA Model Explorer plugin](https://pypi.org/project/mlia-model-explorer/) adds graph visualizations with performance overlays. Version `0.2.0` supports MLIA core `0.13.x`. The `vgf` extra includes the adapter for the compiled model you'll inspect later. The plugin generates a standalone HTML page, so you don't need to install or start a separate Model Explorer server. |
There was a problem hiding this comment.
It might be better to state something like "The plugin generates a standalone HTML page that you can open directly in your browser, without needing to install Model Explorer separately or run any additional services."
Although the interactive Model Explorer application can be served through a local web server, referring to it as a “separate Model Explorer server” might be a little misleading.
|
|
||
| ## (Optional) Open the profiling views in Model Explorer | ||
|
|
||
| Use Model Explorer `0.2.0`, which supports MLIA core `0.13.x`. Install its `executorch` extra in the same environment, even if you installed the `vgf` extra during setup: |
There was a problem hiding this comment.
Might be better to specific it to MLIA Model Explorer 0.2.0 to avoid confusion with Google Model Explorer version.
|
|
||
| ## Prepare artifacts from your ExecuTorch deployment | ||
|
|
||
| Use this workflow when your application runs an ExecuTorch program and you already have its profiling artifacts. MLIA reads the captured data and reports measured performance. The earlier LiteRT conversion and Scenario Runner steps don't generate these artifacts. |
There was a problem hiding this comment.
It might be helpful to emphasise that the profiling data is stored in an ETDump file. For example:
"Use this workflow when your application runs an ExecuTorch program and you have already captured its profiling data in an ETDump file."
| | `run.etdump` or `run.etdp` | One matching runtime capture, including VGF profiling when enabled | | ||
| | `model.pte` (optional) | The deployed program containing a VGF delegate, for companion graph associations | | ||
|
|
||
| For artifact generation, see the [ExecuTorch 1.5 profiling tutorial](https://docs.pytorch.org/executorch/1.5/tutorials/devtools-integration-tutorial.html). VGF cycle analysis additionally requires VGF profiling data in the ETDump. An ordinary runtime-only capture can provide source timings, but it can't provide VGF counters or companion VGF associations. |
There was a problem hiding this comment.
It might be worth including these links for VGF-specific guidance:
https://docs.pytorch.org/executorch/stable/backends/arm-vgf/vgf-neural-statistics.html
https://docs.pytorch.org/executorch/1.0/tutorial-arm-vgf.html
the first explains how to enable VGF-specific statistics in ETDump, while the second demonstrates building and running a VGF-delegated model using executor_runner.
|
|
||
| Mixed-backend dumps keep ordinary runtime timings separate from VGF counters. Instructions with VGF profiling are excluded from ordinary runtime aggregation to avoid counting the same work twice. Runtime event counts describe contributing event records, rather than the number of complete inferences. | ||
|
|
||
| MLIA doesn't estimate missing measurements. If instructions or segments cover different executions, model cycle summaries can be unavailable with a warning while individual measurements remain available. Even matching coverage doesn't prove every expected stripe was captured. Review warnings and, for older captures, any assumed statistics mode or device layout. |
There was a problem hiding this comment.
We might clarify the defaults explicitly here:
For older captures missing decoder metadata, MLIA assumes statistics mode 1 and Mali-G2. Check the reported assumptions against your capture settings.
|
|
||
| ### View measured cycle counts | ||
|
|
||
| The command opens Model Explorer with the capture's results already loaded. In the generated MLIA hierarchy, select a cycle metric using the node-data controls. Expand the hierarchy and select an entity to inspect its reported cycle count. Follow the source mapping, where available, to see related operations in **Source Model**. |
There was a problem hiding this comment.
We might add some clarifying that a source mapping associates operations with measured execution groups; it does not necessarily provide an independent cycle measurement for each operation.
No description provided.