slow forward processing in Pi4

[Hamidreza-Ramezani]

Hello, I hope you don't mind as my question is a bit unrelated about this repo.

I'm trying to use PyTorch to do some ML training tasks on Raspberry pi4. Specifically, I'm trying to do an inference task using MobileNet model. I realized that the forward processing step is done very slow in Pi4 as it doesn't have any GPU. I was wondering if you possibly have any suggestions to increase the performance?

Cheers,

[Qengineering]

@Hamidreza-Ramezani,

Keep in mind that PyTorch, just like TensorFlow, is a massive framework targeted at scientists and scholars. You can modify your ideas easily due to the Python language.
On the other side, you have dedicated frameworks for edge devices like ARM-based phones, Raspberry Pi and other single board computers. Another point is Python; not the fastest of languages. Better to use C++.

If you want performance, use one of those frameworks, like ncnn. Very lightweight, very fast, supports most common models. It can also import onnx. Intentional using C++, but recently a Python port is available. There are other frameworks, like TNN, MNN. All more or less equal to ncnn. All mentioned frameworks don't support training.

When it comes to training a network, the only way is to use cloud solutions like Google Colab. Your RPi will never have the capacity. Even a Jetson Nano with CUDA support will not be capable of training a network due to its limited memory.

[Hamidreza-Ramezani]

Thank you very much for the detailed answer.