Export YOLOv5 to run on Raspberry Pi

Export processing consists of two steps:
1️⃣ Convert Pytorch model weights to MNN model weights.
2️⃣ Run the inference on Raspberry Pi.
You can compare inference time of YOLOv5 model on two frameworks MNN and Pytorch in my Google Colab (👍👍👍Open and run on Google Chrome recommended).

Documentation

ℹ️ MNN is a lightweight deep neural network inference engine.
🔎 You can find more information about MNN in here.
🔎 MNN github repository in here.

Requirements

👍 python>=3.6 is required.

Installation

Step 1: Convert Pytorch model weights to MNN model weights

If you don't want to install anything on your system then use this Google Colab (Recommended). (👍👍👍Open and run on Google Chrome recommended).

And if you want to perform the conversion on your system then follow bellow instructions:

📣 I recommend to create a new conda environment (python version 3.6 recommended):

$ conda create -n yolov5_conversion python=3.6
$ conda activate yolov5_conversion

➡️ Then run below commands and replace yolov5s.pt with your own model weights in path weights/pt/ and also change yolov5s.yaml in path models/ accordingly.

$ git clone https://github.com/AnhPC03/yolov5.git
$ cd yolov5
$ pip install -r requirements.txt
$ bash export_mnn.sh yolov5s 640

✅ With yolov5s is model name and 640 is input size of your Pytorch model.
✅ After you run above commands, you will see successfully message. And you can find MNN converted model in path weights/mnn/.
✅ Attention about model quantization: The size of MNN model weight is much smaller than origin Pytorch model weight because of using --weightQuantBits 8 in export_mnn.sh file. This reduced model size while also improving CPU and hardware accelerator latency, with little degradation in model accuracy. If you want to keep fully model accuracy, feel free to delete --weightQuantBits 8 in export_mnn.sh file.
✅ Because of running on Raspberry Pi, so I decided to quantize my model.

Step 2: Run the inference on Raspberry Pi

⚙️ Setup

📣 If you don't want to install virtual environment into your Raspberry Pi, feel free to skip these below commands.
Otherwise I recommend you creat a virtual environment (python version>=3 is required):

$ sudo pip install virtualenv
$ virtualenv mnn_raspberry_pi
$ source mnn_raspberry_pi/bin/activate

➡️ Follow below steps to install minimum required environment for converting Pytorch to MNN model weight file:

$ git clone https://github.com/AnhPC03/yolov5-export-to-cpu-mnn.git
$ cd yolov5-export-to-cpu-mnn
$ pip3 install -r requirements.txt

✅ Replace content of classes.txt file with your classes when you trained your Pytorch model.

🎉 Run inference
🍻 Result will be saved to results/ folder

$ python3 inference/run_mnn_detector.py \
            --weights path/to/your/mnn/weight \
            --source 0  # webcam
                     file.jpg  # image 
                     file.mp4  # video
                     path/  # directory
                     'https://youtu.be/NUsoVlDFqZg'  # YouTube video
                     'rtsp://example.com/media.mp4'  # RTSP, RTMP, HTTP stream

✅ With path/to/your/mnn/weight is path to MNN model weight which you just converted in the above step.
✅ You can use --nodisplay to do not display image or webcam while inference. Or --nosave to do not save inference results.
✅ For example: my MNN model weight in yolov5-export-to-cpu-mnn/weights/yolov5s.mnn. Then I run inference on images in inference/images/ folder as below:

$ python3 inference/run_mnn_detector.py --weights weights/yolov5s.mnn --source inference/images