Skip to content

Files

Latest commit

ca6ba2e · Oct 12, 2022

History

History
This branch is 1 commit ahead of, 62 commits behind meituan/YOLOv6:main.

partial_quantization

Partial Quantization

The performance of YOLOv6s heavily degrades from 42.4% to 35.6% after traditional PTQ, which is unacceptable. To resolve this issue, we propose partial quantization. First we analyze the quantization sensitivity of all layers, and then we let the most sensitive layers to have full precision as a compromise.

With partial quantization, we finally reach 42.1%, only 0.3% loss in accuracy, while the throughput of the partially quantized model is about 1.56 times that of the FP16 model at a batch size of 32. This method achieves a nice tradeoff between accuracy and throughput.

Prerequirements

pip install --extra-index-url=https://pypi.ngc.nvidia.com --trusted-host pypi.ngc.nvidia.com nvidia-pyindex
pip install --extra-index-url=https://pypi.ngc.nvidia.com --trusted-host pypi.ngc.nvidia.com pytorch_quantization

Sensitivity analysis

Please use the following command to perform sensitivity analysis. Since we randomly sample 128 images from train dataset each time, the sensitivity files will be slightly different.

 python3 sensitivity_analyse.py --weights yolov6s_reopt.pt \
                                --batch-size 32 \
                                --batch-number 4 \
                                --data-root train_data_path

Partial quantization

With the sensitivity file at hand, we then proceed with partial quantization as follows.

python3 partial_quant.py --weights yolov6s_reopt.pt \
                         --calib-weights yolov6s_repot_calib.pt \
                         --sensitivity-file yolov6s_reopt_sensivitiy_128_calib.txt \
                         --quant-boundary 55 \
                         --export-batch-size 1

Deployment

Build a TRT engine

trtexec --workspace=1024 --percentile=99 --streams=1 --int8 --fp16 --avgRuns=10 --onnx=yolov6s_reopt_partial_bs1.sim.onnx --saveEngine=yolov6s_reopt_partial_bs1.sim.trt

Performance

Model Size Precision mAPval
0.5:0.95
SpeedT4
trt b1
(fps)
SpeedT4
trt b32
(fps)
[YOLOv6-s-partial]
bs1
bs32
640 INT8 42.1 503 811
[YOLOv6-s] 640 FP16 42.4 373 520