README.md

How to use megcc benchmark

introduction

megcc benchmark is a easy tool to get the benchmark result of different model in megcc the file struction is shown below:

├── clean.sh
├── CMakeLists.txt
├── main.cpp
├── model
│   ├── model_arm.json
│   ├── model_riscv.json
│   ├── model_x86.json
│   └── request.txt
├── python
│   ├── example.py
│   ├── format.sh
│   └── src
│       ├── benchmark.py
│       └── models.py
├── README.md
├── src
│   ├── benchmark.h
│   ├── build_config.h.in
│   ├── CCbenchmark.cpp
│   ├── CCbenchmark.h
│   ├── MGEbenchmark.cpp
│   └── MGEbenchmark.h
└── tools
    ├── cc_analysis.py
    └── inference_visual.py

in src, it is a c++ application to run benchmark result on different platform. in python, the model conversion, other related preparing work and the benchmarker example is given the tools contains some usable scripts to analysis benchmark results

supported model

mobilenetv2, resnet18, efficientnetb0 shufflenetv2 vgg16

request

mgeconvert > v.1.0.2 
onnx==1.11.0
torch==1.10.0
# or 
git clone https://github.com/MegEngine/mgeconvert.git
cd mgeconvert 
git checkout master
python3 -m pip install . --user --install-option="--targets=onnx"

the mgeconvert can be install by following command:

git clone https://github.com/MegEngine/mgeconvert.git
cd mgeconvert 
git checkout master
python3 -m pip install . --user --install-option="--targets=onnx"

get model and run benchmark example

cd megcc/benchmark
export MEGCC_MGB_TO_TINYNN_PATH=<your_mgb_to_tinynn_path>
python3  python/example.py

if you want to run in other platform, please reference the example add your new run_platform_xxx function in BenchmarkRunner, the example given a ssh remote device test template

analysis megcc log

the output directory is generated by example.py

visualize the inference result of different model

python3 benchmark/tools/inference_visual.py benchmark/output -o figure_dir

visualize the profile result of different kernel in different model

python3 benchmark/tools/cc_analysis.py benchmark/output -o figure_dir