benchmark.md

Benchmarking

1. Introduction
2. Benchmark Support Matrix
3. Get Started with Benchmark
4. Examples

Introduction

The benchmarking feature of Neural Compressor is used to measure the model performance with the objective settings. Users can get the performance of the float32 model and the optimized low precision model in the same scenarios.

Benchmark Support Matrix

Environment	Category
Operating System	linux
	windows
Architecture	x86_64
	aarch64
	gpu

Get Started with Benchmark API

Benchmark provide capability to automatically run with multiple instance through cores_per_instance and num_of_instance config (CPU only). And please make sure cores_per_instance * num_of_instance must be less than CPU physical core numbers. benchmark.fit accept b_dataloader or b_func as input. b_func is customized benchmark function. If user passes the b_dataloader, then b_func is not required.

from neural_compressor.config import BenchmarkConfig
from neural_compressor.benchmark import fit
conf = BenchmarkConfig(warmup=10, iteration=100, cores_per_instance=4, num_of_instance=7)
fit(model='./int8.pb', config=conf, b_dataloader=eval_dataloader)

Examples

Refer to the Benchmark example.