Skip to content
This repository has been archived by the owner on Mar 1, 2023. It is now read-only.

Graph API

Jump to bottom
Dmitry Matveev edited this page Mar 25, 2021 · 47 revisions

What it is?

  • OpenCV 4.0 comes with an experimental Graph API module (see opencv/modules/gapi). This is a new API which allows to enable offload and optimizations for image processing / CV algorithms on pipeline level.

  • The idea behind G-API is to declare image processing task in form of expressions and then submit it for execution – using a number of available backends. At the moment, there’s reference “CPU” (OpenCV-based), "GPU" (also OpenCV T-API-based), and experimental “Fluid’ backends available, with other backends coming up next.

  • G-API is an uncommon OpenCV module since it acts as a framework: it provides means for declaring operations, building graphs of operations, and finally implementing the operations for a particular backend. G-API model enforces separation between interfaces and implementations, so once an algorithm is expressed in G-API terms, it can be scaled/ported/offloaded to a new platform easily.

  • G-API CPU (OpenCV) backend implements G-API standard functions using OpenCV itself (core/imgproc modules) and acts as a quick prototyping/porting/testing backend. If you have an image processing algorithm composed of OpenCV-like functions already, you would be able to switch quickly to G-API by using this backend.

  • G-API Fluid backend implements a cache-efficient execution model and allows to save memory dramatically – e.g. a 1.5GB image processing pipeline fits into 750KB memory footprint with G-API/Fluid. G-API comes with a number of operations implemented for Fluid backend, so one can switch OpenCV/Fluid operations within a graph easily and even mix both in the same graph.

  • G-API GPU backend implements the majority of available functions and allows to run OpenCL kernels on available OpenCL-programmable devices. At the moment, GPU backend is based on OpenCV Transparent API; in future versions it will be extended to support integration of arbitrary OpenCL kernels (and likely be renamed to "OpenCL backend").

  • G-API ONNX backend implements ONNX models inference operations on input data and outputs the results. At the moment, ONNX backend is based on ONNX Runtime C/C++ API.

Building G-API

G-API is built with OpenCV by default, however some features may require additional options or dependencies enabled.

How to build with OpenVINO Toolkit:

TBD

How to build with PlaidML:

TBD

How to build with ONNX RT:

* Build and install the ONNX RT (currently tested with v1.5.1):
```bash
 $ git clone --recursive https://github.com/microsoft/onnxruntime.git
 $ cd onnxruntime
 $ git checkout v1.5.1
 $ git submodule update --init
 $ ./build.sh --config Release --build_shared_lib --parallel \
 $     --cmake_extra_defines CMAKE_INSTALL_PREFIX=install
 $ cd build/Linux/Release
 $ make install
```
* Then specify extra options to OpenCV CMake:    
```bash
 $ cmake /path-to-opencv -DWITH_ONNX=ON -DORT_INSTALL_DIR=/path-to-ort-install-dir
```

Testing G-API

By default, the OpenCV G-API comes with its own test suite (opencv_test_gapi). Note that extra (external) G-API modules may introduce their own test suites. G-API tests are built and run in a regular way:

Linux

$ make -j4 opencv_test_gapi
$ bin/opencv_test_gapi

Windows

$ cmake --build . --target opencv_test_gapi --config Release -- /maxcpucount:4
$ bin\Release\opencv_test_gapi.exe

A tiny fraction of G-API tests requires external test data to be available. This data is taken from the opencv_extra repo:

export OPENCV_TEST_DATA_PATH=/path/to/opencv_extra/testdata

With OpenVINO Inference Engine

When you build G-API with OpenVINO Inference Engine support (-DInferenceEngine_DIR=... -DWITH_INF_ENGINE=ON), some extra tests for inference are enabled and require OPENCV_DNN_TEST_DATA_PATH to be set and models downloaded using the command below!

export OPENCV_DNN_TEST_DATA_PATH=/path/to/opencv_extra/testdata/dnn
openvino$ ./tools/downloader/downloader.py -o ${OPENCV_DNN_TEST_DATA_PATH}/omz_intel_models/2020.3.0 \
    --cache_dir ${OPENCV_DNN_TEST_DATA_PATH}/.omz_cache/ \
    --name age-gender-recognition-retail-0013

With ONNX Runtime

When you build G-API with ONNX Runtime support, tests for inference are enabled and require OPENCV_GAPI_ONNX_MODEL_PATH to be set:

$ export OPENCV_GAPI_ONNX_MODEL_PATH=/path-to/onnx-models/

and models downloaded using the commands:

$ git clone --recursive https://github.com/onnx/models.git
$ cd models
$ git lfs pull --include=path-to-desired-onnx-model --exclude=""

Submitting G-API PRs

G-API supports so-called STANDALONE mode build which is not validated by default. Please put the below lines to the PR description to validate that this mode is not broken by the PR:

force_builders=Custom,Custom Win,Custom Mac
build_gapi_standalone:Linux x64=ade-0.1.1f
build_gapi_standalone:Win64=ade-0.1.1f
build_gapi_standalone:Mac=ade-0.1.1f
build_gapi_standalone:Linux x64 Debug=ade-0.1.1f

Xbuild_image:Custom=centos:7
Xbuildworker:Custom=linux-1
build_gapi_standalone:Custom=ade-0.1.1f

build_image:Custom=ubuntu-openvino-2020.3.0:16.04
build_image:Custom Win=openvino-2020.3.0
build_image:Custom Mac=openvino-2020.3.0

test_modules:Custom=gapi
test_modules:Custom Win=gapi
test_modules:Custom Mac=gapi

buildworker:Custom=linux-1
# disabled due high memory usage: test_opencl:Custom=ON
test_opencl:Custom=OFF
test_bigdata:Custom=1
test_filter:Custom=*

Notes:

  • ADE version may change, refer to the latest correct one (see DownloadADE.cmake).

G-API build depends on a number of external components which may not be built by default, e.g. PlaidML:

force_builders=Custom
buildworker:Custom=linux-1
build_image:Custom=plaidml2
test_modules:Custom=gapi
test_filter:Custom=*ML*

or ONNX Runtime:

force_builders=Custom
buildworker:Custom=linux-1
build_image:Custom=ubuntu-onnx:20.04
test_modules:Custom=gapi
test_filter:Custom=*ONNX*

It is also worth testing if any internal API changes are made.

Materials

© Copyright 2022, OpenCV team

Clone this wiki locally