This document explains how to add a new operator into Tengine.
To add a new operator, you should:
-
Register the new operator in the directory Tengine/operator/.
Operator schema defines an interface for the operator's functionality, which is independent of the operator's implemenation. The operator schema defines:
- op name
- op inputs and outputs
- op parameters
- infershape function: used for tensor shape inference in
Prerun
stage.
-
Implement the operator in the directory Tengine/executor/.
This is the concrete implementation for operator. There can be multiple implementations for different parameters (for example, conv1x1, conv3x3) or architectures (for example, armv8/armv7).
In this section, we use Scale
operator as our example to show how to add it in Tengine step by step.
There are two type of operators:
- Operator without parameters:
template <typename T> OperatorNoParam
- Operator with parameters:
template <typename T, typename P> OperatorWithParam
The scale operator is an operator with parameters.
- Define
Scale
class in file operator/include/operator/scale.hpp:class Scale: public OperatorWithParam<Scale,ScaleParam> { public: Scale() { name_="Scale"; } Scale(const Scale&)= default; ~Scale() {} void SetSchema(void) override; };
- Define
ScaleParam
in file operator/include/operator/scale_param.hpp:struct ScaleParam { int axis; int num_axes; int bias_term; DECLARE_PARSER_STRUCTURE(ScaleParam) { DECLARE_PARSER_ENTRY(axis); DECLARE_PARSER_ENTRY(num_axes); DECLARE_PARSER_ENTRY(bias_term); }; };
-
Set operator schema in file operator/operator/scale.cpp:
void Scale::SetSchema(void) { Input({"input:float32","gamma:float32","bias:float32"}) .Output({"output:float32"}) .SetAttr("axis",1) .SetAttr("num_axes",1) .SetAttr("bias_term",0) .SetDoc(R"DOC(Scale: only caffe flavor scale)DOC"); }
-
Add
obj-y+=scale.o
in file operator/operator/Makefile.
- Add register scale operator in file operator/plugin/init.cpp:
RegisterOp<Scale>("Scale");
-
Add the implementation of scale operator in file executor/operator/common/scale.cpp and register the implemetation in the function
RegisterScaleNodeExec
:namespace ScaleImpl { struct ScaleOps: public NodeOps { bool Run(Node * node) { // your implementation } }; } using namespace ScaleImpl; void RegisterScaleNodeExec(void) { ScaleOps * ops=new ScaleOps(); NodeOpsRegistryManager::RegisterOPImplementor("common", "Scale",ops); }
-
Add
obj-y+=scale.o
in file executor/operator/common/Makefile.
- Add
RegisterScale_NodeExec
in file executor/plugin/init.cpp:extern void RegisterScale_NodeExec(void); RegisterScale_NodeExec();
If you want to test your operator implementation, you can add test in file executor/tests/test_scale.cpp.
Tengine also support dynamic shape for operators. Operators that need to dynamic shape are RPN
in faster_rcnn and detection_output
in SSD. The following will explain how to implement this method.
-
Tell the network from which operator, the shape will be computed dynamically. Add
SetOperatorDynamicShape(op);
in your loadcaffe function
LoadCaffeDetectionOutput()
/LoadCaffeRPN()
in file serializer/caffe/caffe_serializer.cpp . -
Add DynamicProcess and do infer-shape in Run function in file executor/operator/common/rpn.cpp:
bool Run(Node *node) { Tensor *output_tensor = node->GetOutputTensor(0); TShape &out_shape = output_tensor->GetShape(); // dynamic compute num_box // set the output shape here std::vector<int> outdim={1,num_box,4,1}; out_shape.SetDim(outdim); }