[api] Let single handle flexible filter #574
Conversation
- Add a enum for new nnfw `EXECUTORCH_LLAMA` Signed-off-by: Yongjoo Ahn <[email protected]>
anyj0527
requested review from
jaeyun-jung,
again4you,
myungjoo,
gichan-jang,
songgot and
jijoongmoon
as code owners
|
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #574. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
- Let single API handle flexible filter (executorch-llama) Signed-off-by: Yongjoo Ahn <[email protected]>
anyj0527
force-pushed
the
single-llama
branch
from
05378f1 to
83c9879
Compare
- Add a simple test to show how to use executorch-llama with single api Signed-off-by: Yongjoo Ahn <[email protected]>
anyj0527
force-pushed
the
single-llama
branch
from
83c9879 to
ed9a143
Compare
| ml_tensors_info_set_tensor_type (in_info, 0, ML_TENSOR_TYPE_UINT8); | ||
| ml_tensors_info_set_tensor_dimension (in_info, 0, dim); | ||
|
|
||
| ml_tensors_data_create (in_info, &in_data); |
There was a problem hiding this comment.
How about checking if in_data is created properly?
status = ml_tensors_data_create (in_info, &in_data);
EXPECT_EQ (status, ML_ERROR_NONE);
| ml_tensors_info_set_tensor_dimension (in_info, 0, dim); | ||
|
|
||
| ml_tensors_data_create (in_info, &in_data); | ||
| ml_tensors_data_set_tensor_data (in_data, 0, prompt.c_str (), prompt.size () + 1); |
There was a problem hiding this comment.
Let's check the result, how about you?
EXPECT_EQ (status, ML_ERROR_NONE);
| @@ -1083,6 +1090,11 @@ ml_single_open_custom (ml_single_h * single, ml_single_preset * info) | |||
| status = ML_ERROR_STREAMS_PIPE; | |||
| goto error; | |||
| } | |||
| /* handle flexible single */ | |||
| if (info->nnfw == ML_NNFW_TYPE_EXECUTORCH_LLAMA) { | |||
There was a problem hiding this comment.
I think if there are more flexible filter, we may revise this part later.
The llama2c filter I created doesn't seem to be scalable due to API limitations, so I won't add it now.
| @@ -75,6 +75,7 @@ typedef enum { | |||
| ML_NNFW_TYPE_NCNN = 18, /**< Tencent ncnn (Since 9.0) */ | |||
| ML_NNFW_TYPE_TENSORRT = 19, /**< NVidia Tensor-RT (Since 9.0) */ | |||
| ML_NNFW_TYPE_QNN = 20, /**< Qualcomm QNN (Qualcomm® AI Engine Direct) (Since 9.0) */ | |||
| ML_NNFW_TYPE_EXECUTORCH_LLAMA = 21, /**< ExecuTorch Llama runner */ | |||
| @@ -75,6 +75,7 @@ typedef enum { | |||
| ML_NNFW_TYPE_NCNN = 18, /**< Tencent ncnn (Since 9.0) */ | |||
| ML_NNFW_TYPE_TENSORRT = 19, /**< NVidia Tensor-RT (Since 9.0) */ | |||
| ML_NNFW_TYPE_QNN = 20, /**< Qualcomm QNN (Qualcomm® AI Engine Direct) (Since 9.0) */ | |||
| ML_NNFW_TYPE_EXECUTORCH_LLAMA = 21, /**< ExecuTorch Llama runner */ | |||
There was a problem hiding this comment.
Can you separate new PR and add github label for next ACR?
|
Please do not merge this until tizen gbm release is done! |
EXECUTORCH_LLAMA