Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ONNX support for Embeddings and Pipelines #109

Closed
davidmezzetti opened this issue Aug 27, 2021 · 3 comments
Closed

Add ONNX support for Embeddings and Pipelines #109

davidmezzetti opened this issue Aug 27, 2021 · 3 comments
Assignees
Milestone

Comments

@davidmezzetti
Copy link
Member

davidmezzetti commented Aug 27, 2021

Add ONNX support for Embeddings and Pipelines.

Sequence to Sequence models (summarization, transcription, translation) will be added later once ONNX support for encoder-decoder models is more mature.

@davidmezzetti davidmezzetti self-assigned this Aug 27, 2021
@davidmezzetti davidmezzetti added this to the v3.3.0 milestone Aug 27, 2021
@davidmezzetti davidmezzetti changed the title Add support for vectors from ONNX model Add ONNX support to Embeddings and Pipelines Aug 31, 2021
@davidmezzetti davidmezzetti changed the title Add ONNX support to Embeddings and Pipelines Add ONNX support for Embeddings and Pipelines Aug 31, 2021
@nickchomey
Copy link

The HF documentation says that you can export seq2seq to ONNX with the OnnxSeq2SeqConfigWithPast class.
https://huggingface.co/docs/transformers/v4.23.1/en/main_classes/onnx#onnx-configurations

This was added with this PR in March huggingface/transformers#14700

Perhaps it is sufficient to be incorporated into txtai now? It would be great to be able to use ONNX versions of the various HF models, for their increased performance.

@davidmezzetti
Copy link
Member Author

Haven't looked at this for quite a while, so it's possible. Would you mind opening a new issue?

@nickchomey
Copy link

#371 has been opened to track adding seq2seq export to ONNX

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants