New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Port BartTokenizer #1

Open

budgetdevv opened this issue Sep 13, 2024 · 2 comments

Assignees

Labels

Owner

budgetdevv commented Sep 13, 2024

Port BartTokenizer over to C#. Currently it relies on onnx inference, and it is running the slow variant

budgetdevv added the Performance label

budgetdevv self-assigned this

Owner Author

budgetdevv commented Sep 15, 2024

Pretty low priority considering the built-in prompts can be tokenized once and cached, and we only decode once per inference.

Owner Author

budgetdevv commented Sep 20, 2024

Will be using https://github.com/budgetdevv/Tokenizers.NET instead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment