Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Add support for Lite-Mistral-Instruct chat template #8529

Closed
4 tasks done
AmgadHasan opened this issue Jul 17, 2024 · 2 comments
Closed
4 tasks done
Labels
enhancement New feature or request stale

Comments

@AmgadHasan
Copy link

Prerequisites

  • I am running the latest code. Mention the version if possible as well.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

OuteAI has released a new small model that is very coherent for its size.

I am requesting the addition of this model's chat template to llama.cpp's list of supported templates

Motivation

The model is already supported by llama.cpp. However, it's using a new chat template that isn't in the list of supported templates. As a result, llama.cpp assumes ChatML for this model. Due to the model's size, it's very sensitive to the prompt template and gives terrible results with wrong formats and users have terrible experience running it.

Possible Implementation

The model's template is very similar to ChatML so we can just copy-paste the implementation for ChatML and modify it.

@AmgadHasan AmgadHasan added the enhancement New feature or request label Jul 17, 2024
@AmgadHasan
Copy link
Author

I've opened a PR to add support for this template:
#8522

@github-actions github-actions bot added the stale label Aug 17, 2024
Copy link
Contributor

This issue was closed because it has been inactive for 14 days since being marked as stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request stale
Projects
None yet
Development

No branches or pull requests

1 participant