Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added Llama 3 8B support #156

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Added Llama 3 8B support #156

wants to merge 1 commit into from
+19 −4

Conversation

Ualas
Copy link

@Ualas Ualas commented Apr 23, 2024

This pull request introduces support for Llama 3 8B model with GGUF format. The high-level changes are:

  1. Updated llama-cpp-python to v0.2.63. The previous version was causing the API to crash.
  2. Changed DEFAULT_MODEL to Llama 3 8B.
  3. Added API and UI compatibility to the Llama 3 8B model.

@Ualas Ualas mentioned this pull request Apr 23, 2024
Closed
@gyzerok
Copy link

gyzerok commented Apr 26, 2024

@Ualas would it make sense to add 70b as well?

derek-palmer
derek-palmer reviewed May 9, 2024
View reviewed changes
run.sh
Copy link

@derek-palmer derek-palmer May 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend updating the echo to include llama3-8b with the intro of the model to reflect the current availability of models. :)

derek-palmer@f64fc2f

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@derek-palmer derek-palmer derek-palmer left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Ualas @gyzerok @derek-palmer