Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: sync llama cpp & support use jinja template / tool_calls #117

Merged
merged 30 commits into from
Feb 3, 2025
Merged

feat: sync llama cpp & support use jinja template / tool_calls #117

merged 30 commits into from
Feb 3, 2025

Conversation

jhen0409
Copy link
Member

@jhen0409 jhen0409 commented Jan 25, 2025 •
edited
Loading

Sync to current latest commit ggml-org/llama.cpp@6eecde3, for:

  • Jinja template & universal tool call support
  • Metal: Use residency sets (included in changes)

TODO:

  • getFormattedChat
    • add jinja param (bool, default to false)
    • Cached templates in llama_rn_context (the function is slow)
  • completion param
    • add jinja param (bool, default to false)
    • add tools param
  • context: add more info
    • model supported tool_use
    • model chat_template details (take from minja::chat_template)
  • Android: error handling

jhen0409 added 24 commits January 25, 2025 15:45
@jhen0409
feat: sync llama.cpp
87ee079
@jhen0409
fix(cpp): migrate chat_template api
cc3feb4
@jhen0409
Merge branch 'main' into sync-llama-cpp
51c9641
@jhen0409
fix: build libs
7365028
@jhen0409
Merge branch 'main' into sync-llama-cpp
7262e40
@jhen0409
chore: revert unnecessary change
27d9ce9
@jhen0409
fix: build libs
17ecfd3
@jhen0409
fix: add patches
8e22f9b
@jhen0409
Merge branch 'main' into sync-llama-cpp
b91267b
@jhen0409
feat: add jinja option in getFormattedChat
22ab91a
@jhen0409
feat: add jinja & tools params
54014df
@jhen0409
feat(cpp): assign templates to llama_rn_context
f237b63
@jhen0409
fix(ios): remove unnecessary import
20ccb9f
@jhen0409
feat: sync llama.cpp
ce9ca88
@jhen0409
feat(ios): implement tool_calls in completion
3cd7b6a
@jhen0409
fix(ts): grammar_lazy
807cda5
@jhen0409
fix: cmake build
a47e7c2
@jhen0409
feat: sync llama.cpp
57bdfa0
@jhen0409
feat(android): implement tool_calls in completion
e3fc89b
@jhen0409
feat(android): apply tool_calls params in completion
1d975e7
@jhen0409
feat: sync llama.cpp
1c54e23
@jhen0409
feat(android): error handling in getFormattedChatWithJinja
8e9ce7c
@jhen0409
fix: add missing preserved_tokens
45314db
@jhen0409
feat: add model info & use tool_use tmpl if tools are provided
8704a6a
@jhen0409 jhen0409 marked this pull request as ready for review February 3, 2025 05:46
@jhen0409 jhen0409 changed the title feat: sync llama cpp & support use jinja template feat: sync llama cpp & support use jinja template / tool_calls Feb 3, 2025
@jhen0409 jhen0409 merged commit dd2179e into main Feb 3, 2025
6 checks passed
@jhen0409 jhen0409 deleted the sync-llama-cpp branch February 3, 2025 06:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@jhen0409