Add JAIS model(s)
#8118
Add JAIS model(s)
#8118
Conversation
convert-hf-to-gguf.py
Outdated
| @@ -427,9 +427,6 @@ def get_vocab_base_pre(self, tokenizer) -> str: | |||
| # NOTE: if you get an error here, you need to update the convert-hf-to-gguf-update.py script | |||
| # or pull the latest version of the model from Huggingface | |||
| # don't edit the hashes manually! | |||
| if chkhsh == "0ef9807a4087ebef797fc749390439009c3b9eda9ad1a097abbe738f486c01e5": | |||
There was a problem hiding this comment.
Not really sure why this happened...
I ran convert-hf-to-gguf-update.py as instructed
There was a problem hiding this comment.
Most likely you don't have access to the HF repos
convert-hf-to-gguf.py
Outdated
|
|
||
| return tensors | ||
|
|
||
|
|
examples/main/main.cpp
Outdated
| @@ -733,7 +733,6 @@ int main(int argc, char ** argv) { | |||
|
|
|||
| // Console/Stream Output | |||
| fprintf(stdout, "%s", token_str.c_str()); | |||
|
|
|||
ggml.c
Outdated
| @@ -13516,13 +13516,13 @@ static void ggml_compute_forward_soft_max_f32( | |||
| } else { | |||
| for (int i = 0; i < nc; ++i) { | |||
| wp[i] += slope*mp_f32[i]; | |||
|
|
|||
There was a problem hiding this comment.
revert changes in this file
llama.cpp
Outdated
| @@ -6700,7 +6733,6 @@ static bool llm_load_tensors( | |||
| case LLM_ARCH_BITNET: | |||
| { | |||
| model.tok_embd = ml.create_tensor(ctx_input, tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}); | |||
|
|
|||
mofosyne
added
the
Review Complexity : Medium
fmz
force-pushed
the
jais
branch
2 times, most recently
from
22a6497 to
1a51b36
Compare
|
@ggerganov @slaren Thanks in advance! |
|
@slaren @ggerganov |
|
Here is some sample output from Jais-13-chat (ignoring proper system-prompting): Instruction: Translate قهوة into English.What is the English translation for the word قهوة؟ Pronunciation: /kah-wah/ Defintion: A dark, bitter fluid produced by the roasted seeds of several species of shrub of the genus Coffea. [end of text] Instruction: جاوب باللغة الانجليزية: ما هو مشروبك المفضل؟إجابة: My favorite drink is tea. [end of text] |
src/llama.cpp
Outdated
| // TODO: become GGUF KV parameter | ||
| hparams.f_max_alibi_bias = 8.0f; |
There was a problem hiding this comment.
There is already a <arch>.attention.max_alibi_bias parameter. I know that this was copied from other archs that still hardcode this parameter, but I don't think we should do this for new archs. Instead, it should be added as metadata in convert-hf-to-gguf.py with gguf_writer.add_max_alibi_bias.
There was a problem hiding this comment.
Ah perfect
I'll update it shortly
|
@slaren @ggerganov Let me know if you have any more comments on this. If not, can you please merge it when you get the chance (I don't have permission)? |
* Add `JAIS` model(s) * cleanup * address review comments * remove hack * un-hardcode max-alibi-bias * minor tweaks --------- Co-authored-by: fmz <[email protected]>
* Add `JAIS` model(s) * cleanup * address review comments * remove hack * un-hardcode max-alibi-bias * minor tweaks --------- Co-authored-by: fmz <[email protected]>
Add support for Jais and Jais-chat: a new state-of-the-art Arabic-centric foundation and instruction-tuned open generative large language models. https://arxiv.org/abs/2308.16149
The model is essentially GPT2 with some modifications:
For this PR I only added support for 13B and 1.3B Jais