Skip to content

Commit

Permalink
fix q4_0_8_8 format for corrupted tokens issue (ggerganov#10198)
Browse files Browse the repository at this point in the history
Co-authored-by: EC2 Default User <[email protected]>
  • Loading branch information
snadampal and EC2 Default User authored Nov 7, 2024
1 parent 3bcd40b commit 2319126
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions ggml/src/ggml-cpu.c
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,8 @@ static const struct ggml_type_traits_cpu type_traits_cpu[GGML_TYPE_COUNT] = {
.gemm = ggml_gemm_q4_0_4x8_q8_0,
},
[GGML_TYPE_Q4_0_8_8] = {
.vec_dot = NULL,
.vec_dot_type = GGML_TYPE_Q8_0,
.nrows = 1,
.ncols = 8,
.gemv = ggml_gemv_q4_0_8x8_q8_0,
Expand Down

0 comments on commit 2319126

Please sign in to comment.