Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Use a custom batch size threshold for using a batch-specific kernel.
Browse files Browse the repository at this point in the history
MrMage committed Nov 6, 2024
1 parent 419aa38 commit 98b05f3
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion ggml/src/ggml-metal.m
Original file line number Diff line number Diff line change
@@ -1816,7 +1816,7 @@ static void ggml_metal_encode_node(

// find the break-even point where the matrix-matrix kernel becomes more efficient compared
// to the matrix-vector kernel
int ne11_mm_min = 1;
int ne11_mm_min = 6;

#if 0
// the numbers below are measured on M2 Ultra for 7B and 13B models

0 comments on commit 98b05f3

Please sign in to comment.