Skip to content

Commit

Permalink
squash! llama : add early return for empty range
Browse files Browse the repository at this point in the history
Remove the setting of cache.head to 0 when the range is empty.

Signed-off-by: Daniel Bevenius <[email protected]>
  • Loading branch information
danbev committed Jul 5, 2024
1 parent 4eb8073 commit eb572f9
Showing 1 changed file with 1 addition and 4 deletions.
5 changes: 1 addition & 4 deletions src/llama.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3259,10 +3259,7 @@ static void llama_kv_cache_seq_add(
if (p0 < 0) p0 = 0;
if (p1 < 0) p1 = std::numeric_limits<llama_pos>::max();
// If there is no range then return early to avoid looping over the cache.
if (p0 == p1) {
cache.head = 0;
return;
}
if (p0 == p1) return;

if (cache.recurrent) {
// for Mamba-like models, only the pos needs to be shifted
Expand Down

0 comments on commit eb572f9

Please sign in to comment.