Skip to content

Commit

Permalink
chore: update pagedattention.cu (#822)
Browse files Browse the repository at this point in the history
minor fix
  • Loading branch information
eltociear authored Oct 3, 2024
1 parent 329e0e8 commit 9365c76
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions mistralrs-paged-attn/src/pagedattention.cu
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ __device__ void paged_attention_kernel(

// Load the query to registers.
// Each thread in a thread group has a different part of the query.
// For example, if the the thread group size is 4, then the first thread in the group
// For example, if the thread group size is 4, then the first thread in the group
// has 0, 4, 8, ... th vectors of the query, and the second thread has 1, 5, 9, ...
// th vectors of the query, and so on.
// NOTE(woosuk): Because q is split from a qkv tensor, it may not be contiguous.
Expand Down Expand Up @@ -205,7 +205,7 @@ __device__ void paged_attention_kernel(

// Load a key to registers.
// Each thread in a thread group has a different part of the key.
// For example, if the the thread group size is 4, then the first thread in the group
// For example, if the thread group size is 4, then the first thread in the group
// has 0, 4, 8, ... th vectors of the key, and the second thread has 1, 5, 9, ... th
// vectors of the key, and so on.
for (int i = 0; i < NUM_TOKENS_PER_THREAD_GROUP; i++) {
Expand Down

0 comments on commit 9365c76

Please sign in to comment.