Skip to content

Commit

Permalink
server : simplify state machine for slot (ggerganov#9283)
Browse files Browse the repository at this point in the history 
* server : simplify state machine for slot

* add SLOT_STATE_DONE_PROMPT

* pop_deferred_task

* add missing notify_one

* fix passkey test

* metrics : add n_busy_slots_per_decode

* fix test step

* add test

* maybe fix AddressSanitizer?

* fix deque ?

* missing lock

* pop_deferred_task: also notify

* Update examples/server/server.cpp

Co-authored-by: Georgi Gerganov <[email protected]>

---------

Co-authored-by: Georgi Gerganov <[email protected]>
  • Loading branch information
@ngxson @ggerganov @arthw
2 people authored and arthw committed Nov 18, 2024
1 parent ef27803 commit 00aff35
Show file tree
Hide file tree
Showing 4 changed files with 147 additions and 93 deletions.
Loading

0 comments on commit 00aff35

Please sign in to comment.