Skip to content

Commit

Permalink
server: tests: parallel fix server is started twice, add colors to he…
Browse files Browse the repository at this point in the history
…lp to monitor in the CI jobs
  • Loading branch information
phymbert committed Feb 23, 2024
1 parent 530d3ae commit 36ddb96
Show file tree
Hide file tree
Showing 3 changed files with 3 additions and 9 deletions.
1 change: 1 addition & 0 deletions examples/server/tests/features/environment.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@


def before_scenario(context, scenario):
print(f"\x1b[33;42mStarting new scenario: {scenario.name}!\x1b[0m")
port = 8080
if 'PORT' in os.environ:
port = int(os.environ['PORT'])
Expand Down
9 changes: 1 addition & 8 deletions examples/server/tests/features/parallel.feature
Original file line number Diff line number Diff line change
Expand Up @@ -50,17 +50,10 @@ Feature: Parallel
Then all prompts are predicted with <n_predict> tokens
Examples:
| streaming | n_predict |
| disabled | 64 |
| disabled | 128 |
#| enabled | 64 | FIXME: phymbert: need to investigate why in aiohttp with streaming only one token is generated

Scenario: Multi users with total number of tokens to predict exceeds the KV Cache size #3969
Given a server listening on localhost:8080
And a model file stories260K.gguf
And 42 as server seed
And 2 slots
And 64 KV cache size
Then the server is starting
Then the server is healthy
Given a prompt:
"""
Write a very long story about AI.
Expand Down
2 changes: 1 addition & 1 deletion examples/server/tests/features/steps/steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ def step_start_server(context):
with closing(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) as sock:
result = sock.connect_ex((context.server_fqdn, context.server_port))
if result == 0:
print("\x1b[33;42mserver started!\x1b[0m")
print("\x1b[33;46mserver started!\x1b[0m")
return
attempts += 1
if attempts > 20:
Expand Down

0 comments on commit 36ddb96

Please sign in to comment.