Support vllm for openai api, refactor openai non-streaming api (#86) #33
Job | Run time |
---|---|
1m 41s | |
50m 46s | |
43m 23s | |
13m 39s | |
38m 25s | |
28s | |
2m 28s | |
2m 29s | |
1m 20s | |
3m 58s | |
2h 38m 37s |
Job | Run time |
---|---|
1m 41s | |
50m 46s | |
43m 23s | |
13m 39s | |
38m 25s | |
28s | |
2m 28s | |
2m 29s | |
1m 20s | |
3m 58s | |
2h 38m 37s |