Skip to content

Commit

Permalink
Fix temperature default in sampling mode
Browse files Browse the repository at this point in the history
  • Loading branch information
njhill authored and joerunde committed Mar 20, 2024
1 parent eda13e7 commit 1479404
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion vllm/entrypoints/grpc/grpc_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -347,7 +347,7 @@ async def _validate_and_convert_params(
prompt_logprobs=logprobs if resp_options.input_tokens else None,
max_tokens=max_new_tokens,
min_tokens=min_new_tokens,
temperature=sampling.temperature if not greedy else 0.0,
temperature=with_default(sampling.temperature, 1.0) if not greedy else 0.0,
top_k=with_default(sampling.top_k, -1),
top_p=with_default(sampling.top_p, 1.0),
seed=sampling.seed if sampling.HasField("seed") else None,
Expand Down

0 comments on commit 1479404

Please sign in to comment.