[Feature] 可以添stop_token_ids 字段设置么。。。

[YingchaoX]

使用Qwen-14B-Chat作为backend，没有触发stop_token会一直输出文字。。。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Title: [Feature] Can you add stop_token_ids field setting? . .

Using Qwen-14B-Chat as the backend, text will always be output if stop_token is not triggered. . .

[H0llyW00dzZ]

not available at this moment because it will break client openai LLMApi which is the best already

[Yidadaa]

你在服务端设置好默认值就可以了，自建模型的优点不就是这样吗

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

You just need to set the default values ​​on the server. Isn't that the advantage of self-built models?

[CoderCore]

在使用 python -m vllm.entrypoints.openai.api_server 启动API服务器时，若你想为模型推理添加 sample_params（采样参数），通常这些参数不是直接通过命令行传递给API服务器的。相反，sample_params 是在API请求中作为参数传递给模型的。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

When using python -m vllm.entrypoints.openai.api_server to start the API server, if you want to add sample_params (sampling parameters) for model inference, usually these parameters are not passed directly to the API server through the command line. Instead, sample_params are passed as parameters to the model in the API request.

[kevinzhao2233]

在使用 python -m vllm.entrypoints.openai.api_server 启动API服务器时，若你想为模型推理添加 sample_params（采样参数），通常这些参数不是直接通过命令行传递给API服务器的。相反，sample_params 是在API请求中作为参数传递给模型的。

请问你现在是怎么解决这个问题的

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

When using python -m vllm.entrypoints.openai.api_server to start the API server, if you want to add sample_params (sampling parameters) for model inference, usually these parameters are not passed directly to the API server through the command line. Instead, sample_params are passed as parameters to the model in the API request.

May I ask how you solve this problem now?