-
Notifications
You must be signed in to change notification settings - Fork 60.2k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix: add max_tokens when using vision model (#4157)
- Loading branch information
Showing
1 changed file
with
10 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
08fa227
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@fred-bf This submission triggers the use of models outside the DEFAULT_MODELS list, automatically adding a
max_tokens = 4096
parameter, which leads to errorsI used
together.ai + Qwen/Qwen1.5-72B-Chat
. The service provider returned the following error:08fa227
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've been knowing that an error will occur due to the setting
value: Math.max(modelConfig.max_tokens, 4096)
.Ideally,
Math.max
should not be set, as the maximum error handling is already managed by the provider.08fa227
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@H0llyW00dzZ You are right, just created a hot fix for this 43e5dc2
08fa227
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ficapy the issue have been fixed in the latest commit, the original way to detect vision model is regarding all unrecognized model as vision model https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/blob/08fa22749aea8f497811f684bd9c7ef68d698666/app/utils.ts#L297
08fa227
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now, it is better and more straightforward.