feat: Add vision support #4076
feat: Add vision support #4076
Conversation
TheRamU
commented
Feb 19, 2024
TheRamU
commented
- Gemini Pro Vision supported
- GPT-4 Vision supported
- Fixed not reporting to the user when Gemini request error
- Image features are now globally supported in this application
|
@TheRamU is attempting to deploy a commit to the NextChat Team on Vercel. A member of the Team first needs to authorize it. |
|
Your build has completed! |
|
LGTM |
| if (currentModel.startsWith("gpt")) { | ||
| return SUMMARIZE_MODEL; | ||
| } | ||
| if (currentModel.startsWith("gemini-pro")) { |
There was a problem hiding this comment.
Tip
It's more effective to use Summarize with Gemini's own models. For example, gemini-pro refers to the Gemini Pro model, and gemini-pro-vision refers to the Gemini Pro Vision model. These models are more affordable compared to OpenAI hahaha and more efficient, as each model has its own specific token limit. For instance, the token input limit for the gemini-pro-vision model is around 12288 tokens.
Proof:
There was a problem hiding this comment.
Gemini Pro Vision does not enable multiturn chat, which limits its role as a "Summarize Model"
There was a problem hiding this comment.
Gemini Pro Vision does not enable multiturn chat, which limits its role as a "Summarize Model"
rip, its better model, unlike openai gpt-4-vision-preview
|
Show off
|
* Add vision support (ChatGPTNextWeb#4076) * Refactor [UI/UX] [Front End] [Chat] Remove Duplicate "onUserInput" - [+] refactor(chat.tsx): remove duplicate onUserInput call and localStorage.setItem in _Chat function * Feat [UI/UX] [Chat List] Search Support for Multimodal Content - [+] feat(chat-list.tsx): add search support for array of MultimodalContent in ChatList component * Style [UI/UX] [Chat List] Linting - [+] style(chat-list.tsx): improve readability by breaking down lengthy if condition into multiple lines * Adding Back Text Moderation - [+] feat(openai.ts): add support for text moderation in chat method * Feat [LLM APIs] [Google] InlineData - [+] feat(google.ts): add InlineData to MessagePart, refactor message construction - [+] chore(google.ts): add comments for clarity * Style [LLM APIs] [Google] InlineData - [+] style(google.ts): update comment for InlineData interface * Todo [LLM APIs] [Google] InlineData - [+] todo(google.ts): add TODO comment to improve safety settings configuration * Todo [UI/UX] [Front End] [Chat] Summarize - [+] chore(chat.ts): add TODO comment to improve the summary for gemini-pro-vision --------- Co-authored-by: TheRamU <[email protected]>
Is it possible to select the Google visual model so that the conversation does not use multiple rounds of chat, and only uploads single-discussion conversations, without needing to clear the context every time it is used? |
|
能否在选中谷歌视觉模型的时候，对话不使用多轮聊天，仅上传单次对话，不用每次使用需要清除上下文 |
Can the conversation not use multiple rounds of chat when the Google visual model is selected, and only a single conversation can be uploaded without the need to clear the context each time it is used? |
|
Why are my responses using the vision-preview api incomplete 
|
