Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix(Chat): Always ensure Anthropic user message comes first (not just when token_limits are provided) #1685

Merged
merged 3 commits into from
Sep 12, 2024

Conversation

cpsievert
Copy link
Collaborator

Currently chat.messages(format="anthropic") only removes non-user starting messages if token_limits is not None (and we changed the default to None in #1657).

As a result, an example like this (where Chat(messages) are provided) now errors out after #1657:

from anthropic import AsyncAnthropic
from shiny.express import ui

ui.page_opts(
    title="Hello Shiny Chat",
    fillable=True,
    fillable_mobile=True,
)

chat = ui.Chat(id="chat", messages=["Hello! How can I help you today?"])
chat.ui()

llm = AsyncAnthropic()


@chat.on_user_submit
async def _():
    response = await llm.messages.create(
        model="claude-3-5-sonnet-20240620",
        messages=chat.messages(format="anthropic"),
        stream=True,
        max_tokens=1000,
    )
    await chat.append_message_stream(response)
Screenshot 2024-09-12 at 3 01 47 PM

This PR fixes the problem by having chat.messages(format="anthropic") always remove non-user starting messages.

@cpsievert cpsievert force-pushed the fix/chat-anthropic-message-format branch from a5c22d6 to 1986a40 Compare September 12, 2024 20:23
@cpsievert cpsievert force-pushed the fix/chat-anthropic-message-format branch from df1decd to 3a78005 Compare September 12, 2024 20:28
@cpsievert cpsievert merged commit 5454eb0 into main Sep 12, 2024
48 checks passed
@cpsievert cpsievert deleted the fix/chat-anthropic-message-format branch September 12, 2024 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant