feat(agents-api): Add support for claude computer-use

[creatorrr]

from anthropic import Anthropic
from anthropic.types.beta.beta_message import BetaMessage

from litellm import ChatCompletionMessageToolCall, Function, Message
from litellm.types.utils import Choices, ModelResponse

# ...
# ...


    # Check if the model is Anthropic and bash/editor/computer use tool is included
    if "claude" in agent_model.lower() and any(
        tool.type in ["bash_20241022", "text_editor_20241022", "computer_20241022"]
        for tool in agent_tools
    ):
        # Retrieve the API key from the environment variable
        betas = [COMPUTER_USE_BETA_FLAG]
        # Use Anthropic API directly
        client = Anthropic(api_key=anthropic_api_key)

        # Claude Response
        claude_response: BetaMessage = await client.beta.messages.create(
            model=agent_model,
            messages=prompt,
            tools=formatted_agent_tools,
            max_tokens=1024,
            betas=betas,
        )

# Claude returns [ToolUse | TextBlock]
        # We need to convert tool_use to tool_calls
        # And set content = TextBlock.text
        # But we need to ensure no more than one text block is returned
        if (
            len([block for block in claude_response.content if block.type == "text"])
            > 1
        ):
            raise ApplicationError("Claude should only return one message")

        text_block = next(
            (block for block in claude_response.content if block.type == "text"),
            None,
        )

        stop_reason = claude_response.stop_reason

        if stop_reason == "tool_use":
            choice = Choices(
                message=Message(
                    role="assistant",
                    content=text_block.text if text_block else None,
                    tool_calls=[
                        ChatCompletionMessageToolCall(
                            type="function",
                            function=Function(
                                name=block.name,
                                arguments=block.input,
                            ),
                        )
                        for block in claude_response.content
                        if block.type == "tool_use"
                    ],
                ),
                finish_reason="tool_calls",
            )
        else:
            assert text_block, "Claude should always return a text block for stop_reason=stop"

            choice = Choices(
                message=Message(
                    role="assistant",
                    content=text_block.text,
                ),
                finish_reason="stop",
            )

---

Important

Adds support for Anthropic Claude model with new tool types, updates models, and fixes a bug in execute_system.py.

• Behavior:
  • Adds support for Anthropic Claude model with computer_20241022, text_editor_20241022, and bash_20241022 tools in prompt_step.py.
  • Handles tool use in Claude responses, raising an error if multiple messages are returned.
• Models:
  • Adds type field to Tool, CreateToolRequest, PatchToolRequest, and UpdateToolRequest models.
  • Updates ToolType enum to include new tool types.
• Misc:
  • Fixes a bug in execute_system.py related to delete operation handling.
  • Imports AsyncAnthropic in prompt_step.py for asynchronous API calls.

^{This description was created by for 5867ecb. It will automatically update as commits are pushed.}

[sweep-ai]

Hey @creatorrr, here is an example of how you can ask me to improve this pull request:

@Sweep Add unit tests for the `format_agent_tool` function in `prompt_step.py` to verify correct formatting for each tool type:
- Test formatting of function tools
- Test formatting of computer_20241022 tools
- Test formatting of bash_20241022 tools
- Test formatting of text_editor_20241022 tools

📖 For more information on how to use Sweep, please read our documentation.

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to efc9376 in 51 seconds

More details

• Looked at 399 lines of code in 7 files
• Skipped 0 files when reviewing.
• Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_pmG9hYHbuFufaji2

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on 5867ecb in 22 seconds

More details

• Looked at 121 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 2 drafted comments based on config settings.

1. agents-api/agents_api/activities/task_steps/prompt_step.py:9

• Draft comment:
  Redundant import of ModelResponse from litellm.utils. It's already imported from litellm.types.utils. Consider removing one of them.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The import of ModelResponse from litellm.utils is redundant since it's already imported from litellm.types.utils. This can lead to confusion and should be cleaned up.

2. agents-api/agents_api/activities/task_steps/prompt_step.py:189

• Draft comment:
  Consider using datetime.utcnow().timestamp() instead of datetime.now().timestamp() to avoid potential timezone issues.
• Reason this comment was not posted:
  Confidence changes required: 33%
  The prompt_step function uses datetime.now().timestamp() to get the current timestamp. This is generally fine, but if the application is timezone-sensitive, it might be better to use datetime.utcnow().timestamp() to avoid any potential issues with local timezones.

Workflow ID: wflow_Jr4gFzuzZalpbe5Q

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.