cookbooks/pipeline/assistant_function_call.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AppBuilder Assistant SDK FunctionCall\n",
    "\n",
    "您可基于 Assistants SDK，可通过全代码形式创建和调试专属智能体，实现FunctionCall等功能。\n",
    "\n",
    "以下通过一个端到端的示例，介绍如何使用Assistants SDK创建智能体，并调用AppBuilder SDK内置的工具组件。\n",
    "\n",
    "该应用的场景是动物识别，通过Assistant SDK创建智能体，并调用AppBuilder SDK内置的动物识别组件，实现识别动物的功能。\n",
    "\n",
    "Assistants API/SDK 正在内测中，敬请期待公测版本。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一、环境准备\n",
    "\n",
    "首先需要安装AppBuilder-SDK代码库，若已在开发环境安装，则可跳过此步。\n",
    "\n",
    "**注意：**: appbuilder-sdk 的python版本要求 3.9+，安装的SDK version >= 0.7.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 -m pip install appbuilder-sdk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二、代码实现\n",
    "\n",
    "### 2.1 创建Assitant & Thread & Message\n",
    "\n",
    "我们此处调用 `appbuilder.AnimalRecognition()`组件，需要Assitant识别的图片为一只大熊猫，示例图如下：\n",
    "\n",
    "\n",
    "<img src=\"https://bj.bcebos.com/v1/appbuilder/animal_recognize_test.png?authorization=bce-auth-v1%2FALTAKGa8m4qCUasgoljdEDAzLm%2F2024-01-24T12%3A19%3A16Z%2F-1%2Fhost%2F411bad53034fa8f9c6edbe5c4909d76ecf6fad6862cf937c03f8c5260d51c6ae\" alt=\"drawing\" width=\"1000\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import appbuilder\n",
    "\n",
    "# 配置你的密钥，主要在这之前需要首先申请Assistant API的内测资格\n",
    "os.environ[\"APPBUILDER_TOKEN\"] = \"your_appbuilder_token\"\n",
    "\n",
    "# 创建 Assitant\n",
    "assistant = appbuilder.assistant.assistants.create(\n",
    "    name=\"test_function\",\n",
    "    description=\"你是一个热心的朋友\",\n",
    "    instructions=\"请用友善的语气回答问题\",\n",
    "    tools=[\n",
    "        {'type': 'function', 'function': appbuilder.AnimalRecognition().manifests[0]}\n",
    "    ]\n",
    ")\n",
    "\n",
    "# 创建 Thread（即会话：conversation）\n",
    "thread = appbuilder.assistant.threads.create()\n",
    "\n",
    "image_url = \"https://bj.bcebos.com/v1/appbuilder/animal_recognize_test.png?\" \\\n",
    "            \"authorization=bce-auth-v1%2FALTAKGa8m4qCUasgoljdEDAzLm%2F2024-01-24T\" \\\n",
    "            \"12%3A19%3A16Z%2F-1%2Fhost%2F411bad53034fa8f9c6edbe5c4909d76ecf6fad68\" \\\n",
    "            \"62cf937c03f8c5260d51c6ae\"\n",
    "\n",
    "origin_query = \"我有一张图片，url是: {}, 麻烦帮我看看这是什么动物\".format(image_url)\n",
    "\n",
    "# 在Thread中添加一条Message，内容为原始的query\n",
    "appbuilder.assistant.threads.messages.create(\n",
    "    thread_id=thread.id,\n",
    "    content=origin_query,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 第一次运行，触发函数调用\n",
    "\n",
    "我们向Assistant询问 `我有一张图片，url是: X, 麻烦帮我看看这是什么动物`, Assistant会调用我们预先配置好的函数，并返回调用参数，我们收到调用触发的信号后，在本地调用函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 触发第一次run，并收到函数本地调用的信号\n",
    "run_result = appbuilder.assistant.threads.runs.run(\n",
    "    thread_id=thread.id,\n",
    "    assistant_id=assistant.id,\n",
    ")\n",
    "\n",
    "print(\"\\nFirst run result: {}\\n\".format(run_result.model_dump_json(indent=4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "运行结果为：\n",
    "\n",
    "```bash\n",
    "First run result: {\n",
    "    \"id\": \"run_089cffe9a82944bd9cdb87a8a0d8b54f\",\n",
    "    \"object\": \"run.result\",\n",
    "    \"assistant_id\": \"asst_922cacd503354349a1a0a5afb0be0408\",\n",
    "    \"thread_id\": \"thread_523a4aacc6a845a0b8252009af5771ea\",\n",
    "    \"model\": \"\",\n",
    "    \"instructions\": \"\",\n",
    "    \"thought_instructions\": \"\",\n",
    "    \"chat_instructions\": \"\",\n",
    "    \"tools\": null,\n",
    "    \"file_ids\": null,\n",
    "    \"status\": \"requires_action\",\n",
    "    \"required_action\": {\n",
    "        \"type\": \"submit_tool_outputs\",\n",
    "        \"submit_tool_outputs\": {\n",
    "            \"tool_calls\": [\n",
    "                {\n",
    "                    \"id\": \"call-thread_523a4aacc6a845a0b8252009af5771ea-run_089cffe9a82944bd9cdb87a8a0d8b54f-step-1\",\n",
    "                    \"type\": \"function\",\n",
    "                    \"function\": {\n",
    "                        \"name\": \"animal_rec\",\n",
    "                        \"arguments\": \"{\\\"img_url\\\":\\\"https://bj.bcebos.com/v1/appbuilder/animal_recognize_test.png?authorization=bce-auth-v1%2FALTAKGa8m4qCUasgoljdEDAzLm%2F2024-01-24T12%3A19%3A16Z%2F-1%2Fhost%2F411bad53034fa8f9c6edbe5c4909d76ecf6fad6862cf937c03f8c5260d51c6ae\\\"}\",\n",
    "                        \"output\": \"\"\n",
    "                    }\n",
    "                }\n",
    "            ]\n",
    "        }\n",
    "    },\n",
    "    \"last_error\": null,\n",
    "    \"final_answer\": null,\n",
    "    \"created_at\": 1713868218144,\n",
    "    \"started_at\": 1713868218164,\n",
    "    \"expired_at\": 0,\n",
    "    \"cancelled_at\": 0,\n",
    "    \"failed_at\": 0,\n",
    "    \"completed_at\": 0\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 识别函数调用的触发信号，调用本地函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 触发函数调用的标志是： status == 'requires_action'\n",
    "assert run_result.status == 'requires_action'\n",
    "\n",
    "# 获取函数调用的信息\n",
    "tool_call = run_result.required_action.submit_tool_outputs.tool_calls[0]\n",
    "\n",
    "# 调用工具组件的 tool_eval 方法，获取函数本地的运行结果\n",
    "func_res = appbuilder.AnimalRecognition().tool_eval(\n",
    "        name=\"animal_rec\",\n",
    "        streaming=True,\n",
    "        origin_query=origin_query,\n",
    "        **eval(tool_call.function.arguments))\n",
    "\n",
    "func_message = \"\"\n",
    "for res in func_res:\n",
    "    func_message += res\n",
    "print(\"\\nFunction result: {}\\n\".format(func_message))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "运行结果为：\n",
    "\n",
    "\n",
    "```bash\n",
    "Function result: 模型识别结果为：\n",
    "类别: 国宝大熊猫 置信度: 0.975161\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 上传函数运行结果，继续触发运行，得到完整对话结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run_result = appbuilder.assistant.threads.runs.run(\n",
    "    thread_id=thread.id,\n",
    "    assistant_id=assistant.id,\n",
    "    tool_output={\n",
    "        \"tool_call_id\":tool_call.id,\n",
    "        \"output\": func_message,\n",
    "        \"run_id\": run_result.id\n",
    "    },\n",
    ")\n",
    "print(\"\\nFinal run result: {}\\n\".format(run_result.model_dump_json(indent=4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "运行结果为：\n",
    "\n",
    "```bash\n",
    "\n",
    "Final run result: {\n",
    "    \"id\": \"run_089cffe9a82944bd9cdb87a8a0d8b54f\",\n",
    "    \"object\": \"run.result\",\n",
    "    \"assistant_id\": \"asst_922cacd503354349a1a0a5afb0be0408\",\n",
    "    \"thread_id\": \"thread_523a4aacc6a845a0b8252009af5771ea\",\n",
    "    \"model\": \"\",\n",
    "    \"instructions\": \"\",\n",
    "    \"thought_instructions\": \"\",\n",
    "    \"chat_instructions\": \"\",\n",
    "    \"tools\": null,\n",
    "    \"file_ids\": null,\n",
    "    \"status\": \"completed\",\n",
    "    \"required_action\": null,\n",
    "    \"last_error\": null,\n",
    "    \"final_answer\": {\n",
    "        \"type\": \"message\",\n",
    "        \"message\": {\n",
    "            \"message_id\": \"chatmsg_8d54f8c3c6714665baeda5eed013a5e9\",\n",
    "            \"content\": {\n",
    "                \"type\": \"text\",\n",
    "                \"text\": {\n",
    "                    \"value\": \"根据您的图片，我识别出图片中的动物是**国宝大熊猫**，识别结果的置信度很高，为0.975161。如果您还有其他问题或需要识别其他图片，请随时告诉我。\",\n",
    "                    \"annotations\": null\n",
    "                }\n",
    "            }\n",
    "        }\n",
    "    },\n",
    "    \"created_at\": 1713868218144,\n",
    "    \"started_at\": 1713868239553,\n",
    "    \"expired_at\": 0,\n",
    "    \"cancelled_at\": 0,\n",
    "    \"failed_at\": 0,\n",
    "    \"completed_at\": 1713868251090\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结\n",
    "\n",
    "在本示例中，展示了如何使用Assistant SDK，并调用AppBuilder中的能力组件，原子化的进行Agent流程编排与交互"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}