09_Smart_RAG.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Smart RAG\n",
    "- Know what you know and what you don't know.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip3 install -qU  markdownify  langchain-upstage rank_bm25"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The dotenv extension is already loaded. To reload it, use:\n",
      "  %reload_ext dotenv\n"
     ]
    }
   ],
   "source": [
    "\n",
    "%load_ext dotenv\n",
    "%dotenv\n",
    "# UPSTAGE_API_KEY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "solar_summary = \"\"\"\n",
    "SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling\n",
    "\n",
    "We introduce SOLAR 10.7B, a large language model (LLM) with 10.7 billion parameters, \n",
    "demonstrating superior performance in various natural language processing (NLP) tasks. \n",
    "Inspired by recent efforts to efficiently up-scale LLMs, \n",
    "we present a method for scaling LLMs called depth up-scaling (DUS), \n",
    "which encompasses depthwise scaling and continued pretraining.\n",
    "In contrast to other LLM up-scaling methods that use mixture-of-experts, \n",
    "DUS does not require complex changes to train and inference efficiently. \n",
    "We show experimentally that DUS is simple yet effective \n",
    "in scaling up high-performance LLMs from small ones. \n",
    "Building on the DUS model, we additionally present SOLAR 10.7B-Instruct, \n",
    "a variant fine-tuned for instruction-following capabilities, \n",
    "surpassing Mixtral-8x7B-Instruct. \n",
    "SOLAR 10.7B is publicly available under the Apache 2.0 license, \n",
    "promoting broad access and application in the LLM field.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import PromptTemplate\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_upstage import ChatUpstage\n",
    "\n",
    "llm = ChatUpstage()\n",
    "\n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"\"\"\n",
    "    Please provide answer from the following context. \n",
    "    If the answer is not present in the context, please write \"The information is not present in the context.\"\n",
    "\n",
    "    ---\n",
    "    Question: {question}\n",
    "    ---\n",
    "    Context: {context}\n",
    "    \"\"\"\n",
    ")\n",
    "chain = prompt_template | llm | StrOutputParser()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The answer is: DUS stands for Depth Up-Scaling.'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.invoke({\"question\": \"What is DUS?\", \"context\": solar_summary})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The information is not present in the context.'"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chain.invoke({\"question\": \"How to get to Seoul from SF\", \"context\": solar_summary})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# RAG or Search?\n",
    "def is_in(question, context):\n",
    "    is_in_conetxt = \"\"\"As a helpful assistant, \n",
    "please use your best judgment to determine if the answer to the question is within the given context. \n",
    "If the answer is present in the context, please respond with \"yes\". \n",
    "If not, please respond with \"no\". \n",
    "Only provide \"yes\" or \"no\" and avoid including any additional information. \n",
    "Please do your best. Here is the question and the context:\n",
    "---\n",
    "CONTEXT: {context}\n",
    "---\n",
    "QUESTION: {question}\n",
    "---\n",
    "OUTPUT (yes or no):\"\"\"\n",
    "\n",
    "    is_in_prompt = PromptTemplate.from_template(is_in_conetxt)\n",
    "    chain = is_in_prompt | ChatUpstage() | StrOutputParser()\n",
    "\n",
    "    response = chain.invoke({\"context\": context, \"question\": question})\n",
    "    print(response)\n",
    "    return response.lower().startswith(\"yes\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "no\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "is_in(\"How to get to Seoul from SF\", solar_summary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "yes\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "is_in(\"What is DUS?\", solar_summary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Smart RAG, Self-Improving RAG\n",
    "import os\n",
    "from tavily import TavilyClient\n",
    "\n",
    "\n",
    "def smart_rag(question, context):\n",
    "    if not is_in(question, context):\n",
    "        print(\"Searching in tavily\")\n",
    "        tavily = TavilyClient(api_key=os.environ[\"TAVILY_API_KEY\"])\n",
    "        context = tavily.search(query=question)\n",
    "\n",
    "    chain = prompt_template | llm | StrOutputParser()\n",
    "    return chain.invoke({\"context\": context, \"question\": question})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "yes\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'The answer to the question \"What is DUS?\" is:\\n\\nDepth Up-Scaling (DUS)'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "smart_rag(\"What is DUS?\", solar_summary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "no.\n",
      "Searching in tavily\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'The answer to \"How to get to Seoul from SF?\" is:\\n\\n1. Fly from San Francisco (SFO) to Seoul (ICN) with airlines such as ANA, Japan Airlines, Asiana Airlines, Korean Air, and United Airlines.\\n2. Take a train from Incheon Int\\'l Airport T1 to Seoul Station.\\n3. Take the BART from Civic Center / UN Plaza to Milpitas and then fly from San Jose (SJC) to Incheon (ICN).\\n\\nPlease note that the cheapest flights from San Francisco to Seoul start at $453 with AIR PREMIA.'"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "smart_rag(\"How to get to Seoul from SF?\", solar_summary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Excercise\n",
    "\n",
    "The `is_in` function sometimes works, but other times it does not. You can significantly improve it by providing a more detailed description and adding two or three examples in the prompt to see how it performs better."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}