graph_examples.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d362591e-9f54-46f9-b703-49062ac3072f",
   "metadata": {},
   "source": [
    "# Knowledge Graph RAG\n",
    "\n",
    "<img src=\"./media/graph_start.png\" width=600>\n",
    "\n",
    "*[Improving Knowledge Graph Completion with Generative LM and neighbors](https://deeppavlov.ai/research/tpost/bn15u1y4v1-improving-knowledge-graph-completion-wit)*\n",
    "\n",
    "In the evolving landscape of AI and information retrieval, knowledge graphs have emerged as a powerful way to represent complex, interconnected information. A knowledge graph is a knowledge base that uses a graph-structured data model or topology to represent and operate on data. Knowledge graphs are often used to store interlinked descriptions of entities – objects, events, situations or abstract concepts – while also encoding the free-form semantics or relationships underlying these entities. [Source: Wikipedia](https://en.wikipedia.org/wiki/Knowledge_graph)\n",
    "\n",
    "What makes knowledge graphs particularly powerful is their ability to mirror human cognition in data. They more explicitly map the relationships between objects, concepts, or ideas together through both their semantic and relational connections. This approach closely parallels how our brains naturally understand and internalize information – not as isolated facts, but as a web of interconnected concepts and relationships.\n",
    "\n",
    "<img src=\"./media/coffee_graph_ex.png\" width=400>\n",
    "\n",
    "Looking at a concept like \"coffee,\" we don't just know it's a beverage; we automatically connect it to related concepts like beans, brewing methods, caffeine, morning routines, and social interactions. Knowledge graphs capture these natural associations in a structured way.\n",
    "\n",
    "Traditional RAG systems, while effective at semantic similarity-based retrieval, often struggle to capture broader conceptual relationships across text chunks. Knowledge Graph RAG addresses this limitation by introducing a structured, hierarchical approach to information organization and retrieval. By representing data in a graph format, these systems can traverse relationships between concepts, enabling more sophisticated query understanding and response generation. This approach allows for targeted querying along specific relationship paths, handles complex multi-hop questions, and provides clearer reasoning through explicit connection paths. The result is a more nuanced and interpretable system that combines the structured reasoning of knowledge graphs with the natural language capabilities of large language models.\n",
    "\n",
    "While [knowledge graphs are not a new concept](https://blog.google/products/search/introducing-knowledge-graph-things-not/), their creation has traditionally been a resource-intensive process. Early knowledge graphs were built either through manual curation by domain experts or by converting existing structured data from relational databases. This limited both their scale and adaptability to new domains.\n",
    "\n",
    "<img src=\"./media/table_comp.png\" width=600>\n",
    "\n",
    "*[What is a Knowledge Graph (KG)?](https://zilliz.com/learn/what-is-knowledge-graph)*\n",
    "\n",
    "The introduction of LLMs has transformed this landscape. LLMs' capabilities in NLP, reasoning, and relationship extraction now enable automated construction of knowledge graphs from unstructured text. These models can identify entities, infer relationships, and structure information in ways that previously required extensive manual labor. As a plus, this allows knowledge graphs to be dynamically updated and expanded as new information becomes available, making them more practical and scalable for real-world applications.\n",
    "\n",
    "To see this in action ourselves, and compare it to traditional vector similarity techniques, we'll take a look at Microsoft's Open Source [GraphRAG](https://microsoft.github.io/graphrag/) and how it works behind the scenes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb752dad-d6bf-436f-a175-03a1d491bb3e",
   "metadata": {},
   "source": [
    "---\n",
    "## 3 Main Components of Knowledge Graphs\n",
    "\n",
    "**Entity**\n",
    "\n",
    "<img src=\"./media/entities.png\" width=500>\n",
    "\n",
    "An Entity is a distinct object, person, place, event, or concept that has been extracted from a chunk of text through LLM analysis. Entities form the nodes of the knowledge graph. During the creation of the knowledge graph, when duplicate entities are found they are merged while preserving their various descriptions, creating a comprehensive representation of each unique entity.\n",
    "\n",
    "**Relationship**\n",
    "\n",
    "<img src=\"./media/relationship.png\" width=400>\n",
    "\n",
    "A Relationship defines a connection between two entities in the knowledge graph. These connections are extracted directly from text units through LLM analysis, alongside entities. Each relationship includes a source entity, target entity, and descriptive information about their connection. When duplicate relationships are found between the same entities, they are merged by combining their descriptions to create a more complete understanding of the connection.\n",
    "\n",
    "**Community**\n",
    "\n",
    "<img src=\"./media/communities.png\" width=400>\n",
    "\n",
    "A Community is a cluster of related entities and relationships identified through hierarchical community detection, generally using the [Leiden Algorithm](https://en.wikipedia.org/wiki/Leiden_algorithm). Communities create a structured way to understand different levels of granularity within the knowledge graph, from broad overviews at the top level to detailed local clusters at lower levels. This hierarchical structure helps in organizing and navigating complex knowledge graphs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5d7e77f-a543-49e2-813e-c1305f7a058d",
   "metadata": {},
   "source": [
    "---\n",
    "## GraphRAG Creation Data Flow\n",
    "\n",
    "<img src=./media/graph_building.png width=1000>\n",
    "\n",
    "Indexxing in GraphRAG is an extensive process, where we load the document, split it into chunks, create sub graphs at a chunk level, combine these subgraphs into our final graph, algorithmically identify communities, then document the communities main features."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19896a38-bcf7-4664-9f57-cb12cf15cdea",
   "metadata": {},
   "source": [
    "### **Loading and Splitting Our Text**\n",
    "\n",
    "For our example, we'll be using [The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities](https://arxiv.org/pdf/2408.13296).\n",
    "\n",
    "This will be loaded as a text file (remove index, glossary, and references) and split into 1200 token, 100 token overlap chunks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97e6a581-74a1-455e-8634-2c94d6496e45",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from langchain_text_splitters import TokenTextSplitter\n",
    "\n",
    "with open(\"./ragtest/input/ft_guide.txt\", 'r') as file:\n",
    "    content = file.read()\n",
    "\n",
    "text_splitter = TokenTextSplitter(chunk_size=1200, chunk_overlap=100)\n",
    "\n",
    "texts = text_splitter.split_text(content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2df9e65e-531c-4e94-9808-4ede865621d7",
   "metadata": {},
   "source": [
    "**Entity and Relationship Extraction Prompt**\n",
    "\n",
    "This is a [tuned](https://microsoft.github.io/graphrag/prompt_tuning/auto_prompt_tuning/) entity extraction prompt used in our real GraphRAG implementation, extracted in this format to see what's happening."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "b6236c47-8584-41d6-8553-f516e282d319",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "llm = ChatOpenAI(temperature=0.0, model=\"gpt-4o\")\n",
    "\n",
    "prompt_template = \"\"\"\n",
    "-Goal-\n",
    "Given a text document that is potentially relevant to this activity and a list of entity types, identify all entities of those types from the text and all relationships among the identified entities.\n",
    "\n",
    "-Steps-\n",
    "1. Identify all entities. For each identified entity, extract the following information:\n",
    "- entity_name: Name of the entity, capitalized\n",
    "- entity_type: One of the following types: [large language model, differential privacy, federated learning, healthcare, adversarial training, security measures, open-source tool, dataset, learning rate, AdaGrad, RMSprop, adapter architecture, LoRA, API, model support, evaluation metrics, deployment, Python library, hardware accelerators, hyperparameters, data preprocessing, data imbalance, GPU-based deployment, distributed inference]\n",
    "- entity_description: Comprehensive description of the entity's attributes and activities\n",
    "Format each entity as (\"entity\"{{tuple_delimiter}}<entity_name>{{tuple_delimiter}}<entity_type>{{tuple_delimiter}}<entity_description>)\n",
    "\n",
    "2. From the entities identified in step 1, identify all pairs of (source_entity, target_entity) that are *clearly related* to each other.\n",
    "For each pair of related entities, extract the following information:\n",
    "- source_entity: name of the source entity, as identified in step 1\n",
    "- target_entity: name of the target entity, as identified in step 1\n",
    "- relationship_description: explanation as to why you think the source entity and the target entity are related to each other\n",
    "- relationship_strength: an integer score between 1 to 10, indicating strength of the relationship between the source entity and target entity\n",
    "Format each relationship as (\"relationship\"{{tuple_delimiter}}<source_entity>{{tuple_delimiter}}<target_entity>{{tuple_delimiter}}<relationship_description>{{tuple_delimiter}}<relationship_strength>)\n",
    "\n",
    "3. Return output in The primary language of the provided text is \"English.\" as a single list of all the entities and relationships identified in steps 1 and 2. Use **{{record_delimiter}}** as the list delimiter.\n",
    "\n",
    "4. If you have to translate into The primary language of the provided text is \"English.\", just translate the descriptions, nothing else!\n",
    "\n",
    "5. When finished, output {{completion_delimiter}}.\n",
    "\n",
    "-Examples-\n",
    "######################\n",
    "\n",
    "Example 1:\n",
    "\n",
    "entity_types: [large language model, differential privacy, federated learning, healthcare, adversarial training, security measures, open-source tool, dataset, learning rate, AdaGrad, RMSprop, adapter architecture, LoRA, API, model support, evaluation metrics, deployment, Python library, hardware accelerators, hyperparameters, data preprocessing, data imbalance, GPU-based deployment, distributed inference]\n",
    "text:\n",
    " LLMs to create synthetic samples that mimic clients’ private data distribution using\n",
    "differential privacy. This approach significantly boosts SLMs’ performance by approximately 5% while\n",
    "maintaining data privacy with a minimal privacy budget, outperforming traditional methods relying\n",
    "solely on local private data.\n",
    "In healthcare, federated fine-tuning can allow hospitals to collaboratively train models on patient data\n",
    "without transferring sensitive information. This approach ensures data privacy while enabling the de-\n",
    "velopment of robust, generalisable AI systems.\n",
    "8https://ai.meta.com/responsible-ai/\n",
    "9https://huggingface.co/docs/hub/en/model-cards\n",
    "10https://www.tensorflow.org/responsible_ai/privacy/guide\n",
    "101 Frameworks for Enhancing Security\n",
    "Adversarial training and robust security measures[111] are essential for protecting fine-tuned models\n",
    "against attacks. The adversarial training approach involves training models with adversarial examples\n",
    "to improve their resilience against malicious inputs. Microsoft Azure’s\n",
    "------------------------\n",
    "output:\n",
    "(\"entity\"{{tuple_delimiter}}DIFFERENTIAL PRIVACY{{tuple_delimiter}}differential privacy{{tuple_delimiter}}Differential privacy is a technique used to create synthetic samples that mimic clients' private data distribution while maintaining data privacy with a minimal privacy budget{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}HEALTHCARE{{tuple_delimiter}}healthcare{{tuple_delimiter}}In healthcare, federated fine-tuning allows hospitals to collaboratively train models on patient data without transferring sensitive information, ensuring data privacy{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}FEDERATED LEARNING{{tuple_delimiter}}federated learning{{tuple_delimiter}}Federated learning is a method that enables collaborative model training on decentralized data sources, such as hospitals, without sharing sensitive information{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}ADVERSARIAL TRAINING{{tuple_delimiter}}adversarial training{{tuple_delimiter}}Adversarial training involves training models with adversarial examples to improve their resilience against malicious inputs{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}SECURITY MEASURES{{tuple_delimiter}}security measures{{tuple_delimiter}}Robust security measures are essential for protecting fine-tuned models against attacks{{record_delimiter}}\n",
    "(\"relationship\"{{tuple_delimiter}}DIFFERENTIAL PRIVACY{{tuple_delimiter}}FEDERATED LEARNING{{tuple_delimiter}}Differential privacy is used in federated learning to maintain data privacy while training models collaboratively{{tuple_delimiter}}8{{record_delimiter}}\n",
    "(\"relationship\"{{tuple_delimiter}}HEALTHCARE{{tuple_delimiter}}FEDERATED LEARNING{{tuple_delimiter}}Federated learning is applied in healthcare to train models on patient data without transferring sensitive information{{tuple_delimiter}}9{{record_delimiter}}\n",
    "(\"relationship\"{{tuple_delimiter}}ADVERSARIAL TRAINING{{tuple_delimiter}}SECURITY MEASURES{{tuple_delimiter}}Adversarial training is a security measure used to protect models against attacks by improving their resilience{{tuple_delimiter}}8{{completion_delimiter}}\n",
    "#############################\n",
    "\n",
    "\n",
    "Example 2:\n",
    "\n",
    "entity_types: [large language model, differential privacy, federated learning, healthcare, adversarial training, security measures, open-source tool, dataset, learning rate, AdaGrad, RMSprop, adapter architecture, LoRA, API, model support, evaluation metrics, deployment, Python library, hardware accelerators, hyperparameters, data preprocessing, data imbalance, GPU-based deployment, distributed inference]\n",
    "text:\n",
    "ARD [82] is an innovative open-source tool developed to enhance the safety of interactions\n",
    "with large language models (LLMs). This tool addresses three critical moderation tasks: detecting\n",
    "2https://huggingface.co/docs/transformers/en/model_doc/auto#transformers.AutoModelForCausalLM\n",
    "63 harmful intent in user prompts, identifying safety risks in model responses, and determining when a\n",
    "model appropriately refuses unsafe requests. Central to its development is WILDGUARD MIX3, a\n",
    "meticulously curated dataset comprising 92,000 labelled examples that include both benign prompts and\n",
    "adversarial attempts to bypass safety measures. The dataset is divided into WILDGUARD TRAIN, used\n",
    "for training the model, and WILDGUARD TEST, consisting of high-quality human-annotated examples\n",
    "for evaluation.\n",
    "The WILDGUARD model itself is fine-tuned on the Mistral-7B language model using the WILDGUARD\n",
    "TRAIN dataset, enabling it to perform all\n",
    "------------------------\n",
    "output:\n",
    "```plaintext\n",
    "(\"entity\"{{tuple_delimiter}}ARD{{tuple_delimiter}}open-source tool{{tuple_delimiter}}ARD is an innovative open-source tool developed to enhance the safety of interactions with large language models by addressing moderation tasks such as detecting harmful intent, identifying safety risks, and determining appropriate refusals of unsafe requests)\n",
    "{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}LARGE LANGUAGE MODELS{{tuple_delimiter}}large language model{{tuple_delimiter}}Large language models (LLMs) are advanced AI models designed to understand and generate human-like text, which ARD aims to interact with safely)\n",
    "{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}WILDGUARD MIX3{{tuple_delimiter}}dataset{{tuple_delimiter}}WILDGUARD MIX3 is a meticulously curated dataset comprising 92,000 labeled examples, including benign prompts and adversarial attempts, used for training and evaluating safety measures in language models)\n",
    "{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}WILDGUARD TRAIN{{tuple_delimiter}}dataset{{tuple_delimiter}}WILDGUARD TRAIN is a subset of the WILDGUARD MIX3 dataset used specifically for training the model on safety measures)\n",
    "{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}WILDGUARD TEST{{tuple_delimiter}}dataset{{tuple_delimiter}}WILDGUARD TEST is a subset of the WILDGUARD MIX3 dataset consisting of high-quality human-annotated examples used for evaluating the model's performance)\n",
    "{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}MISTRAL-7B{{tuple_delimiter}}large language model{{tuple_delimiter}}Mistral-7B is a language model that the WILDGUARD model is fine-tuned on using the WILDGUARD TRAIN dataset to enhance its safety performance)\n",
    "{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}ADVERSARIAL ATTEMPTS{{tuple_delimiter}}adversarial training{{tuple_delimiter}}Adversarial attempts are part of the WILDGUARD MIX3 dataset, used to test and improve the model's ability to handle unsafe or harmful inputs)\n",
    "{{record_delimiter}}\n",
    "(\"entity\"{{tuple_delimiter}}SAFETY MEASURES{{tuple_delimiter}}security measures{{tuple_delimiter}}Safety measures are protocols and techniques implemented to ensure that large language models interact safely with users, which ARD and the WILDGUARD dataset aim to enhance)\n",
    "{{record_delimiter}}\n",
    "(\"relationship\"{{tuple_delimiter}}ARD{{tuple_delimiter}}LARGE LANGUAGE MODELS{{tuple_delimiter}}ARD is designed to enhance the safety of interactions with large language models by addressing critical moderation tasks{{tuple_delimiter}}8)\n",
    "{{record_delimiter}}\n",
    "(\"relationship\"{{tuple_delimiter}}ARD{{tuple_delimiter}}WILDGUARD MIX3{{tuple_delimiter}}ARD uses the WILDGUARD MIX3 dataset to train and evaluate its moderation capabilities{{tuple_delimiter}}7)\n",
    "{{record_delimiter}}\n",
    "(\"relationship\"{{tuple_delimiter}}WILDGUARD MIX3{{tuple_delimiter}}WILDGUARD TRAIN{{tuple_delimiter}}WILDGUARD TRAIN is a subset of the WILDGUARD MIX3 dataset used for training{{tuple_delimiter}}9)\n",
    "{{record_delimiter}}\n",
    "(\"relationship\"{{tuple_delimiter}}WILDGUARD MIX3{{tuple_delimiter}}WILDGUARD TEST{{tuple_delimiter}}WILDGUARD TEST is a subset of the WILDGUARD MIX3 dataset used for evaluation{{tuple_delimiter}}9)\n",
    "{{record_delimiter}}\n",
    "(\"relationship\"{{tuple_delimiter}}WILDGUARD TRAIN{{tuple_delimiter}}MISTRAL-7B{{tuple_delimiter}}The WILDGUARD TRAIN dataset is used to fine-tune the Mistral-7B language model{{tuple_delimiter}}8)\n",
    "{{record_delimiter}}\n",
    "(\"relationship\"{{tuple_delimiter}}ADVERSARIAL ATTEMPTS{{tuple_delimiter}}SAFETY MEASURES{{tuple_delimiter}}Adversarial attempts are used to test and improve safety measures in language models{{tuple_delimiter}}7)\n",
    "{{completion_delimiter}}\n",
    "```\n",
    "#############################\n",
    "\n",
    "\n",
    "\n",
    "-Real Data-\n",
    "######################\n",
    "entity_types: [large language model, differential privacy, federated learning, healthcare, adversarial training, security measures, open-source tool, dataset, learning rate, AdaGrad, RMSprop, adapter architecture, LoRA, API, model support, evaluation metrics, deployment, Python library, hardware accelerators, hyperparameters, data preprocessing, data imbalance, GPU-based deployment, distributed inference]\n",
    "text: {input_text}\n",
    "######################\n",
    "output:\n",
    "\"\"\"\n",
    "\n",
    "prompt = ChatPromptTemplate.from_template(prompt_template)\n",
    "\n",
    "chain = prompt | llm | StrOutputParser()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0742c48f-043a-4653-ae9a-550d0b929386",
   "metadata": {},
   "source": [
    "**Creating a Response**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "81a8e812-85b4-446b-9e54-ca8a227947f2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
       "</pre>\n"
      ],
      "text/plain": [
       "HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "response = chain.invoke({\"input_text\": texts[25]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "1176f935-4ffb-4e37-8daa-505edced7bc1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "```plaintext\n",
      "(\"entity\"{tuple_delimiter}EVALUATION METRICS{tuple_delimiter}evaluation metrics{tuple_delimiter}Evaluation metrics are used to measure the performance of AI models, including metrics like cross-entropy, perplexity, factuality, and context relevance)\n",
      "{record_delimiter}\n",
      "(\"entity\"{tuple_delimiter}HYPERPARAMETERS{tuple_delimiter}hyperparameters{tuple_delimiter}Hyperparameters are key settings in model training, such as learning rate, batch size, and number of training epochs, which are adjusted to optimize model performance)\n",
      "{record_delimiter}\n",
      "(\"entity\"{tuple_delimiter}CROSS-ENTROPY{tuple_delimiter}evaluation metrics{tuple_delimiter}Cross-entropy is a key metric for evaluating large language models (LLMs) during training or fine-tuning, quantifying the difference between predicted and actual data distributions)\n",
      "{record_delimiter}\n",
      "(\"entity\"{tuple_delimiter}PERPLEXITY{tuple_delimiter}evaluation metrics{tuple_delimiter}Perplexity measures how well a probability distribution or model predicts a sample, indicating the model's uncertainty about the next word in a sequence)\n",
      "{record_delimiter}\n",
      "(\"entity\"{tuple_delimiter}FACTUALITY{tuple_delimiter}evaluation metrics{tuple_delimiter}Factuality assesses the accuracy of the information produced by the LLM, important for applications where misinformation could have serious consequences)\n",
      "{record_delimiter}\n",
      "(\"entity\"{tuple_delimiter}LLM UNCERTAINTY{tuple_delimiter}evaluation metrics{tuple_delimiter}LLM uncertainty is measured using log probability to identify low-quality generations, with lower uncertainty indicating higher output quality)\n",
      "{record_delimiter}\n",
      "(\"entity\"{tuple_delimiter}PROMPT PERPLEXITY{tuple_delimiter}evaluation metrics{tuple_delimiter}Prompt perplexity evaluates how well the model understands the input prompt, with lower values indicating clearer and more comprehensible prompts)\n",
      "{record_delimiter}\n",
      "(\"entity\"{tuple_delimiter}CONTEXT RELEVANCE{tuple_delimiter}evaluation metrics{tuple_delimiter}Context relevance measures how pertinent the retrieved context is to the user query in retrieval-augmented generation systems, improving response quality)\n",
      "{record_delimiter}\n",
      "(\"relationship\"{tuple_delimiter}CROSS-ENTROPY{tuple_delimiter}PERPLEXITY{tuple_delimiter}Both cross-entropy and perplexity are metrics used to evaluate the performance of large language models, focusing on prediction accuracy and uncertainty{tuple_delimiter}7)\n",
      "{record_delimiter}\n",
      "(\"relationship\"{tuple_delimiter}HYPERPARAMETERS{tuple_delimiter}EVALUATION METRICS{tuple_delimiter}Hyperparameters are adjusted based on evaluation metrics to optimize model performance and prevent overfitting{tuple_delimiter}8)\n",
      "{completion_delimiter}\n",
      "```\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55fd5fa6-626b-4c6b-ad17-84d4d3bc5bf4",
   "metadata": {},
   "source": [
    "We see the extraction of **entities**:\n",
    "\n",
    "`(\"entity\"{tuple_delimiter}EVALUATION METRICS{tuple_delimiter}evaluation metrics{tuple_delimiter}Evaluation metrics are criteria  used to assess the performance of AI models, including metrics like cross-entropy, perplexity, factuality, and context relevance)`\n",
    "\n",
    "As well as **relationships**:\n",
    "\n",
    "`(\"relationship\"{tuple_delimiter}EVALUATION METRICS{tuple_delimiter}CONTEXT RELEVANCE{tuple_delimiter}Context relevance is an evaluation metric that ensures the model uses the most pertinent information for generating responses{tuple_delimiter}8)`\n",
    "\n",
    "Following this, these per chunk subgraphs are merged together - any entities with the same name and type are merged by creating an array of their descriptions. Similarly, any relationships with the same source and target are merged by creating an array of their descriptions. These lists are then summarized one more time "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad41d036-00da-4dba-8fcf-1cee5b683d52",
   "metadata": {},
   "source": [
    "### **Looking at Final Entities and Relationships**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "26613855-7891-4b16-ad84-758f8a0ed8fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>human_readable_id</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>text_unit_ids</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>e3a7f24b-88b6-4481-b3a7-c35075a9671f</td>\n",
       "      <td>0</td>\n",
       "      <td>GPT-3</td>\n",
       "      <td>ORGANIZATION</td>\n",
       "      <td>GPT-3 is a large language model developed by O...</td>\n",
       "      <td>[ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>f55cae4e-dd0d-47a2-912b-f7680147dd31</td>\n",
       "      <td>1</td>\n",
       "      <td>GPT-4</td>\n",
       "      <td>ORGANIZATION</td>\n",
       "      <td>GPT-4 is an advanced large language model deve...</td>\n",
       "      <td>[ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>f3e3e46b-6746-45a7-9a26-1432f14c45e4</td>\n",
       "      <td>2</td>\n",
       "      <td>BERT</td>\n",
       "      <td>ORGANIZATION</td>\n",
       "      <td>BERT, which stands for Bidirectional Encoder R...</td>\n",
       "      <td>[ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0491a417-2e18-41c4-ae1e-3e39bf2eb98f</td>\n",
       "      <td>3</td>\n",
       "      <td>PALM</td>\n",
       "      <td>ORGANIZATION</td>\n",
       "      <td>PaLM is a large language model developed by Go...</td>\n",
       "      <td>[ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2b7f14f5-d1d5-49f6-bace-46fd1767f99e</td>\n",
       "      <td>4</td>\n",
       "      <td>LLAMA</td>\n",
       "      <td>ORGANIZATION</td>\n",
       "      <td>LLAMA is a versatile and advanced model known ...</td>\n",
       "      <td>[ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     id  human_readable_id  title  \\\n",
       "0  e3a7f24b-88b6-4481-b3a7-c35075a9671f                  0  GPT-3   \n",
       "1  f55cae4e-dd0d-47a2-912b-f7680147dd31                  1  GPT-4   \n",
       "2  f3e3e46b-6746-45a7-9a26-1432f14c45e4                  2   BERT   \n",
       "3  0491a417-2e18-41c4-ae1e-3e39bf2eb98f                  3   PALM   \n",
       "4  2b7f14f5-d1d5-49f6-bace-46fd1767f99e                  4  LLAMA   \n",
       "\n",
       "           type                                        description  \\\n",
       "0  ORGANIZATION  GPT-3 is a large language model developed by O...   \n",
       "1  ORGANIZATION  GPT-4 is an advanced large language model deve...   \n",
       "2  ORGANIZATION  BERT, which stands for Bidirectional Encoder R...   \n",
       "3  ORGANIZATION  PaLM is a large language model developed by Go...   \n",
       "4  ORGANIZATION  LLAMA is a versatile and advanced model known ...   \n",
       "\n",
       "                                       text_unit_ids  \n",
       "0  [ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...  \n",
       "1  [ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...  \n",
       "2  [ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...  \n",
       "3  [ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...  \n",
       "4  [ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...  "
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "entities = pd.read_parquet('./ragtest/output/create_final_entities.parquet')\n",
    "\n",
    "entities.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "9c4bf9fc-bb4e-4546-897b-991236079323",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>human_readable_id</th>\n",
       "      <th>source</th>\n",
       "      <th>target</th>\n",
       "      <th>description</th>\n",
       "      <th>weight</th>\n",
       "      <th>combined_degree</th>\n",
       "      <th>text_unit_ids</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>b895553a-f860-4d15-bba2-a42f1464e810</td>\n",
       "      <td>0</td>\n",
       "      <td>GPT-3</td>\n",
       "      <td>GPT-4</td>\n",
       "      <td>GPT-4 is an advanced version of GPT-3, buildin...</td>\n",
       "      <td>8.0</td>\n",
       "      <td>20</td>\n",
       "      <td>[ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1548feb2-5a6a-43e6-ab44-81252056193e</td>\n",
       "      <td>1</td>\n",
       "      <td>GPT-3</td>\n",
       "      <td>CHATGPT</td>\n",
       "      <td>ChatGPT is based on the GPT architecture, spec...</td>\n",
       "      <td>7.0</td>\n",
       "      <td>14</td>\n",
       "      <td>[ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>e538721d-c023-4994-b918-1efece80ea7e</td>\n",
       "      <td>2</td>\n",
       "      <td>GPT-3</td>\n",
       "      <td>BERT</td>\n",
       "      <td>Both BERT and GPT-3 are pre-trained language m...</td>\n",
       "      <td>6.0</td>\n",
       "      <td>18</td>\n",
       "      <td>[ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>063a2941-df53-4b5c-a66a-45e60bdba604</td>\n",
       "      <td>3</td>\n",
       "      <td>GPT-3</td>\n",
       "      <td>REINFORCEMENT LEARNING FROM HUMAN FEEDBACK (RLHF)</td>\n",
       "      <td>RLHF is used in training GPT-3 to refine its o...</td>\n",
       "      <td>7.0</td>\n",
       "      <td>13</td>\n",
       "      <td>[ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7e0cd0b4-4688-434f-a194-f2b9ce397a94</td>\n",
       "      <td>4</td>\n",
       "      <td>GPT-3</td>\n",
       "      <td>PROMPT ENGINEERING</td>\n",
       "      <td>Prompt engineering is a technique used to guid...</td>\n",
       "      <td>6.0</td>\n",
       "      <td>14</td>\n",
       "      <td>[ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     id  human_readable_id source  \\\n",
       "0  b895553a-f860-4d15-bba2-a42f1464e810                  0  GPT-3   \n",
       "1  1548feb2-5a6a-43e6-ab44-81252056193e                  1  GPT-3   \n",
       "2  e538721d-c023-4994-b918-1efece80ea7e                  2  GPT-3   \n",
       "3  063a2941-df53-4b5c-a66a-45e60bdba604                  3  GPT-3   \n",
       "4  7e0cd0b4-4688-434f-a194-f2b9ce397a94                  4  GPT-3   \n",
       "\n",
       "                                              target  \\\n",
       "0                                              GPT-4   \n",
       "1                                            CHATGPT   \n",
       "2                                               BERT   \n",
       "3  REINFORCEMENT LEARNING FROM HUMAN FEEDBACK (RLHF)   \n",
       "4                                 PROMPT ENGINEERING   \n",
       "\n",
       "                                         description  weight  combined_degree  \\\n",
       "0  GPT-4 is an advanced version of GPT-3, buildin...     8.0               20   \n",
       "1  ChatGPT is based on the GPT architecture, spec...     7.0               14   \n",
       "2  Both BERT and GPT-3 are pre-trained language m...     6.0               18   \n",
       "3  RLHF is used in training GPT-3 to refine its o...     7.0               13   \n",
       "4  Prompt engineering is a technique used to guid...     6.0               14   \n",
       "\n",
       "                                       text_unit_ids  \n",
       "0  [ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...  \n",
       "1  [ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...  \n",
       "2  [ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...  \n",
       "3  [ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...  \n",
       "4  [ca73c495111f5cadd87e6a7a01aed66647ae6623fdf41...  "
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "relationships = pd.read_parquet('./ragtest/output/create_final_relationships.parquet')\n",
    "\n",
    "relationships.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0c77836-f9ae-4602-a4d0-760915248e0a",
   "metadata": {},
   "source": [
    "### **Community Detection & Node Embedding**\n",
    "\n",
    "<img src=\"./media/leidan.png\" width=600>\n",
    "\n",
    "After we have our basic graph with entities and relationships, we analyze its structure in two ways. Community Detection uses the [Leiden algorithm](https://en.wikipedia.org/wiki/Leiden_algorithm) to find explicit groupings in the graph, creating a hierarchy of related entities. The lower in the hierarchy, the more granular the community. Node Embedding uses [Node2Vec](https://arxiv.org/abs/1607.00653) to create vector representations of each entity, capturing implicit relationships in the graph structure. These complementary approaches let us understand both obvious connections through communities and subtle patterns through embeddings.\n",
    "\n",
    "Combining all of this with our relationships gives us our final nodes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "058d8f6d-6eb7-45fe-bf1c-e28055a683c5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>human_readable_id</th>\n",
       "      <th>title</th>\n",
       "      <th>community</th>\n",
       "      <th>level</th>\n",
       "      <th>degree</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>e3a7f24b-88b6-4481-b3a7-c35075a9671f</td>\n",
       "      <td>0</td>\n",
       "      <td>GPT-3</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>-4.875545</td>\n",
       "      <td>4.017587</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>e3a7f24b-88b6-4481-b3a7-c35075a9671f</td>\n",
       "      <td>0</td>\n",
       "      <td>GPT-3</td>\n",
       "      <td>43</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>-4.875545</td>\n",
       "      <td>4.017587</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>f55cae4e-dd0d-47a2-912b-f7680147dd31</td>\n",
       "      <td>1</td>\n",
       "      <td>GPT-4</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>-4.561064</td>\n",
       "      <td>1.505724</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>f55cae4e-dd0d-47a2-912b-f7680147dd31</td>\n",
       "      <td>1</td>\n",
       "      <td>GPT-4</td>\n",
       "      <td>46</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>-4.561064</td>\n",
       "      <td>1.505724</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>f3e3e46b-6746-45a7-9a26-1432f14c45e4</td>\n",
       "      <td>2</td>\n",
       "      <td>BERT</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>-5.710580</td>\n",
       "      <td>3.546957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>f3e3e46b-6746-45a7-9a26-1432f14c45e4</td>\n",
       "      <td>2</td>\n",
       "      <td>BERT</td>\n",
       "      <td>44</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>-5.710580</td>\n",
       "      <td>3.546957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0491a417-2e18-41c4-ae1e-3e39bf2eb98f</td>\n",
       "      <td>3</td>\n",
       "      <td>PALM</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>-5.309392</td>\n",
       "      <td>1.548029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0491a417-2e18-41c4-ae1e-3e39bf2eb98f</td>\n",
       "      <td>3</td>\n",
       "      <td>PALM</td>\n",
       "      <td>46</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>-5.309392</td>\n",
       "      <td>1.548029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2b7f14f5-d1d5-49f6-bace-46fd1767f99e</td>\n",
       "      <td>4</td>\n",
       "      <td>LLAMA</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>-6.644573</td>\n",
       "      <td>0.421999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2b7f14f5-d1d5-49f6-bace-46fd1767f99e</td>\n",
       "      <td>4</td>\n",
       "      <td>LLAMA</td>\n",
       "      <td>27</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>-6.644573</td>\n",
       "      <td>0.421999</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     id  human_readable_id  title  community  \\\n",
       "0  e3a7f24b-88b6-4481-b3a7-c35075a9671f                  0  GPT-3          8   \n",
       "1  e3a7f24b-88b6-4481-b3a7-c35075a9671f                  0  GPT-3         43   \n",
       "2  f55cae4e-dd0d-47a2-912b-f7680147dd31                  1  GPT-4          8   \n",
       "3  f55cae4e-dd0d-47a2-912b-f7680147dd31                  1  GPT-4         46   \n",
       "4  f3e3e46b-6746-45a7-9a26-1432f14c45e4                  2   BERT          8   \n",
       "5  f3e3e46b-6746-45a7-9a26-1432f14c45e4                  2   BERT         44   \n",
       "6  0491a417-2e18-41c4-ae1e-3e39bf2eb98f                  3   PALM          8   \n",
       "7  0491a417-2e18-41c4-ae1e-3e39bf2eb98f                  3   PALM         46   \n",
       "8  2b7f14f5-d1d5-49f6-bace-46fd1767f99e                  4  LLAMA          3   \n",
       "9  2b7f14f5-d1d5-49f6-bace-46fd1767f99e                  4  LLAMA         27   \n",
       "\n",
       "   level  degree         x         y  \n",
       "0      0      12 -4.875545  4.017587  \n",
       "1      1      12 -4.875545  4.017587  \n",
       "2      0       8 -4.561064  1.505724  \n",
       "3      1       8 -4.561064  1.505724  \n",
       "4      0       6 -5.710580  3.546957  \n",
       "5      1       6 -5.710580  3.546957  \n",
       "6      0       3 -5.309392  1.548029  \n",
       "7      1       3 -5.309392  1.548029  \n",
       "8      0       4 -6.644573  0.421999  \n",
       "9      1       4 -6.644573  0.421999  "
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nodes = pd.read_parquet('./ragtest/output/create_final_nodes.parquet')\n",
    "\n",
    "nodes.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "457b12c9-fc5c-47cf-a597-6c6bfb3177ee",
   "metadata": {},
   "source": [
    "At this step the graph is effectively created, however we can introduce a few extra steps that will allow us to do some advanced retrieval."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62d8ed95-677b-405e-82d3-6fb1a3f1917c",
   "metadata": {},
   "source": [
    "### Community Report Generation & Summarization\n",
    "\n",
    "Now that we have clear community grouping, we can aggregate the main concepts across hierarchical node communities with another generation step, and a shorthand summary of that summary. Similar to the nodes, these summaries are also ran through an embedding model and stored in a vector store."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "702e5559-ed4e-4964-bf72-113912974102",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>human_readable_id</th>\n",
       "      <th>community</th>\n",
       "      <th>parent</th>\n",
       "      <th>level</th>\n",
       "      <th>title</th>\n",
       "      <th>summary</th>\n",
       "      <th>full_content</th>\n",
       "      <th>rank</th>\n",
       "      <th>rank_explanation</th>\n",
       "      <th>findings</th>\n",
       "      <th>full_content_json</th>\n",
       "      <th>period</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a85d59a64a054114982b1ce6e1ced591</td>\n",
       "      <td>61</td>\n",
       "      <td>61</td>\n",
       "      <td>32</td>\n",
       "      <td>2</td>\n",
       "      <td>Amazon Bedrock and AI Model Providers</td>\n",
       "      <td>The community is centered around Amazon Bedroc...</td>\n",
       "      <td># Amazon Bedrock and AI Model Providers\\n\\nThe...</td>\n",
       "      <td>8.5</td>\n",
       "      <td>The impact severity rating is high due to Amaz...</td>\n",
       "      <td>[{'explanation': 'Amazon Bedrock is a pivotal ...</td>\n",
       "      <td>{\\n    \"title\": \"Amazon Bedrock and AI Model P...</td>\n",
       "      <td>2024-12-18</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>6aafc6eeddd848bc8ffbfb9177790c26</td>\n",
       "      <td>62</td>\n",
       "      <td>62</td>\n",
       "      <td>32</td>\n",
       "      <td>2</td>\n",
       "      <td>AWS and SageMaker JumpStart</td>\n",
       "      <td>The community is centered around Amazon Web Se...</td>\n",
       "      <td># AWS and SageMaker JumpStart\\n\\nThe community...</td>\n",
       "      <td>8.5</td>\n",
       "      <td>The impact severity rating is high due to AWS'...</td>\n",
       "      <td>[{'explanation': 'Amazon Web Services (AWS) is...</td>\n",
       "      <td>{\\n    \"title\": \"AWS and SageMaker JumpStart\",...</td>\n",
       "      <td>2024-12-18</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>e13e3ed0a0b74fd090319957ae9f3e1e</td>\n",
       "      <td>14</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>PPO for LLM Alignment and Reinforcement Learni...</td>\n",
       "      <td>The community centers around the study 'PPO fo...</td>\n",
       "      <td># PPO for LLM Alignment and Reinforcement Lear...</td>\n",
       "      <td>7.5</td>\n",
       "      <td>The impact severity rating is high due to the ...</td>\n",
       "      <td>[{'explanation': 'The study 'PPO for LLM Align...</td>\n",
       "      <td>{\\n    \"title\": \"PPO for LLM Alignment and Rei...</td>\n",
       "      <td>2024-12-18</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>828baab1461b439ea71203ad8fd0aae5</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>HuggingFace and Advanced NLP Tools</td>\n",
       "      <td>The community is centered around HuggingFace, ...</td>\n",
       "      <td># HuggingFace and Advanced NLP Tools\\n\\nThe co...</td>\n",
       "      <td>8.5</td>\n",
       "      <td>The impact severity rating is high due to Hugg...</td>\n",
       "      <td>[{'explanation': 'HuggingFace is a prominent e...</td>\n",
       "      <td>{\\n    \"title\": \"HuggingFace and Advanced NLP ...</td>\n",
       "      <td>2024-12-18</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>791da6e7031e45228442b277e7d912c6</td>\n",
       "      <td>16</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>OpenAI and AI Development Platforms</td>\n",
       "      <td>The community is centered around OpenAI, a lea...</td>\n",
       "      <td># OpenAI and AI Development Platforms\\n\\nThe c...</td>\n",
       "      <td>8.5</td>\n",
       "      <td>The impact severity rating is high due to the ...</td>\n",
       "      <td>[{'explanation': 'OpenAI is a central entity i...</td>\n",
       "      <td>{\\n    \"title\": \"OpenAI and AI Development Pla...</td>\n",
       "      <td>2024-12-18</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 id  human_readable_id  community  parent  \\\n",
       "0  a85d59a64a054114982b1ce6e1ced591                 61         61      32   \n",
       "1  6aafc6eeddd848bc8ffbfb9177790c26                 62         62      32   \n",
       "2  e13e3ed0a0b74fd090319957ae9f3e1e                 14         14       0   \n",
       "3  828baab1461b439ea71203ad8fd0aae5                 15         15       0   \n",
       "4  791da6e7031e45228442b277e7d912c6                 16         16       0   \n",
       "\n",
       "   level                                              title  \\\n",
       "0      2              Amazon Bedrock and AI Model Providers   \n",
       "1      2                        AWS and SageMaker JumpStart   \n",
       "2      1  PPO for LLM Alignment and Reinforcement Learni...   \n",
       "3      1                 HuggingFace and Advanced NLP Tools   \n",
       "4      1                OpenAI and AI Development Platforms   \n",
       "\n",
       "                                             summary  \\\n",
       "0  The community is centered around Amazon Bedroc...   \n",
       "1  The community is centered around Amazon Web Se...   \n",
       "2  The community centers around the study 'PPO fo...   \n",
       "3  The community is centered around HuggingFace, ...   \n",
       "4  The community is centered around OpenAI, a lea...   \n",
       "\n",
       "                                        full_content  rank  \\\n",
       "0  # Amazon Bedrock and AI Model Providers\\n\\nThe...   8.5   \n",
       "1  # AWS and SageMaker JumpStart\\n\\nThe community...   8.5   \n",
       "2  # PPO for LLM Alignment and Reinforcement Lear...   7.5   \n",
       "3  # HuggingFace and Advanced NLP Tools\\n\\nThe co...   8.5   \n",
       "4  # OpenAI and AI Development Platforms\\n\\nThe c...   8.5   \n",
       "\n",
       "                                    rank_explanation  \\\n",
       "0  The impact severity rating is high due to Amaz...   \n",
       "1  The impact severity rating is high due to AWS'...   \n",
       "2  The impact severity rating is high due to the ...   \n",
       "3  The impact severity rating is high due to Hugg...   \n",
       "4  The impact severity rating is high due to the ...   \n",
       "\n",
       "                                            findings  \\\n",
       "0  [{'explanation': 'Amazon Bedrock is a pivotal ...   \n",
       "1  [{'explanation': 'Amazon Web Services (AWS) is...   \n",
       "2  [{'explanation': 'The study 'PPO for LLM Align...   \n",
       "3  [{'explanation': 'HuggingFace is a prominent e...   \n",
       "4  [{'explanation': 'OpenAI is a central entity i...   \n",
       "\n",
       "                                   full_content_json      period  size  \n",
       "0  {\\n    \"title\": \"Amazon Bedrock and AI Model P...  2024-12-18     9  \n",
       "1  {\\n    \"title\": \"AWS and SageMaker JumpStart\",...  2024-12-18     2  \n",
       "2  {\\n    \"title\": \"PPO for LLM Alignment and Rei...  2024-12-18     7  \n",
       "3  {\\n    \"title\": \"HuggingFace and Advanced NLP ...  2024-12-18     7  \n",
       "4  {\\n    \"title\": \"OpenAI and AI Development Pla...  2024-12-18     7  "
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "community_reports = pd.read_parquet('./ragtest/output/create_final_community_reports.parquet')\n",
    "\n",
    "community_reports.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "0ca51c33-92db-47c1-9f00-0a8e934332fe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Amazon Bedrock and AI Model Providers\n",
      "\n",
      "The community is centered around Amazon Bedrock, a service by AWS that facilitates access to foundation models from various AI innovators. Key entities include AI21 Labs, Anthropic, Cohere, Mistral AI, and Stability AI, all of which provide models accessible through Amazon Bedrock. The service integrates with AWS infrastructure, including AWS Lambda and AWS SageMaker, to support scalable AI model deployment.\n",
      "\n",
      "## Amazon Bedrock as a central service\n",
      "\n",
      "Amazon Bedrock is a pivotal service within the AWS ecosystem, designed to simplify access to high-performing foundation models for generative AI applications. It integrates seamlessly with other AWS services, such as Amazon S3, AWS Lambda, and AWS SageMaker, to facilitate the fine-tuning and deployment of AI models. This integration underscores its importance in the AI landscape, providing a comprehensive suite of tools for scalable AI model deployment [Data: Entities (206); Relationships (281, 326, 327)].\n",
      "\n",
      "## AI21 Labs' contribution to Amazon Bedrock\n",
      "\n",
      "AI21 Labs is one of the key providers of foundation models available through Amazon Bedrock. The company's advanced AI models contribute significantly to the capabilities offered by Amazon Bedrock, enhancing its utility for natural language processing tasks. This partnership highlights the collaborative nature of the AI community in advancing technology through shared resources and expertise [Data: Entities (267); Relationships (318)].\n",
      "\n",
      "## Anthropic's role in AI safety and research\n",
      "\n",
      "Anthropic is an AI safety and research company that provides foundation models accessible through Amazon Bedrock. Its focus on AI safety is crucial in the development of responsible AI technologies, ensuring that the models deployed via Amazon Bedrock adhere to ethical standards. This relationship emphasizes the importance of integrating safety considerations into AI development [Data: Entities (268); Relationships (319)].\n",
      "\n",
      "## Cohere's NLP models in Amazon Bedrock\n",
      "\n",
      "Cohere offers natural language processing models that are part of the foundation models available through Amazon Bedrock. These models enhance the service's capabilities in processing and understanding human language, making it a valuable tool for businesses and developers seeking to implement NLP solutions. Cohere's involvement underscores the diversity of AI models supported by Amazon Bedrock [Data: Entities (269); Relationships (320)].\n",
      "\n",
      "## Integration with AWS Lambda and AWS SageMaker\n",
      "\n",
      "Amazon Bedrock's integration with AWS Lambda and AWS SageMaker is a key feature that supports the deployment and management of AI models. AWS Lambda provides serverless computing capabilities, allowing for efficient resource management, while AWS SageMaker offers tools for building, training, and deploying machine learning models. This integration facilitates a seamless workflow for AI model development and deployment, enhancing the overall efficiency and scalability of AI applications [Data: Entities (276, 277); Relationships (326, 327)].\n"
     ]
    }
   ],
   "source": [
    "print(community_reports[\"full_content\"][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "42c3e76b-e59f-4577-b1a0-3b07f0ebb499",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The community is centered around Amazon Bedrock, a service by AWS that facilitates access to foundation models from various AI innovators. Key entities include AI21 Labs, Anthropic, Cohere, Mistral AI, and Stability AI, all of which provide models accessible through Amazon Bedrock. The service integrates with AWS infrastructure, including AWS Lambda and AWS SageMaker, to support scalable AI model deployment.\n"
     ]
    }
   ],
   "source": [
    "print(community_reports[\"summary\"][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb5243d1-e1d2-4525-ab3c-e1457db2eea7",
   "metadata": {},
   "source": [
    "### The Final Graph!\n",
    "\n",
    "<img src=\"./media/ghraphrag_viz.svg\" width=800>\n",
    "\n",
    "*[Full Size PDF](./ghraphrag_viz.pdf)*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "172bf775-ee3b-4d4a-a973-317f1681b8af",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## GraphRAG Retrieval\n",
    "\n",
    "<img src=\"./media/kg_retrieval.png\" width=600>\n",
    "\n",
    "*[Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://arxiv.org/pdf/2306.08302)*\n",
    "\n",
    "With our knowledge graph constructed, and hierarchichal communities delineated, we can now perform multiple types of search that can both take advantage of the graph structure, and multiple levels of specificity across our communities. Specifically:\n",
    "\n",
    "1. **Global Search**: Uses the LLM Generated community reports from a specified level of the graph's community hierarchy as context data to generate response.\n",
    "2. **Local Search**: Combines structured data from the knowledge graph with unstructured data from the input document(s) to augment the LLM context with relevant entity information.\n",
    "3. **Drift Search**: Dynamic Reasoning and Inference with Flexible Traversal, an approach to local search queries by including community information in the search process, thus combining global and local search."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f14a5823-048b-4a54-9da5-51305b8a1c7a",
   "metadata": {},
   "source": [
    "**GraphRAG Retrieval Function**\n",
    "\n",
    "*Note: Wrapping the [GraphRAG CLI tool](https://microsoft.github.io/graphrag/cli/) as a function here instead of using their [library](https://microsoft.github.io/graphrag/examples_notebooks/api_overview/) for an easier example. As such, notebook needs to be running in the same GraphRAG environment/kernal.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "ae01778e-518f-42fe-929e-dd2ef63a8753",
   "metadata": {},
   "outputs": [],
   "source": [
    "import subprocess\n",
    "import shlex\n",
    "from typing import Optional\n",
    "\n",
    "def query_graphrag(\n",
    "    query: str,\n",
    "    method: str = \"global\",\n",
    "    root_path: str = \"./ragtest\",\n",
    "    timeout: Optional[int] = None,\n",
    "    community_level: int = 2,\n",
    "    dynamic_community_selection: bool = False\n",
    ") -> str:\n",
    "    \"\"\"\n",
    "    Execute a GraphRAG query using the CLI tool.\n",
    "    \n",
    "    Args:\n",
    "        query (str): The query string to process\n",
    "        method (str): Query method (e.g., \"global\", \"local\", or \"drift\")\n",
    "        root_path (str): Path to the root directory\n",
    "        timeout (int, optional): Timeout in seconds for the command\n",
    "        community_level (int): The community level in the Leiden community hierarchy (default: 2)\n",
    "        dynamic_community_selection (bool): Whether to use global search with dynamic community selection (default: False)\n",
    "    \n",
    "    Returns:\n",
    "        str: The output from GraphRAG\n",
    "        \n",
    "    Raises:\n",
    "        subprocess.CalledProcessError: If the command fails\n",
    "        subprocess.TimeoutExpired: If the command times out\n",
    "        ValueError: If community_level is negative\n",
    "    \"\"\"\n",
    "    # Validate community level\n",
    "    if community_level < 0:\n",
    "        raise ValueError(\"Community level must be non-negative\")\n",
    "    \n",
    "    # Construct the base command\n",
    "    command = [\n",
    "        'graphrag', 'query',\n",
    "        '--root', root_path,\n",
    "        '--method', method,\n",
    "        '--query', query,\n",
    "        '--community-level', str(community_level)\n",
    "    ]\n",
    "    \n",
    "    # Add dynamic community selection flag if enabled\n",
    "    if dynamic_community_selection:\n",
    "        command.append('--dynamic-community-selection')\n",
    "    \n",
    "    try:\n",
    "        # Execute the command and capture output\n",
    "        result = subprocess.run(\n",
    "            command,\n",
    "            capture_output=True,\n",
    "            text=True,\n",
    "            timeout=timeout\n",
    "        )\n",
    "        \n",
    "        # Check if the command was successful\n",
    "        result.check_returncode()\n",
    "        \n",
    "        return result.stdout.strip()\n",
    "        \n",
    "    except subprocess.CalledProcessError as e:\n",
    "        error_message = f\"Command failed with exit code {e.returncode}\\nError: {e.stderr}\"\n",
    "        raise subprocess.CalledProcessError(\n",
    "            e.returncode,\n",
    "            e.cmd,\n",
    "            output=e.output,\n",
    "            stderr=error_message\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c02da6ea-e0ad-41d4-982a-f9058184add0",
   "metadata": {},
   "source": [
    "### Local Search\n",
    "\n",
    "<img src=\"./media/local_search.png\" width=900>\n",
    "\n",
    "The GraphRAG approach to local search is the most similar to regular semantic RAG search. It combines structured data from the knowledge graph with unstructured data from the input documents to augment the LLM context with relevant entity information. In essence, we are going to first search for relevant entities to the query using semantic search. These become the entry points on our graph that we can now traverse. Starting at these points, we look at connected chunks of text, community reports, other entities, and relationships between them. All of the data retrieved is filtered and ranked to fit into a pre-defined context window."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "ad477f20-1019-4da6-a70a-9cc9e362fb03",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Query result:\n",
      "INFO: Vector Store Args: {\n",
      "    \"type\": \"lancedb\",\n",
      "    \"db_uri\": \"/Users/adamlucek/Desktop/github/GraphRAG/ragtest/output/lancedb\",\n",
      "    \"container_name\": \"==== REDACTED ====\",\n",
      "    \"overwrite\": true\n",
      "}\n",
      "creating llm client with {'api_key': 'REDACTED,len=51', 'type': \"openai_chat\", 'encoding_model': 'cl100k_base', 'model': 'gpt-4o', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'n': 1, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'audience': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25, 'responses': None}\n",
      "creating embedding llm client with {'api_key': 'REDACTED,len=51', 'type': \"openai_embedding\", 'encoding_model': 'cl100k_base', 'model': 'text-embedding-3-small', 'max_tokens': 4000, 'temperature': 0, 'top_p': 1, 'n': 1, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'audience': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25, 'responses': None}\n",
      "\n",
      "SUCCESS: Local Search Response:\n",
      "When a company is deciding between Retrieval-Augmented Generation (RAG), fine-tuning, and various Parameter-Efficient Fine-Tuning (PEFT) approaches, several factors must be considered, including the specific application requirements, available resources, and desired outcomes. Each method offers distinct advantages and is suited to different scenarios.\n",
      "\n",
      "### Retrieval-Augmented Generation (RAG)\n",
      "\n",
      "RAG is particularly beneficial for applications that require integrating external data to enhance the accuracy and relevance of generated content. It is ideal for scenarios where the model needs to access up-to-date or domain-specific information without extensive retraining. For instance, RAG is effectively used in question-and-answer systems, customer support automation, and summarization tasks, where the ability to retrieve and incorporate external data can significantly improve performance [Data: Reports (20); Entities (20); Relationships (14, 21, 22, 23)]. Companies might choose RAG when they need to maintain high precision and adaptability in dynamic environments, such as customer service or technical support, where the context and information are constantly evolving.\n",
      "\n",
      "### Fine-Tuning\n",
      "\n",
      "Fine-tuning involves adapting a pre-trained model to perform better on specific tasks by updating its parameters with domain-specific data. This approach is essential when a company needs to tailor a model to a particular application or domain, ensuring that it can handle specific tasks with high accuracy. Fine-tuning is a comprehensive process that includes stages such as dataset preparation, model initialization, and evaluation, making it suitable for applications where precise model adaptation is crucial [Data: Reports (19); Entities (17); Relationships (10, 12, 24, 27, 29, 274, 275, 276, 277); Sources (3, 12, 13)]. Companies might opt for fine-tuning when they have the resources to manage the computational demands and when the task requires a high degree of customization.\n",
      "\n",
      "### Parameter-Efficient Fine-Tuning (PEFT)\n",
      "\n",
      "PEFT methods, such as LoRA and QLoRA, offer a more resource-efficient alternative to traditional fine-tuning by updating only a small subset of model parameters. This approach is particularly advantageous in scenarios where computational resources are limited, as it reduces the memory and processing requirements while maintaining performance levels comparable to full fine-tuning [Data: Entities (25, 104, 110, 130); Relationships (24, 137, 140, 104); Sources (6, 15)]. PEFT is suitable for companies looking to optimize the financial and environmental costs associated with fine-tuning large language models, especially in low-data scenarios or when deploying models on less powerful hardware.\n",
      "\n",
      "### Decision-Making Considerations\n",
      "\n",
      "In choosing between these methods, companies should evaluate the specific needs of their application, the availability of computational resources, and the importance of model adaptability and precision. RAG is ideal for applications requiring real-time data integration, fine-tuning is best for highly customized tasks, and PEFT is suitable for resource-constrained environments. By aligning the choice of method with their strategic goals and operational constraints, companies can effectively enhance their AI capabilities and achieve desired outcomes.\n"
     ]
    }
   ],
   "source": [
    "result = query_graphrag(\n",
    "    query=\"How does a company choose between RAG, fine-tuning, and different PEFT approaches?\",\n",
    "    method=\"local\"\n",
    ")\n",
    "print(\"Query result:\")\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b8368cd-25bb-4460-802f-8ed76a39bb2d",
   "metadata": {},
   "source": [
    "### Global Search\n",
    "\n",
    "<img src=\"./media/global_search.png\" width=1000>\n",
    "\n",
    "Through the semantic clustering of communities during the indexxing process outlined above we created community reports as summaries of high level themes across these groupings. Having this community summary data at various levels allows us to do something that traditional RAG performs poorly at, answering queries about broad themes and ideas across our unstructured data.\n",
    "\n",
    "To capture as much broad information as possible in an efficient manner, GraphRAG implements a [map reduce](https://en.wikipedia.org/wiki/MapReduce) approach. Given a query, relevant community node reports at a specific hierarchical level are retrieved. These are shuffled and chunked, where each chunk is used to generate a list of points that each have their own \"importance score\". These intermediate points are ranked and filtered, attempting to maintain the most important points. These become the aggregate intermediary response, which is passed to the LLM as the context for the final response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "77b59560-c02f-4c2e-a969-8042caef03bf",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Query result:\n",
      "creating llm client with {'api_key': 'REDACTED,len=51', 'type': \"openai_chat\", 'encoding_model': 'cl100k_base', 'model': 'gpt-4o', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'n': 1, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'audience': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25, 'responses': None}\n",
      "\n",
      "SUCCESS: Global Search Response:\n",
      "### Choosing Between RAG, Fine-Tuning, and PEFT Approaches\n",
      "\n",
      "When a company is deciding between Retrieval-Augmented Generation (RAG), fine-tuning, and Parameter-Efficient Fine-Tuning (PEFT) approaches, several key factors must be considered. These factors include the specific requirements of the application, the need for external data integration, computational resource constraints, and the desired level of model adaptation and performance.\n",
      "\n",
      "#### Application Requirements\n",
      "\n",
      "The choice largely depends on the specific needs of the application. For instance, if the application requires high precision and adaptability, such as in customer support or information retrieval systems, RAG may be the preferred choice. RAG enhances large language models by integrating external data, allowing for more accurate and contextually relevant content generation without extensive fine-tuning [Data: Reports (19, 20, 35, 21, 36, 38, 60)].\n",
      "\n",
      "#### Computational Resources\n",
      "\n",
      "Computational resources are another critical consideration. Fine-tuning involves further training pre-trained models to improve their performance on specific tasks by updating the model's parameters using a smaller, domain-specific dataset. This process can be resource-intensive but is essential for adapting models to targeted applications and domains [Data: Reports (19, 45, 31, 60)].\n",
      "\n",
      "In contrast, PEFT approaches, such as LoRA and QLoRA, are designed to enhance memory efficiency and reduce computational costs during the fine-tuning process. These techniques allow for fine-tuning on less powerful hardware while maintaining performance levels comparable to traditional methods. PEFT is particularly useful for fine-tuning multimodal models and deploying NLP models on mobile devices [Data: Reports (36, 38, 19, 35)].\n",
      "\n",
      "#### Model Customization and Contextual Relevance\n",
      "\n",
      "The desired level of model customization and contextual relevance also plays a significant role. Fine-tuning is crucial for enhancing the performance of large language models (LLMs) on specific tasks, addressing challenges such as scalability, memory requirements, and resource efficiency. It allows for the customization of models to meet specialized needs and ensures they perform optimally in their intended environments [Data: Reports (45, 31, 60)].\n",
      "\n",
      "RAG systems, on the other hand, improve the accuracy and contextuality of the outputs produced by LLMs, making them suitable for applications where context is crucial [Data: Reports (20, 21, 53)].\n",
      "\n",
      "### Conclusion\n",
      "\n",
      "Ultimately, the decision between RAG, fine-tuning, and PEFT approaches should be guided by the specific needs and constraints of the company, such as the importance of context relevance, the need for task-specific optimization, and resource availability. By carefully evaluating these factors, companies can select the most appropriate method to achieve their desired outcomes [Data: Reports (19, 20, 35, 21, 36, 38, 60, 53, 45)].\n"
     ]
    }
   ],
   "source": [
    "result = query_graphrag(\n",
    "    query=\"How does a company choose between RAG, fine-tuning, and different PEFT approaches?\",\n",
    "    method=\"global\"\n",
    ")\n",
    "print(\"Query result:\")\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c35d9948-ba7e-49af-ac49-62b835d62174",
   "metadata": {},
   "source": [
    "### DRIFT Search\n",
    "\n",
    "<img src=\"./media/drift_search.png\" width=1000>\n",
    "\n",
    "[Dynamic Reasoning and Inference with Flexible Traversal](https://www.microsoft.com/en-us/research/blog/introducing-drift-search-combining-global-and-local-search-methods-to-improve-quality-and-efficiency/), or DRIFT, is a novel GraphRAG concept introduced by Microsoft as an approach to local search queries that include community information in the search process.\n",
    "\n",
    "The user's query is initially processed through [Hypothetical Document Embedding (HyDE)](https://arxiv.org/pdf/2212.10496), which creates a hypothetical document similar to those found in the graph already, but using the user's topic query. This document is embedded and used for semantic retrieval of the top-k relevant community reports. From these matches, we generate an initial answer along with several follow-up questions as a lightweight version of global search. They refer to this as the primer.\n",
    "\n",
    "Once this primer phase is complete, we execute local searches for each follow-up question generated. Each local search produces both intermediate answers and new follow-up questions, creating a refinement loop. This loop runs for two iterations (noted future research planned to develop reward functions for smarter termination). An important note that makes these local searches unique is that they are informed by both community-level knowledge and detailed entity/relationship data. This allows the DRIFT process to find relevant information even when the initial query diverges from the indexing persona, and it can adapt its approach based on emerging information during the search.\n",
    "\n",
    "The final output is structured as a hierarchy of questions and answers, ranked by their relevance to the original query. Map reduce is used again with an equal weighting on all intermediate answers, then passed to the language model for a final response. DRIFT cleverly combines global and local search with guided exploration to provide both broad context and specific details in responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "0ffcac67-ab84-4dd7-bf7e-cf4b2a4e7cbd",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Query result:\n",
      "INFO: Vector Store Args: {\n",
      "    \"type\": \"lancedb\",\n",
      "    \"db_uri\": \"/Users/adamlucek/Desktop/github/GraphRAG/ragtest/output/lancedb\",\n",
      "    \"container_name\": \"==== REDACTED ====\",\n",
      "    \"overwrite\": true\n",
      "}\n",
      "creating llm client with {'api_key': 'REDACTED,len=51', 'type': \"openai_chat\", 'encoding_model': 'cl100k_base', 'model': 'gpt-4o', 'max_tokens': 4000, 'temperature': 0.0, 'top_p': 1.0, 'n': 1, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'audience': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25, 'responses': None}\n",
      "creating embedding llm client with {'api_key': 'REDACTED,len=51', 'type': \"openai_embedding\", 'encoding_model': 'cl100k_base', 'model': 'text-embedding-3-small', 'max_tokens': 4000, 'temperature': 0, 'top_p': 1, 'n': 1, 'frequency_penalty': 0.0, 'presence_penalty': 0.0, 'request_timeout': 180.0, 'api_base': None, 'api_version': None, 'organization': None, 'proxy': None, 'audience': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25, 'responses': None}\n",
      "\n",
      "SUCCESS: DRIFT Search Response:\n",
      "# Understanding the Choice Between RAG, Fine-Tuning, and PEFT Approaches\n",
      "\n",
      "When a company is faced with the decision to choose between Retrieval-Augmented Generation (RAG), fine-tuning, and various Parameter-Efficient Fine-Tuning (PEFT) approaches, several factors must be taken into account, determined by the specific application requirements and resource availability.\n",
      "\n",
      "## Retrieving Capabilities with RAG\n",
      "RAG is particularly advantageous in scenarios where it is crucial to integrate external information into language models for enhanced accuracy and relevance. It is highly recommended for applications like question and answer systems, customer support, and summarization tasks, where real-time context retrieval from large datasets improves response precision without needing extensive and expensive model retraining. The strength of RAG lies in its ability to quickly adapt to various knowledge domains by leveraging existing data without modifying the underlying language model significantly.\n",
      "\n",
      "## The Focus and Depth of Fine-Tuning\n",
      "Traditional fine-tuning is the go-to choice when a clear, defined improvement in task performance is needed using domain-specific datasets. It involves re-calibrating a pre-trained model’s parameters to cater precisely to industry-specific or task-specific requirements. This approach can enhance performance metrics significantly but incurs high computational costs and demands substantial amounts of fine-tuning data, making it ideal for well-funded projects aiming for precise task optimization.\n",
      "\n",
      "## Resource-Efficiency with PEFT Techniques\n",
      "On the other end of the spectrum, PEFT approaches such as DORA and LoRA present themselves as highly resource-efficient alternatives to standard fine-tuning. They are suitable for companies with limited computational resources but still prioritize performance enhancements. By selectively adjusting additional smaller model parameters, PEFT methods allow substantial flexibility and significant model adaptability for various applications without the full overhead of conventional fine-tuning.\n",
      "\n",
      "Ultimately, the decision for a company involves balancing resource availability (both in terms of data and computing power), desired performance efficiency, and the specific features of the task at hand. Companies with broad application needs across different domains might favor RAG's flexibility; domain-specific improvements would benefit more from deep fine-tuning, while constrained environments should consider PEFT methods for their adaptability and efficiency.\n",
      "\n",
      "# Understanding the Choice Between RAG, Fine-Tuning, and PEFT Approaches\n",
      "\n",
      "When choosing between Retrieval-Augmented Generation (RAG), fine-tuning, and Parameter-Efficient Fine-Tuning (PEFT) approaches, businesses must evaluate their specific needs in terms of model performance, resource constraints, and flexibility. Each approach involves distinct methods for optimally leveraging machine learning models and NLP systems.\n",
      "\n",
      "**Retrieval-Augmented Generation (RAG):** RAG is a strategy combining retrieval and generative models to enhance the context provided to language models like GPT-3. This approach allows models to answer queries more accurately by accessing external knowledge bases. It is particularly beneficial when dealing with tasks requiring extensive context or up-to-date information. Companies would favor RAG when the model needs to maintain relevance in dynamic information environments or when the contextual accuracy is crucial.\n",
      "\n",
      "**Fine-Tuning:** This traditional method involves adjusting the entire model's parameters to optimize performance on a specific task. Fine-tuning is often resource-intensive, requiring significant computational power and time, which can prove a bottleneck for large models or organizations with limited resources. It produces highly task-specific outcomes, making it an ideal choice when high accuracy is paramount and resources are abundant. Fine-tuning is suitable for specialized tasks where the nuances of data need careful embedding into the model.\n",
      "\n",
      "**Parameter-Efficient Fine-Tuning (PEFT):** Techniques like QLoRA and LoRA are designed to address the resource constraints associated with fine-tuning. By adjusting only a subset of parameters, these methods provide a more efficient approach, maintaining model performance while requiring less computational power. Companies might choose PEFT when deploying large models on resource-constrained devices, like mobile platforms, or when exploring multiple tasks with shared resources. PEFT's importance is growing in real-world applications where cost and efficiency are central concerns.\n",
      "\n",
      "In summary, choosing between these approaches depends on an organization’s priorities: RAG offers dynamic and contextual adaptability; fine-tuning promises high precision at the cost of resources; and PEFT balances resource demands with performance, making it suitable for scalable deployment. Organizations are encouraged to deeply understand their specific contextual and technical requirements to select the most appropriate approach.\n",
      "\n",
      "## How Companies Choose Between RAG, Fine-Tuning, and PEFT Approaches\n",
      "\n",
      "When deciding between Retrieval-Augmented Generation (RAG), fine-tuning, and Parameter-Efficient Fine-Tuning (PEFT) techniques, companies must consider a variety of factors related to their unique needs and resources. Each approach serves distinct purposes and offers specific advantages, making them suitable for different scenarios.\n",
      "\n",
      "### Retrieval-Augmented Generation (RAG)\n",
      "RAG is particularly beneficial when a company requires a model to access a vast external data set to create highly contextual and informed responses. This method is preferred when the task involves processing or retrieving large amounts of dynamic information. The power of RAG lies in its ability to provide more comprehensive answers by leveraging external data sources, enhancing the model's output quality. As such, organizations where information retrieval and integration are crucial components of the application, such as in customer support systems or knowledge-based industries, may find RAG an optimal choice.\n",
      "\n",
      "### Fine-Tuning\n",
      "Fine-tuning a large language model involves adjusting its parameters based on specific tasks or domain data to improve its performance on said tasks. This method is ideal for companies aiming to develop a bespoke solution tailored to precise needs or niche applications. Fine-tuning a model can significantly increase its utility in specialized fields such as legal research or medical analysis, where the accuracy and specificity of information are critical. Though resource-intensive, the specificity in outcome makes traditional fine-tuning valuable for businesses that require high degrees of customization.\n",
      "\n",
      "### Parameter-Efficient Fine-Tuning (PEFT)\n",
      "PEFT techniques, such as LoRA and adapter-based methods, offer a way to enhance large language models without the typical computational cost associated with traditional fine-tuning. This strategy updates only a small subset of a model's parameters, thus making it a cost-effective alternative for companies with limited resources or those that need to frequently update models with new data. PEFT is advantageous for businesses that need flexibility without a significant investment in computational power, popular among startups or organizations exploring new avenues without extensive AI infrastructure.\n",
      "\n",
      "In summary, a company's choice between RAG, fine-tuning, and PEFT depends on their specific requirements, the nature and volume of data they interact with, and the computational resources at their disposal. By aligning their strategy with the strengths of each approach, businesses can optimize their use of AI technologies to meet their unique operational goals.\n",
      "\n",
      "# Evaluating Company Choices Among RAG, Fine-Tuning, and PEFT Approaches\n",
      "\n",
      "When a company considers enhancing its machine learning models, particularly large language models (LLMs), it encounters a range of techniques, including Retrieval-Augmented Generation (RAG), fine-tuning, and Parameter-Efficient Fine-Tuning (PEFT) methods like Half Fine-Tuning (HFT). Each of these techniques offers unique benefits and is suitable for different objectives and scenarios. Understanding the distinction and potential applications is crucial for making an informed decision.\n",
      "\n",
      "RAG is highly effective in situations where models need to provide accurate responses based on external information not inherently within the model's parameters. By integrating information retrieval techniques, RAG allows models to access a database or document store to augment their generated text, making it ideal for applications where dynamic content access and up-to-date information are critical, such as live customer service or research queries.\n",
      "\n",
      "In contrast, fine-tuning broadly involves adjusting an already pre-trained model to suit new, specific tasks or datasets. Traditional supervised fine-tuning is useful when the primary goal is to improve a model’s performance on specific tasks while leveraging existing knowledge. For example, in supervised fine-tuning, Half Fine-Tuning (HFT) offers an innovative balance by freezing a portion of the model's parameters, thus maintaining foundational knowledge while developing new competencies. This efficiency is especially important when handling models like the LLAMA 2-7B, where sustaining knowledge over several iterations is vital.\n",
      "\n",
      "PEFT techniques, such as HFT, provide a focused approach to fine-tuning by optimizing resource usage and reducing computational demands. They ensure that changes to the model preserve core functionalities while adapting to new requirements, which is critically important in environments where computational resources and model interpretability are constrained.\n",
      "\n",
      "The choice among RAG, fine-tuning, and PEFT approaches depends on factors such as the company's resource availability, existing model performance, specific use-cases, and the need for adaptability versus resource conservation. Companies prioritizing task-specific improvements with limited computational power might lean toward PEFT or HFT, whereas RAG would be preferable for scenarios needing comprehensive and current information access capabilities.\n",
      "\n",
      "Ultimately, the decision must align with strategic objectives, model lifecycle considerations, and the specific complexities of the data and questions the model is expected to handle.\n",
      "\n",
      "# Factors in Choosing Between RAG, Fine-Tuning, and PEFT Approaches\n",
      "\n",
      "When a company is deciding between Retrieval-Augmented Generation (RAG), fine-tuning, and Parameter-Efficient Fine-Tuning (PEFT) methods for large language models (LLMs), several important factors must be considered. These methods each have unique advantages and challenges that make them suitable for different scenarios.\n",
      "\n",
      "Retrieval-Augmented Generation (RAG) enhances the performance of LLMs by integrating retrieval mechanisms that bring relevant information into the context before generating results. This approach is particularly useful when the task requires the contextually relevant retrieval of information from extensive databases or when the generation task is highly dependent on external knowledge. RAG systems improve both the accuracy and the contextual relevance of LLM outputs, making them valuable for applications where precision in response to contextual queries is critical.\n",
      "\n",
      "Fine-tuning involves adjusting the LLM using a carefully prepared dataset tailored to a specific task. This process includes essential stages such as dataset preparation, instruction tuning, and topic mapping. Fine-tuning is especially advantageous when the intention is to specialize a model for a specific application, ensuring nuanced comprehension and performance improvement for the target task. It provides more control over the model's behavior, as specific datasets align the model with intended outcomes, offering enhanced reliability.\n",
      "\n",
      "Parameter-Efficient Fine-Tuning (PEFT) approaches are designed to fine-tune LLMs with fewer resource requirements compared to traditional fine-tuning. Methods like LoRA (Low-Rank Adaptation), Adapter modules, and prefix tuning are examples of PEFT techniques. They focus on adapting a small set of parameters while maintaining the majority of the LLM fixed, thus reducing computational costs and the necessary dataset size. PEFT is particularly beneficial for scenarios with constrained computational resources or when rapidly deploying updates or adaptations is necessary without retraining the entire model.\n",
      "\n",
      "Ultimately, the choice between these approaches depends on the specific requirements and constraints of the task at hand, including the need for real-time retrieval capabilities (favoring RAG), the specificity and quality of output (favoring fine-tuning), or capacity and resource efficiency (favoring PEFT).\n"
     ]
    }
   ],
   "source": [
    "result = query_graphrag(\n",
    "    query=\"How does a company choose between RAG, fine-tuning, and different PEFT approaches?\",\n",
    "    method=\"drift\"\n",
    ")\n",
    "print(\"Query result:\")\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b2e04a5-aa55-4955-b2c0-b86dd462f549",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Comparing to Regular Vector Database Retrieval\n",
    "\n",
    "<img src=\"./media/basic_retrieval.png\" width=600>\n",
    " \n",
    "To give some comparison, let's look back at traditional chunking, embedding, and similarity retrieval RAG"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e12fa20-81d9-43cc-a641-6e8ba7cb5e9b",
   "metadata": {},
   "source": [
    "**Instantiate our Database**\n",
    "\n",
    "For this we'll be using [ChromaDB](https://www.trychroma.com) with the same chunks as were loaded into our graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a361a631-4e98-47d5-9b2d-48810c2eab41",
   "metadata": {},
   "outputs": [],
   "source": [
    "import chromadb\n",
    "\n",
    "chroma_client = chromadb.PersistentClient(path=\"./notebook/chromadb\")\n",
    "paper_collection = chroma_client.get_or_create_collection(name=\"paper_collection\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a160165-6ed9-4ac9-be3b-db5bff88beb4",
   "metadata": {},
   "source": [
    "**Embed Chunks Into Collection**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24350f0f-32ef-41ea-9d7c-5e970ca172f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "i = 0\n",
    "for text in texts:\n",
    "    paper_collection.add(\n",
    "        documents=[text],\n",
    "        ids=f\"chunk_{i}\"\n",
    "    )\n",
    "    i += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f3961f0-bdce-4d1d-8d49-86355bb7d505",
   "metadata": {},
   "source": [
    "**Retrieval Function**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "841f2276-fc27-4008-9e63-6ec9d4db66ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "def chroma_retrieval(query, num_results=5):\n",
    "    results = paper_collection.query(\n",
    "        query_texts=[query],\n",
    "        n_results=num_results\n",
    "    )\n",
    "    return results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac2bb61e-5d36-47fc-b2ec-6557293f5ef4",
   "metadata": {},
   "source": [
    "**RAG Prompt & Chain**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "593a23ff-c34c-47b4-b6d8-c2ca59fad69f",
   "metadata": {},
   "outputs": [],
   "source": [
    "rag_prompt_template = \"\"\"\n",
    "Generate a response of the target length and format that responds to the user's question, summarizing all information in the input data tables appropriate for the response length and format, and incorporating any relevant general knowledge.\n",
    "\n",
    "If you don't know the answer, just say so. Do not make anything up.\n",
    "\n",
    "Do not include information where the supporting evidence for it is not provided.\n",
    "\n",
    "Context: {retrieved_docs}\n",
    "\n",
    "User Question: {query}\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "rag_prompt = ChatPromptTemplate.from_template(rag_prompt_template)\n",
    "\n",
    "rag_chain = rag_prompt | llm | StrOutputParser()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3211bbff-9927-464b-942d-964005707475",
   "metadata": {},
   "source": [
    "**RAG Function**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "e654479e-0d40-4de3-8d03-1de08b54a3fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "def chroma_rag(query):\n",
    "    retrieved_docs = chroma_retrieval(query)[\"documents\"][0]\n",
    "    response = rag_chain.invoke({\"retrieved_docs\": retrieved_docs, \"query\": query})\n",
    "    return response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9fba855e-b29c-4169-8302-b1b5cceee76c",
   "metadata": {},
   "source": [
    "**RAG Response**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "72201a27-6dac-4f3b-8ff9-717f4998470c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
       "</pre>\n"
      ],
      "text/plain": [
       "HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "When choosing between Retrieval-Augmented Generation (RAG), fine-tuning, and different Parameter-Efficient Fine-Tuning (PEFT) approaches, a company should consider several factors:\n",
      "\n",
      "1. **Data Access and Updates**: RAG is preferable for applications requiring access to external data sources or environments where data frequently updates. It provides dynamic data retrieval capabilities and is less prone to generating incorrect information.\n",
      "\n",
      "2. **Model Behavior and Domain-Specific Knowledge**: Fine-tuning is suitable when the model needs to adjust its behavior, writing style, or incorporate domain-specific knowledge. It is effective if there is ample domain-specific, labeled training data available.\n",
      "\n",
      "3. **Resource Constraints and Efficiency**: PEFT approaches like LoRA and DEFT are designed to reduce computational and resource requirements. LoRA focuses on low-rank matrices to reduce memory usage and computational load, while DEFT optimizes the fine-tuning process by focusing on the most critical data samples.\n",
      "\n",
      "4. **Task-Specific Adaptation**: If the goal is to adapt a model for specific tasks with minimal data, PEFT methods like DEFT and adapter-based techniques can be beneficial. They allow for efficient fine-tuning with fewer resources.\n",
      "\n",
      "5. **Transparency and Interpretability**: RAG systems offer more transparency and interpretability in the model’s decision-making process compared to solely fine-tuned models.\n",
      "\n",
      "6. **Scalability and Deployment**: For large-scale deployments, PEFT methods can significantly reduce computational costs by focusing on influential data samples and using surrogate models.\n",
      "\n",
      "In summary, the choice depends on the specific needs of the application, such as data access, model behavior, resource availability, and the importance of transparency and scalability.\n"
     ]
    }
   ],
   "source": [
    "response = chroma_rag(\"How does a company choose between RAG, fine-tuning, and different PEFT approaches?\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c6a630f-3268-45ec-b58f-dbb42106864a",
   "metadata": {},
   "source": [
    "---\n",
    "## Discussion\n",
    "\n",
    "**Traditional/Naive RAG:**\n",
    "\n",
    "Benefits:\n",
    "- Simpler implementation and deployment\n",
    "- Works well for straightforward information retrieval tasks\n",
    "- Good at handling unstructured text data\n",
    "- Lower computational overhead\n",
    "\n",
    "Drawbacks:\n",
    "- Loses structural information when chunking documents\n",
    "- Can break up related content during text segmentation\n",
    "- Limited ability to capture relationships between different pieces of information\n",
    "- May struggle with complex reasoning tasks requiring connecting multiple facts\n",
    "- Potential for incomplete or fragmented answers due to chunking boundaries\n",
    "\n",
    "**GraphRAG:**\n",
    "\n",
    "Benefits:\n",
    "- Preserves structural relationships and hierarchies in the knowledge\n",
    "- Better at capturing connections between related information\n",
    "- Can provide more complete and contextual answers\n",
    "- Improved retrieval accuracy by leveraging graph structure\n",
    "- Better supports complex reasoning across multiple facts\n",
    "- Can maintain document coherence better than chunk-based approaches\n",
    "- More interpretable due to explicit knowledge representation\n",
    "\n",
    "Drawbacks:\n",
    "- More complex to implement and maintain\n",
    "- Requires additional processing to construct and update knowledge graphs\n",
    "- Higher computational overhead for graph operations\n",
    "- May require domain expertise to define graph schema/structure\n",
    "- More challenging to scale to very large datasets\n",
    "- Additional storage requirements for graph structure\n",
    "\n",
    "**Key Differentiators:**\n",
    "1. Knowledge Representation: Traditional RAG treats everything as flat text chunks, while GraphRAG maintains structured relationships in a graph format\n",
    "\n",
    "2. Context Preservation: GraphRAG better preserves context and relationships between different pieces of information compared to the chunking approach of traditional RAG\n",
    "\n",
    "3. Reasoning Capability: GraphRAG enables better multi-hop reasoning and connection of related facts through graph traversal, while traditional RAG is more limited to direct retrieval\n",
    "\n",
    "4. Answer Quality: GraphRAG tends to produce more complete and coherent answers since it can access related information through graph connections rather than being limited by chunk boundaries\n",
    "\n",
    "The choice between traditional RAG and GraphRAG often depends on the specific use case, with GraphRAG being particularly valuable when maintaining relationships between information is important or when complex reasoning is required. An important note as well, GraphRAG approaches still rely on regular embedding and retrieval methods themselves. They compliment eahcother!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b8c5872-2185-46dc-aaef-2108bc490a80",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "graphrag",
   "language": "python",
   "name": "graphrag"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}