Merge pull request #1215 from zc277584121/master

fix topk, add comparision table
milvus-io · Dec 5, 2023 · 0c8dc60 · 0c8dc60
2 parents d1b960f + ae1107a
commit 0c8dc60
Show file tree

Hide file tree

Showing 2 changed files with 14 additions and 2 deletions.
diff --git a/evaluation/README.md b/evaluation/README.md
@@ -4,4 +4,16 @@ Evaluate the performance of RAG pipelines based on [ragas](https://github.com/ex
 
 
 - [evaluate_fiqa_openai.ipynb](evaluate_fiqa_openai.ipynb) Use Ragas to evaluate the OpenAI Assistant
-- [evaluate_fiqa_customized_RAG.ipynb](evaluate_fiqa_customized_RAG.ipynb) Use Ragas to evaluate the customized RAG pipeline based on milvus
+- [evaluate_fiqa_customized_RAG.ipynb](evaluate_fiqa_customized_RAG.ipynb) Use Ragas to evaluate the customized RAG pipeline based on milvus
+
+The following outlines a comparison between two experimental setups:
+
+|  | OpenAI assistant | Customized RAG pipeline |
+| --- | --- | --- |
+| LLM model | gpt-4-1106-preview | gpt-4-1106-preview |
+| Vector DB | Not Disclosed | milvus |
+| Embedding model | Not Disclosed | BAAI/bge-base-en |
+| Chunk size | Not Disclosed | 1000 |
+| Chunk overlap | Not Disclosed | 40 |
+| topk | Not Disclosed | 5 |
+| Use Agent | Yes | Yes |
diff --git a/evaluation/evaluate_fiqa_customized_RAG.ipynb b/evaluation/evaluate_fiqa_customized_RAG.ipynb
@@ -302,7 +302,7 @@
    "outputs": [],
    "source": [
     "def search_milvus(question, top_k=5):\n",
-    "    contexts = vector_db.similarity_search(question)\n",
+    "    contexts = vector_db.similarity_search(question, k=top_k)\n",
     "    return contexts[:top_k]"
    ],
    "metadata": {