diff --git a/README.md b/README.md index fb187e0..a43950a 100644 --- a/README.md +++ b/README.md @@ -62,6 +62,7 @@ Disclaimer: Examples contributed by the community and partners do not represent | [Indexify Integration - PDF Summarization](third_party/Indexify/pdf-summarization) | summarization, PDF | Indexify | | [langgraph_code_assistant_mistral.ipynb](third_party/langchain/langgraph_code_assistant_mistral.ipynb) | code | Langchain | | [langgraph_crag_mistral.ipynb](third_party/langchain/langgraph_crag_mistral.ipynb) | RAG | Langchain | +| [langtrace_mistral.ipynb](third_party/langtrace/langtrace_mistral.ipynb) | OTEL Observability | Langtrace | | [llamaindex_agentic_rag.ipynb](third_party/LlamaIndex/llamaindex_agentic_rag.ipynb) | RAG, agent | LLamaIndex | | [llamaindex_mistralai_finetuning.ipynb](third_party/LlamaIndex/llamaindex_mistralai_finetuning.ipynb) | fine-tuning | LLamaIndex | | [llamaindex_mistral_multi_modal.ipynb](third_party/LlamaIndex/llamaindex_mistral_multi_modal.ipynb) | MultiModalLLM-Pixtral | LLamaIndex | diff --git a/third_party/Langtrace/README.md b/third_party/Langtrace/README.md new file mode 100644 index 0000000..f9b3152 --- /dev/null +++ b/third_party/Langtrace/README.md @@ -0,0 +1,19 @@ +# Trace and Observe your Mistral AI Models with Langtrace + +## What is Langtrace? +Langtrace [(GitHub)](https://github.com/Scale3-Labs/langtrace) is an open-source and open telemetry based LLM application observability platform. + +## What can Langtrace help you with? +- Capture model traces that is rich in metadata such as prompts, completions and hyper parameter settings. +- Capture traces from all the other parts of your stack such as vector DB, framrworks etc. +- Annotate and evaluate model generations to measure baseline performance and accuracy. +- Run automated evaluations to test for regressions and compare models/model settings. +- Manage and version control prompts. + +## How to get started with Langtrace? +- Head over to https://langtrace.ai and sign up for a free account. +- Create a project and grab your API key. +- Follow the instructions on this [notebook](langtrace_mistral.ipynb) + +## Feedback +If you have any feedback, please create a GitHub [Issue](https://github.com/Scale3-Labs/langtrace/issues) or reach out to us directly on [Discord](https://discord.langtrace.ai/). diff --git a/third_party/Langtrace/TraceTree1.png b/third_party/Langtrace/TraceTree1.png new file mode 100644 index 0000000..9d495d8 Binary files /dev/null and b/third_party/Langtrace/TraceTree1.png differ diff --git a/third_party/Langtrace/TraceTree2.png b/third_party/Langtrace/TraceTree2.png new file mode 100644 index 0000000..d33d2ac Binary files /dev/null and b/third_party/Langtrace/TraceTree2.png differ diff --git a/third_party/Langtrace/constitution.pdf b/third_party/Langtrace/constitution.pdf new file mode 100644 index 0000000..447cb0d Binary files /dev/null and b/third_party/Langtrace/constitution.pdf differ diff --git a/third_party/Langtrace/langtrace_mistral.ipynb b/third_party/Langtrace/langtrace_mistral.ipynb new file mode 100644 index 0000000..dbde384 --- /dev/null +++ b/third_party/Langtrace/langtrace_mistral.ipynb @@ -0,0 +1 @@ +{"cells":[{"cell_type":"markdown","metadata":{"id":"g7j3YX8-WvMX"},"source":["# Setting up Langtrace with Mistral\n","\n","This Notebook shows the instructions for setting up OpenTelemetry based tracing for Mistral with Langtrace AI.\n","\n","The Goal for this notebook to showcase a simple RAG app where you can chat with the United states consititution pdf."]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"collapsed":true,"executionInfo":{"elapsed":4081,"status":"ok","timestamp":1731615462519,"user":{"displayName":"Karthik Kalyanaraman","userId":"06165623146440115541"},"user_tz":480},"id":"8HpEYe58UiOl","outputId":"232de61c-8294-403e-b031-a76a1f894612"},"outputs":[{"name":"stdout","output_type":"stream","text":["Requirement already satisfied: mistralai in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (1.2.3)\n","Requirement already satisfied: langtrace-python-sdk in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (3.3.10)\n","Requirement already satisfied: chromadb in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (0.5.20)\n","Requirement already satisfied: pypdf in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (5.1.0)\n","Requirement already satisfied: langchain in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (0.3.8)\n","Requirement already satisfied: langchain-community in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (0.3.8)\n","Requirement already satisfied: eval-type-backport<0.3.0,>=0.2.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (0.2.0)\n","Requirement already satisfied: httpx<0.28.0,>=0.27.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (0.27.2)\n","Requirement already satisfied: jsonpath-python<2.0.0,>=1.0.6 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (1.0.6)\n","Requirement already satisfied: pydantic<3.0.0,>=2.9.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (2.10.1)\n","Requirement already satisfied: python-dateutil==2.8.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (2.8.2)\n","Requirement already satisfied: typing-inspect<0.10.0,>=0.9.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from mistralai) (0.9.0)\n","Requirement already satisfied: six>=1.5 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from python-dateutil==2.8.2->mistralai) (1.16.0)\n","Requirement already satisfied: colorama>=0.4.6 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (0.4.6)\n","Requirement already satisfied: fsspec>=2024.6.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (2024.10.0)\n","Requirement already satisfied: opentelemetry-api>=1.25.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: opentelemetry-exporter-otlp-proto-grpc>=1.25.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: opentelemetry-exporter-otlp-proto-http>=1.25.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: opentelemetry-instrumentation-sqlalchemy>=0.46b0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (0.49b2)\n","Requirement already satisfied: opentelemetry-instrumentation>=0.47b0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (0.49b2)\n","Requirement already satisfied: opentelemetry-sdk>=1.25.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: sentry-sdk>=2.14.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (2.19.0)\n","Requirement already satisfied: sqlalchemy in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (2.0.35)\n","Requirement already satisfied: tiktoken>=0.1.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (0.8.0)\n","Requirement already satisfied: trace-attributes==7.1.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (7.1.0)\n","Requirement already satisfied: transformers>=4.11.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (4.46.3)\n","Requirement already satisfied: ujson>=5.10.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langtrace-python-sdk) (5.10.0)\n","Requirement already satisfied: build>=1.0.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (1.2.2.post1)\n","Requirement already satisfied: chroma-hnswlib==0.7.6 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.7.6)\n","Requirement already satisfied: fastapi>=0.95.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.115.5)\n","Requirement already satisfied: uvicorn[standard]>=0.18.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.32.1)\n","Requirement already satisfied: numpy>=1.22.5 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (1.26.4)\n","Requirement already satisfied: posthog>=2.4.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (3.7.3)\n","Requirement already satisfied: typing-extensions>=4.5.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (4.12.2)\n","Requirement already satisfied: onnxruntime>=1.14.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (1.20.1)\n","Requirement already satisfied: opentelemetry-instrumentation-fastapi>=0.41b0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.49b2)\n","Requirement already satisfied: tokenizers>=0.13.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.20.3)\n","Requirement already satisfied: pypika>=0.48.9 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.48.9)\n","Requirement already satisfied: tqdm>=4.65.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (4.67.1)\n","Requirement already satisfied: overrides>=7.3.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (7.7.0)\n","Requirement already satisfied: importlib-resources in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (6.4.5)\n","Requirement already satisfied: grpcio>=1.58.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (1.68.0)\n","Requirement already satisfied: bcrypt>=4.0.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (4.2.1)\n","Requirement already satisfied: typer>=0.9.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (0.13.1)\n","Requirement already satisfied: kubernetes>=28.1.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (31.0.0)\n","Requirement already satisfied: tenacity>=8.2.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (9.0.0)\n","Requirement already satisfied: PyYAML>=6.0.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (6.0.2)\n","Requirement already satisfied: mmh3>=4.0.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (5.0.1)\n","Requirement already satisfied: orjson>=3.9.12 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (3.10.12)\n","Requirement already satisfied: rich>=10.11.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from chromadb) (13.9.4)\n","Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain) (3.11.7)\n","Requirement already satisfied: langchain-core<0.4.0,>=0.3.21 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain) (0.3.21)\n","Requirement already satisfied: langchain-text-splitters<0.4.0,>=0.3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain) (0.3.2)\n","Requirement already satisfied: langsmith<0.2.0,>=0.1.17 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain) (0.1.146)\n","Requirement already satisfied: requests<3,>=2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain) (2.32.3)\n","Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain-community) (0.6.7)\n","Requirement already satisfied: httpx-sse<0.5.0,>=0.4.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain-community) (0.4.0)\n","Requirement already satisfied: pydantic-settings<3.0.0,>=2.4.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain-community) (2.6.1)\n","Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (2.4.3)\n","Requirement already satisfied: aiosignal>=1.1.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n","Requirement already satisfied: attrs>=17.3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (24.2.0)\n","Requirement already satisfied: frozenlist>=1.1.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.5.0)\n","Requirement already satisfied: multidict<7.0,>=4.5 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.1.0)\n","Requirement already satisfied: propcache>=0.2.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (0.2.0)\n","Requirement already satisfied: yarl<2.0,>=1.17.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.18.0)\n","Requirement already satisfied: packaging>=19.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from build>=1.0.3->chromadb) (24.2)\n","Requirement already satisfied: pyproject_hooks in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from build>=1.0.3->chromadb) (1.2.0)\n","Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain-community) (3.23.1)\n","Requirement already satisfied: starlette<0.42.0,>=0.40.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from fastapi>=0.95.2->chromadb) (0.41.3)\n","Requirement already satisfied: anyio in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpx<0.28.0,>=0.27.0->mistralai) (4.6.2.post1)\n","Requirement already satisfied: certifi in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpx<0.28.0,>=0.27.0->mistralai) (2024.8.30)\n","Requirement already satisfied: httpcore==1.* in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpx<0.28.0,>=0.27.0->mistralai) (1.0.7)\n","Requirement already satisfied: idna in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpx<0.28.0,>=0.27.0->mistralai) (3.10)\n","Requirement already satisfied: sniffio in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpx<0.28.0,>=0.27.0->mistralai) (1.3.1)\n","Requirement already satisfied: h11<0.15,>=0.13 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from httpcore==1.*->httpx<0.28.0,>=0.27.0->mistralai) (0.14.0)\n","Requirement already satisfied: google-auth>=1.0.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (2.36.0)\n","Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (1.8.0)\n","Requirement already satisfied: requests-oauthlib in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (2.0.0)\n","Requirement already satisfied: oauthlib>=3.2.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (3.2.2)\n","Requirement already satisfied: urllib3>=1.24.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (2.2.3)\n","Requirement already satisfied: durationpy>=0.7 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from kubernetes>=28.1.0->chromadb) (0.9)\n","Requirement already satisfied: jsonpatch<2.0,>=1.33 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langchain-core<0.4.0,>=0.3.21->langchain) (1.33)\n","Requirement already satisfied: requests-toolbelt<2.0.0,>=1.0.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from langsmith<0.2.0,>=0.1.17->langchain) (1.0.0)\n","Requirement already satisfied: coloredlogs in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from onnxruntime>=1.14.1->chromadb) (15.0.1)\n","Requirement already satisfied: flatbuffers in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from onnxruntime>=1.14.1->chromadb) (24.3.25)\n","Requirement already satisfied: protobuf in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from onnxruntime>=1.14.1->chromadb) (5.28.3)\n","Requirement already satisfied: sympy in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from onnxruntime>=1.14.1->chromadb) (1.13.3)\n","Requirement already satisfied: deprecated>=1.2.6 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-api>=1.25.0->langtrace-python-sdk) (1.2.15)\n","Requirement already satisfied: importlib-metadata<=8.5.0,>=6.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-api>=1.25.0->langtrace-python-sdk) (8.5.0)\n","Requirement already satisfied: googleapis-common-protos~=1.52 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.25.0->langtrace-python-sdk) (1.66.0)\n","Requirement already satisfied: opentelemetry-exporter-otlp-proto-common==1.28.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.25.0->langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: opentelemetry-proto==1.28.2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.25.0->langtrace-python-sdk) (1.28.2)\n","Requirement already satisfied: opentelemetry-semantic-conventions==0.49b2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-instrumentation>=0.47b0->langtrace-python-sdk) (0.49b2)\n","Requirement already satisfied: wrapt<2.0.0,>=1.0.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-instrumentation>=0.47b0->langtrace-python-sdk) (1.17.0)\n","Requirement already satisfied: opentelemetry-instrumentation-asgi==0.49b2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb) (0.49b2)\n","Requirement already satisfied: opentelemetry-util-http==0.49b2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-instrumentation-fastapi>=0.41b0->chromadb) (0.49b2)\n","Requirement already satisfied: asgiref~=3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from opentelemetry-instrumentation-asgi==0.49b2->opentelemetry-instrumentation-fastapi>=0.41b0->chromadb) (3.8.1)\n","Requirement already satisfied: monotonic>=1.5 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from posthog>=2.4.0->chromadb) (1.6)\n","Requirement already satisfied: backoff>=1.10.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from posthog>=2.4.0->chromadb) (2.2.1)\n","Requirement already satisfied: annotated-types>=0.6.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from pydantic<3.0.0,>=2.9.0->mistralai) (0.7.0)\n","Requirement already satisfied: pydantic-core==2.27.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from pydantic<3.0.0,>=2.9.0->mistralai) (2.27.1)\n","Requirement already satisfied: python-dotenv>=0.21.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from pydantic-settings<3.0.0,>=2.4.0->langchain-community) (1.0.1)\n","Requirement already satisfied: charset-normalizer<4,>=2 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from requests<3,>=2->langchain) (3.4.0)\n","Requirement already satisfied: markdown-it-py>=2.2.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from rich>=10.11.0->chromadb) (3.0.0)\n","Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from rich>=10.11.0->chromadb) (2.18.0)\n","Requirement already satisfied: regex>=2022.1.18 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from tiktoken>=0.1.1->langtrace-python-sdk) (2024.11.6)\n","Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from tokenizers>=0.13.2->chromadb) (0.26.2)\n","Requirement already satisfied: filelock in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from transformers>=4.11.3->langtrace-python-sdk) (3.16.1)\n","Requirement already satisfied: safetensors>=0.4.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from transformers>=4.11.3->langtrace-python-sdk) (0.4.5)\n","Requirement already satisfied: click>=8.0.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from typer>=0.9.0->chromadb) (8.1.7)\n","Requirement already satisfied: shellingham>=1.3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from typer>=0.9.0->chromadb) (1.5.4)\n","Requirement already satisfied: mypy-extensions>=0.3.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from typing-inspect<0.10.0,>=0.9.0->mistralai) (1.0.0)\n","Requirement already satisfied: httptools>=0.6.3 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.6.4)\n","Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (0.21.0)\n","Requirement already satisfied: watchfiles>=0.13 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (1.0.0)\n","Requirement already satisfied: websockets>=10.4 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from uvicorn[standard]>=0.18.3->chromadb) (14.1)\n","Requirement already satisfied: cachetools<6.0,>=2.0.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (5.5.0)\n","Requirement already satisfied: pyasn1-modules>=0.2.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (0.4.1)\n","Requirement already satisfied: rsa<5,>=3.1.4 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (4.9)\n","Requirement already satisfied: zipp>=3.20 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from importlib-metadata<=8.5.0,>=6.0->opentelemetry-api>=1.25.0->langtrace-python-sdk) (3.21.0)\n","Requirement already satisfied: jsonpointer>=1.9 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from jsonpatch<2.0,>=1.33->langchain-core<0.4.0,>=0.3.21->langchain) (3.0.0)\n","Requirement already satisfied: mdurl~=0.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from markdown-it-py>=2.2.0->rich>=10.11.0->chromadb) (0.1.2)\n","Requirement already satisfied: humanfriendly>=9.1 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from coloredlogs->onnxruntime>=1.14.1->chromadb) (10.0)\n","Requirement already satisfied: mpmath<1.4,>=1.1.0 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from sympy->onnxruntime>=1.14.1->chromadb) (1.3.0)\n","Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes>=28.1.0->chromadb) (0.6.1)\n","\n","\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n","\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n","Note: you may need to restart the kernel to use updated packages.\n"]}],"source":["%pip install mistralai langtrace-python-sdk chromadb pypdf langchain langchain-community"]},{"cell_type":"markdown","metadata":{},"source":["## Imports & Initialize clients\n"]},{"cell_type":"code","execution_count":2,"metadata":{"id":"RPYBLKHLU3NW"},"outputs":[{"name":"stdout","output_type":"stream","text":["\u001b[32mInitializing Langtrace SDK..\u001b[39m\n","\u001b[37m⭐ Leave our github a star to stay on top of our updates - https://github.com/Scale3-Labs/langtrace\u001b[39m\n","\u001b[34mExporting spans to Langtrace cloud..\u001b[39m\n"]},{"name":"stderr","output_type":"stream","text":["/Users/ali/Work/mistral-cookbook/.venv/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n"," from .autonotebook import tqdm as notebook_tqdm\n"]}],"source":["import chromadb\n","from mistralai import Mistral\n","from langchain_community.document_loaders import PyPDFLoader\n","from langchain.text_splitter import RecursiveCharacterTextSplitter\n","from chromadb.utils import embedding_functions\n","from langtrace_python_sdk import langtrace, with_langtrace_root_span\n","\n","\n","langtrace.init(api_key='')\n","mistral = Mistral(api_key='')\n","client = chromadb.Client()\n","\n"]},{"cell_type":"markdown","metadata":{},"source":["## Use Langchain to split pdf into chunks"]},{"cell_type":"code","execution_count":3,"metadata":{},"outputs":[],"source":["def load_and_split_pdf(pdf_path):\n"," loader = PyPDFLoader(pdf_path)\n"," data = loader.load()\n"," text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)\n"," chunks = text_splitter.split_documents(data)\n"," return chunks"]},{"cell_type":"markdown","metadata":{},"source":["## Setup Chroma & Insert pdf chunks\n","Create a chroma collection, specifying the default embedding function which will be used in our RAG when inserting pdf chunks"]},{"cell_type":"code","execution_count":4,"metadata":{},"outputs":[],"source":["def setup_chroma():\n","\n"," return client.get_or_create_collection(\n"," name=\"mistral-rag\",\n"," embedding_function=embedding_functions.DefaultEmbeddingFunction(),\n"," )\n","\n","\n","def add_documents_to_collection(collection, chunks):\n"," for i, chunk in enumerate(chunks):\n"," collection.add(\n"," documents=[chunk.page_content],\n"," metadatas=[{\"source\": chunk.metadata[\"source\"]}],\n"," ids=[str(i)],\n"," )"]},{"cell_type":"markdown","metadata":{},"source":["## Query Collection\n","1. take query from user, get nearest 3 results from chunked pdf\n","2. construct a prompt structure\n","3. Give query and prompt to mistral for the actual response"]},{"cell_type":"code","execution_count":5,"metadata":{"id":"LAcp_amwVL4S"},"outputs":[],"source":["def query_pdf(collection, query):\n"," results = collection.query(query_texts=[query], n_results=3)\n"," # Construct the prompt with context\n"," context = \"\\n\".join(results[\"documents\"][0])\n"," prompt = f\"\"\"Use the following context to answer the question. If you cannot answer based on the context, say so.\n","\n"," Context:\n"," {context}\n","\n"," Question: {query}\n","\n"," Answer:\"\"\"\n"," response = mistral.chat.complete(\n"," model=\"mistral-large-latest\",\n"," messages=[{\"role\": \"user\", \"content\": prompt}],\n"," )\n"," return response.choices[0].message.content"]},{"cell_type":"markdown","metadata":{},"source":["## Run everything together and monitor using Langtrace."]},{"cell_type":"code","execution_count":6,"metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["Creating collection\n","Loading and splitting pdf\n","Adding documents to collection\n","Querying pdf\n","Based on the context, the purpose of the Constitution is stated in the Preamble: \"to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity.\"\n"]}],"source":["@with_langtrace_root_span(\"main\")\n","def main():\n"," print(\"Creating collection\")\n"," collection = setup_chroma()\n"," print(\"Loading and splitting pdf\")\n"," chunks = load_and_split_pdf(\"https://www.govinfo.gov/content/pkg/CDOC-112hdoc129/pdf/CDOC-112hdoc129.pdf\")\n"," print(\"Adding documents to collection\")\n"," add_documents_to_collection(collection, chunks)\n"," print(\"Querying pdf\")\n"," print(query_pdf(collection, \"What is the purpose of the constitution?\"))\n","\n","\n","if __name__ == \"__main__\":\n"," main()"]},{"cell_type":"markdown","metadata":{"id":"vp05Kt2YVgbv"},"source":["That's it! Now you should be able to see the traces for all your inference calls on Langtrace!"]},{"cell_type":"markdown","metadata":{},"source":["## First Two Screenshots showcase the Trace and span structure of the whole RAG App.\n","\n","\n","![Trace 1](./TraceTree1.png)\n","\n","![Trace 2](./TraceTree2.png)\n"]},{"cell_type":"markdown","metadata":{},"source":["## Second Two Screenshots are details of Mistral's Run.\n","\n","- You can see what prompt is specfically fetched from chromadb and sent to mistral as well as the response\n","\n","![Trace 1](./mistral-langtrace1.png)\n","\n","![Trace 2](./mistral-langtrace2.png)"]}],"metadata":{"colab":{"authorship_tag":"ABX9TyPFTAJHx2doU1tWgCnfqECl","provenance":[]},"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.5"}},"nbformat":4,"nbformat_minor":0} diff --git a/third_party/Langtrace/mistral-langtrace1.png b/third_party/Langtrace/mistral-langtrace1.png new file mode 100644 index 0000000..8de8a93 Binary files /dev/null and b/third_party/Langtrace/mistral-langtrace1.png differ diff --git a/third_party/Langtrace/mistral-langtrace2.png b/third_party/Langtrace/mistral-langtrace2.png new file mode 100644 index 0000000..242205d Binary files /dev/null and b/third_party/Langtrace/mistral-langtrace2.png differ diff --git a/third_party/Langtrace/requirements.txt b/third_party/Langtrace/requirements.txt new file mode 100644 index 0000000..d65c2f0 --- /dev/null +++ b/third_party/Langtrace/requirements.txt @@ -0,0 +1,8 @@ +mistralai +langchain +langchain-community +chromadb +python-dotenv +langtrace-python-sdk +pypdf +