Skip to content

Commit

Permalink
spellcheck
Browse files Browse the repository at this point in the history
  • Loading branch information
HomelessDinosaur committed Dec 19, 2024
1 parent 2d6b407 commit a8a1c54
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions docs/guides/python/llama-rag.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ We'll organize our project structure like so:

## Setting up our LLM

We'll define a `ModelParameters` class which will have parameters used throughout our application. By putting it in a class, it means it will lazily load the llm, [embed model](https://docs.llamaindex.ai/en/stable/module_guides/models/embeddings/), and tokenizer so that it doesn't slow down other modules that don't require everything to be initialised. At this point we can also create a prompt template for prompts with our query engine. It will just sanitize some of the hallucinations so that if the model does not know an answer it won't pretend like it does. We'll also define two functions that will convert a prompt or message into the required Llama 3.1 format.
We'll define a `ModelParameters` class which will have parameters used throughout our application. By putting it in a class, it means it will lazily load the LLM, [embed model](https://docs.llamaindex.ai/en/stable/module_guides/models/embeddings/), and tokenizer so that it doesn't slow down other modules that don't require everything to be initialized. At this point we can also create a prompt template for prompts with our query engine. It will just sanitize some of the hallucinations so that if the model does not know an answer it won't pretend like it does. We'll also define two functions that will convert a prompt or message into the required Llama 3.1 format.

```python title:common/model_parameters.py
import os
Expand Down Expand Up @@ -192,7 +192,7 @@ The next step is where we embed our context into the LLM. For this example we wi
git clone https://github.com/nitrictech/docs.git nitric-docs
```

We'll create a script which will download the [LLM](https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF), the embed model (using a recommended [model](https://huggingface.co/BAAI/bge-large-en-v1.5) from Hugging Face), and create the vectorised documentation using the embed model.
We'll create a script which will download the [LLM](https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF), the embed model (using a recommended [model](https://huggingface.co/BAAI/bge-large-en-v1.5) from Hugging Face), and convert the documentation into a vector model using the embed model.

```python title:model_utilities.py
import os
Expand Down Expand Up @@ -262,7 +262,7 @@ download_llm()
build_query_engine()
```

You can then run this using the following command. This should output the models and the vectorised documentation into the `./models` folder.
You can then run the script using the following command. This should output the models and the vector model into the `./models` folder.

```bash
uv run model_utilities.py
Expand Down Expand Up @@ -456,7 +456,7 @@ RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --extra ml --frozen --no-dev --no-python-downloads
```

To ensure an optimised docker image, update the `python.dockerfile.dockerignore` to include the models folder.
To ensure an optimized docker image, update the `python.dockerfile.dockerignore` to include the models folder.

```text
.mypy_cache/
Expand Down

0 comments on commit a8a1c54

Please sign in to comment.