Skip to content

Commit

Permalink
Update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
sidjha1 committed Nov 3, 2024
1 parent 1d8aa5d commit 84ff6e5
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 17 deletions.
20 changes: 9 additions & 11 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,22 +14,20 @@ pip install -r requirements-dev.txt
## Dev Flow
After making your changes, please make a PR to get your changes merged upstream.

## Running vLLM Models
To use vLLM for model serving, you just need to make an OpenAI compatible vLLM server. Then, the `OpenAIModel` class can be used to point to the server. See an example below.
## Running Models
To run a model, you can use the `LM` class in `lotus.models.LM`. We use the `litellm` library to interface with the model.
This allows you to use any model provider that is supported by `litellm`.

Create the server
Here's an example of creating an `LM` object for `gpt-4o`
```
python -m vllm.entrypoints.openai.api_server --model meta-llama/Meta-Llama-3.1-70B-Instruct --port 8000 --tensor-parallel-size 8
from lotus.models import LM
lm = LM(model="gpt-4o")
```

In LOTUS, you should instantiate your model as follows
Here's an example of creating an `LM` object to use `llama3.2` on Ollama
```
from lotus.models import OpenAIModel
lm = OpenAIModel(
model="meta-llama/Meta-Llama-3.1-70B-Instruct",
api_base="http://localhost:8000/v1",
provider="vllm",
)
from lotus.models import LM
lm = LM(model="ollama/llama3.2")
```

## Helpful Examples
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,10 @@ If you're already familiar with Pandas, getting started will be a breeze! Below
```python
import pandas as pd
import lotus
from lotus.models import OpenAIModel
from lotus.models import LM

# configure the LM, and remember to export your API key
lm = OpenAIModel()
lm = LM()
lotus.settings.configure(lm=lm)

# create dataframes with course names and skills
Expand Down
8 changes: 4 additions & 4 deletions docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,10 @@ This can be achieved by applying a semantic filter followed by a semantic aggreg
import pandas as pd
import lotus
from lotus.models import E5Model, OpenAIModel
from lotus.models import E5Model, LM
# Configure models for LOTUS
lm = OpenAIModel(max_tokens=512)
lm = LM()
rm = E5Model()
lotus.settings.configure(lm=lm, rm=rm)
Expand Down Expand Up @@ -90,7 +90,7 @@ If we wanted the challenge of taking courses with a high workload, we can also u

.. code-block:: python
top_2_hardest = df.sem_topk("What {Description} indicates the highest workload?", 2)
top_2_hardest = df.sem_topk("What {Description} indicates the highest workload?", K=2)
LOTUS's semantic join operator can be used to join two dataframes based on a predicate.
Suppose we had a second dataframe containing skills we wanted to get better at (SQL and Chip Design in our case).
Expand All @@ -113,7 +113,7 @@ Let's create a semantic index on the course description column and then search f
# Create a semantic index on the description column and save it to the index_dir directory
df = df.sem_index("Description", "index_dir")
top_conv_df = df.sem_search("Description", "Convolutional Neural Network", 1)
top_conv_df = df.sem_search("Description", "Convolutional Neural Network", K=1)
Another useful operator is the semantic map operator. Let's see how it can be used to get some next topics to explore for each class.
Additionally, let's provide some examples to the model that can be used for demonstrations.
Expand Down

0 comments on commit 84ff6e5

Please sign in to comment.