docs: Fix appsettings.Local.*.json examples and update readme to match

README.md

@@ -11,7 +11,7 @@ Cellm is an Excel extension that lets you use Large Language Models (LLMs) like
 ## What is Cellm?
 Similar to Excel's `=SUM()` function that outputs the sum of a range of numbers, Cellm's `=PROMPT()` function outputs the AI response to a range of text. 
 
-For example, you can write `=PROMPT(A1:A10, "Extract all person names mentioned in the text.")` in a cell's formula and drag the cell to apply the prompt to many rows. Cellm is useful when you want to use AI for repetitive tasks that would normally require copy-pasting data in and out of a chat window many times.
+For example, you can write `=PROMPT(A1, "Extract all person names mentioned in the text.")` in a cell's formula and drag the cell to apply the prompt to many rows. Cellm is useful when you want to use AI for repetitive tasks that would normally require copy-pasting data in and out of a chat window many times.
 
 ## Key features
 This extension does one thing and one thing well.
@@ -30,7 +30,7 @@ In this example, we copy the papers' titles and abstracts into Excel and write t
 
 We then use autofill to apply the prompt to many papers. Simple and powerful.
 
-A single paper is misclassified because the original inclusion and exclusion criteria were summarized in one sentence. This is a good example, however, because it shows that these models rely entirely on your input and can make mistakes.
+Note that a paper is misclassified. The models _will_ make mistakes. It is your responsibility to cross validate if a model is accurate enough for your use case and upgrade model or use another approach if not.
 
 ## Getting Started
 
@@ -39,7 +39,7 @@ Cellm must be built from source and installed via Excel. Follow the steps below.
 ### Requirements
 
 - Windows
-- [.NET 6.0 SDK](https://dotnet.microsoft.com/en-us/download/dotnet/6.0)
+- [.NET 9.0 SDK](https://dotnet.microsoft.com/en-us/download/dotnet/9.0)
 - [Excel 2010 or higher (desktop app)](https://www.microsoft.com/en-us/microsoft-365/excel)
 
 ### Build
@@ -54,32 +54,30 @@ Cellm must be built from source and installed via Excel. Follow the steps below.
    cd cellm
    ```
 
-3. Add your Anthropic API key. Rename `src/Cellm/appsettings.Anthropic.json` to `src/Cellm/appsettings.Local.json` and insert it. Example:
-    ```json
-    {
-      "AnthropicConfiguration": {
-        "ApiKey": "YOUR_ANTHROPIC_APIKEY"
-      }
-    }
-    ```
-
-   Cellm uses Anthropic as the default model provider. You can also use models from OpenAI, Mistral, Google, or run models locally. See the `appsettings.Local.*.json` files for examples.
-
-4. Install dependencies:
+3. Install dependencies:
    ```cmd
    dotnet restore
    ```
 
-5. Build the project:
+4. Build the project:
    ```cmd
    dotnet build --configuration Release
    ```
 
+5. Cellm uses Ollama and the Gemma 2 2B model by default. Download and install [Ollama](https://ollama.com/) and run the following command in your Windows terminal to download the model:
+    ```cmd
+    ollama pull gemma2:2b
+    ```
+
+    To use other models, see the [Models](#models) section below.
+
+These steps will build Cellm on your computer. Continue with the steps below to install Cellm in Excel.
+
 ### Install
 
 1. In Excel, go to File > Options > Add-Ins.
 2. In the `Manage` drop-down menu, select `Excel Add-ins` and click `Go...`.
-3. Click `Browse...` and select the `Cellm-AddIn64.xll` file in the bin/Release/net6.0-windows folder.
+3. Click `Browse...` and select the `Cellm-AddIn64.xll` file in the bin/Release/net9.0-windows. This folder is located in the root of git repository that you cloned.
 4. Check the box next to Cellm and click `OK`.
 
 ## Usage
@@ -166,70 +164,93 @@ Cellm is useful for repetitive tasks on both structured and unstructured data. H
 
 These use cases are starting points. Experiment with different instructions to find what works best for your data. It works best when combined with human judgment and expertise in your specific domain.
 
-## Run Models Locally
+## Models
 
-### Requirements
+### Hosted LLMs
+
+Cellm supports hosted models from Anthropic, Google, OpenAI, and any OpenAI-compatible provider. To use e.g. Claude 3.5 Sonnet from Anthropic:
 
-- [Docker](https://www.docker.com/products/docker-desktop/) (optional)
-- A GPU and [NVIDIA CUDA Toolkit 12.4](https://developer.nvidia.com/cuda-downloads) or higher (optional)
+1. Rename `src/Cellm/appsettings.Anthropic.json` to `src/Cellm/appsettings.Local.json`.
+
+2. Add your Anthropic API key to `src/Cellm/appsettings.Local.json`:
+    ```json
+    {
+        "AnthropicConfiguration": {
+            "ApiKey": "ADD_YOUR_ANTHROPIC_API_KEY_HERE"
+        },
+        "ProviderConfiguration": {
+            "DefaultProvider": "Anthropic"
+        }
+    }
+    ```
+
+See the `appsettings.Local.*.json` files for examples on other providers.
+
+### Local LLMs
 
-#### Local LLMs
+Cellm supports local models that run on your computer via Llamafiles, Ollama, or vLLM. This ensures none of your data ever leaves your machine. And it's free. 
 
-Cellm can run LLM models locally on your computer via Llamafiles, Ollama, or vLLM. This ensures none of your data ever leaves your machine. And it's free.
+#### Ollama 
 
-Cellm uses Gemma 2 2B model with 4-bit quantization by default. This clever little model runs fine on a CPU.
+Cellm uses Ollama Gemma 2 2B model by default. This clever little model runs fine on a CPU. For any model larger than 3B you will need a GPU. Ollama will automatically use your GPU if you have one. To get started:
 
-For Ollama and vLLM you will need docker, and for models larger than 3B you will need a GPU.
+1. Download and install Ollama from [https://ollama.com/](https://ollama.com/).
+2. Download the model by running the following command in your Windows terminal:
+   ```cmd
+   ollama pull gemma2:2b
+   ```
+
+See [https://ollama.com/search](https://ollama.com/search) for a complete list of supported models.
 
-### LLamafile
+#### LLamafile
 
-Llamafile is a stand-alone executable that is very easy to setup. Cellm will automatically download a Llamafile model and start a Llamafile server the first time you call `=PROMPT()`. 
+Llamafile is a stand-alone executable that is very easy to setup. To get started:
 
-To get started:
+1. Download a llamafile from https://github.com/Mozilla-Ocho/llamafile (e.g. [Gemma 2 2B](https://huggingface.co/Mozilla/gemma-2-2b-it-llamafile/blob/main/gemma-2-2b-it.Q6_K.llamafile?download=true)).
+2. Run the following command in your Windows terminal:
+    ```cmd
+    .\gemma-2-2b-it.Q6_K.llamafile.exe --server --v2
+    ```
 
-1. Rename `appsettings.Llamafile.json` to `appsettings.Local.json`.
-2. Build and install Cellm.
-3. Run e.g. `=PROMPT(A1, "Extract keywords")` in a formula.
-4. Wait 5-10 min depending on your internet connection. The model will reply once it is ready. 
+    To offload inference to your NVIDIA or AMD GPU, run:
 
-This will use the Llama 3.2 1B model. To use other models, edit the appsettings file and rebuild.
+    ```cmd
+    .\gemma-2-2b-it.Q6_K.llamafile.exe --server --v2 -ngl 999
+    ```
 
-Use `appsettings.Llamafile.GPU.json` to offload Llamafile inference to your NVIDIA or AMD GPU.
+3. Rename `appsettings.Llamafile.json` to `appsettings.Local.json`.
+4. Build and install Cellm.
 
-### Ollama and vLLM
+### Dockerized Ollama and vLLM
 
-Ollama and vLLM are LLM inference servers for running models locally. Ollama is designed for easy of use and vLLM is designed to run models efficiently with high throughput. Both Ollama and vLLM are packaged up with docker compose files in the `docker/` folder.
+Both Ollama and vLLM are packaged up with docker compose files in the `docker/` folder. Ollama is designed for easy of use and vLLM is designed to run many requests in parallel which is particularly useful for this Add-In. Open WebUI in included in both Ollama and vLLM docker compose files so you can test the local model outside of Cellm. It is available at `http://localhost:3000`.
 
 To get started, we recommend using Ollama with the Gemma 2 2B model:
 
-1. Rename `appsettings.Ollama.json` to `appsettings.Local.json`, 
-2. Build and install Cellm.
-3. Run the following command in the `docker/` directory:   
+1. Build and install Cellm.
+2. Run the following command in the `docker/` directory:   
    ```cmd
    docker compose -f docker-compose.Ollama.yml up --detach
    docker compose -f docker-compose.Ollama.yml exec backend ollama pull gemma2:2b
    docker compose -f docker-compose.Ollama.yml down  // When you want to shut it down
    ```
 
-If you want to speed up inference, you can use your GPU as well:
+To use other Ollama models, pull another of the [supported models](https://ollama.com/search). If you want to speed up inference, you can use your GPU as well:
 
 ```cmd
 docker compose -f docker-compose.Ollama.yml -f docker-compose.Ollama.GPU.yml up --detach
 ```
 
-A GPU is practically required if you want to use models larger than Gemma 2 2b.
-
-If you want to speed up running many requests in parallel, you can use vLLM instead of Ollama. You must supply the docker compose file with a Huggingface API key either via an environment variable or editing the docker compose file directy. Look at the vLLM docker compose file for details. If you don't know what a Huggingface API key is, just use Ollama. 
+If you want to further speed up running many requests in parallel, you can use vLLM instead of Ollama. You must supply the docker compose file with a Huggingface API key either via an environment variable or editing the docker compose file directy. Look at the vLLM docker compose file for details. If you don't know what a Huggingface API key is, just use Ollama. 
 
 To start vLLM:
 
 ```cmd
 docker compose -f docker-compose.vLLM.GPU.yml up --detach
 ```
 
-To use other Ollama models, pull another of the [supported models](https://ollama.com/library). To use other vLLM models, change the "--model" argument to another Huggingface model.
+To use other vLLM models, change the "--model" argument in the docker compose file to another Huggingface model.
 
-Open WebUI in included in both Ollama and vLLM docker compose files so you can test the local model outside of Cellm. It is available at `http://localhost:3000`.
 ## Dos and Don'ts
 
 Do:

src/Cellm/Models/Providers/Anthropic/AnthropicConfiguration.cs

@@ -10,6 +10,8 @@ internal class AnthropicConfiguration : IProviderConfiguration
 
     public string ApiKey { get; init; }
 
+    public List<string> Models { get; init; } = new();
+
     public AnthropicConfiguration()
     {
         BaseAddress = default!;

src/Cellm/Models/Providers/Mistral/MistralConfiguration.cs

@@ -0,0 +1,10 @@
+namespace Cellm.Models.Providers.Mistral;
+
+internal class MistralConfiguration : IProviderConfiguration
+{
+    public Uri BaseAddress { get; init; } = default!;
+
+    public string DefaultModel { get; init; } = string.Empty;
+
+    public List<string> Models { get; init; } = new();
+}

src/Cellm/Models/Providers/Ollama/OllamaConfiguration.cs

@@ -8,5 +8,5 @@ internal class OllamaConfiguration : IProviderConfiguration
 
     public string DefaultModel { get; init; } = string.Empty;
 
-    public bool EnableServer { get; init; } = false;
+    public List<string> Models { get; init; } = new();
 }

src/Cellm/Models/Providers/OpenAi/OpenAiConfiguration.cs

@@ -5,4 +5,6 @@ internal class OpenAiConfiguration : IProviderConfiguration
     public string DefaultModel { get; init; } = string.Empty;
 
     public string ApiKey { get; init; } = string.Empty;
+
+    public List<string> Models { get; init; } = new();
 }

src/Cellm/Models/Providers/OpenAi/OpenAiRequestHandler.cs

@@ -4,7 +4,8 @@
 
 namespace Cellm.Models.Providers.OpenAi;
 
-internal class OpenAiRequestHandler([FromKeyedServices(Provider.OpenAi)] IChatClient chatClient) : IModelRequestHandler<OpenAiRequest, OpenAiResponse>
+internal class OpenAiRequestHandler(
+    [FromKeyedServices(Provider.OpenAi)] IChatClient chatClient) : IModelRequestHandler<OpenAiRequest, OpenAiResponse>
 {
 
     public async Task<OpenAiResponse> Handle(OpenAiRequest request, CancellationToken cancellationToken)

src/Cellm/Models/Providers/OpenAiCompatible/OpenAiCompatibleConfiguration.cs

@@ -7,4 +7,6 @@ internal class OpenAiCompatibleConfiguration
     public string DefaultModel { get; init; } = string.Empty;
 
     public string ApiKey { get; init; } = string.Empty;
+
+    public int HttpTimeoutInSeconds = 3600;
 }

src/Cellm/Models/Providers/OpenAiCompatible/OpenAiCompatibleRequestHandler.cs

@@ -1,11 +1,9 @@
 using Cellm.Models.Prompts;
-using Microsoft.Extensions.Options;
 
 namespace Cellm.Models.Providers.OpenAiCompatible;
 
 internal class OpenAiCompatibleRequestHandler(
-    OpenAiCompatibleChatClientFactory openAiCompatibleChatClientFactory,
-    IOptions<OpenAiCompatibleConfiguration> openAiCompatibleConfiguration)
+    OpenAiCompatibleChatClientFactory openAiCompatibleChatClientFactory)
     : IModelRequestHandler<OpenAiCompatibleRequest, OpenAiCompatibleResponse>
 {
 

src/Cellm/Models/ServiceCollectionExtensions.cs

@@ -74,7 +74,9 @@ public static IServiceCollection AddOpenAiCompatibleChatClient(this IServiceColl
             .AddSingleton<OpenAiCompatibleChatClientFactory>()
             .AddHttpClient<OpenAiCompatibleChatClientFactory>(openAiCompatibleHttpClient =>
             {
-                openAiCompatibleHttpClient.Timeout = Timeout.InfiniteTimeSpan;
+                openAiCompatibleHttpClient.Timeout = TimeSpan.FromSeconds(configuration
+                    .GetSection(nameof(ProviderConfiguration))
+                    .GetValue<int>(nameof(ProviderConfiguration.HttpTimeoutInSeconds)));
             })
             .AddResilienceHandler(nameof(OpenAiCompatibleChatClientFactory), resiliencePipelineConfigurator.ConfigureResiliencePipeline);
 

src/Cellm/appsettings.Local.Anthropic.json

@@ -1,10 +1,8 @@
 {
     "AnthropicConfiguration": {
-        "DefaultModel": "claude-3-5-sonnet-20241022",
         "ApiKey": "YOUR_ANTHROPIC_API_KEY"
     },
-    "CellmConfiguration": {
-        "DefaultProvider": "Anthropic",
-        "HttpTimeoutInSeconds": 30
+    "ProviderConfiguration": {
+        "DefaultProvider": "Anthropic"
     }
 }

src/Cellm/appsettings.Local.Google.json

@@ -5,8 +5,7 @@
         "ApiKey": "YOUR_GEMINI_API_KEY"
 
     },
-    "CellmConfiguration": {
-        "DefaultProvider": "OpenAiCompatible",
-        "HttpTimeoutInSeconds": 30
+    "ProviderConfiguration": {
+        "DefaultProvider": "OpenAiCompatible"
     }
 }

src/Cellm/appsettings.Local.Llamafile.json

@@ -1,5 +1,6 @@
 {
-    "CellmConfiguration": {
+    "ProviderConfiguration": {
+        "BaseAddress": "http://127.0.0.1:8080",
         "DefaultProvider": "Llamafile"
     }
 }

src/Cellm/appsettings.Local.Mistral.json

@@ -1,11 +1,10 @@
 {
-    "OpenAiCompatibleConfiguration": {
+    "MistralConfiguration": {
         "BaseAddress": "https://api.mistral.ai/v1",
         "DefaultModel": "mistral-small-latest",
         "ApiKey": "YOUR_MISTRAL_API_KEY"
     },
-    "CellmConfiguration": {
-        "DefaultProvider": "OpenAiCompatible",
-        "HttpTimeoutInSeconds": 30
+    "ProviderConfiguration": {
+        "DefaultProvider": "OpenAiCompatible"
     }
 }

src/Cellm/appsettings.Local.Ollama.json

@@ -1,5 +1,5 @@
 {
-    "CellmConfiguration": {
+    "ProviderConfiguration": {
         "DefaultProvider": "Ollama"
     }
 }

src/Cellm/appsettings.Local.OpenAi.json

@@ -3,8 +3,7 @@
         "DefaultModel": "gpt-4o-mini",
         "ApiKey": "YOUR_OPENAI_API_KEY"
     },
-    "CellmConfiguration": {
-        "DefaultProvider": "OpenAI",
-        "HttpTimeoutInSeconds": 30
+    "ProviderConfiguration": {
+        "DefaultProvider": "OpenAI"
     }
 }

src/Cellm/appsettings.Local.vLLM.json

@@ -2,7 +2,7 @@
     "OpenAiCompatibleConfiguration": {
         "BaseAddress": "http://localhost:8000/v1"
     },
-    "CellmConfiguration": {
+    "ProviderConfiguration": {
         "DefaultProvider": "OpenAiCompatible"
     }
 }

src/Cellm/appsettings.json

@@ -9,7 +9,6 @@
         "DefaultModel": "gemma-2-2b"
     },
     "OllamaConfiguration": {
-        "ZipUrl": "https://github.com/ollama/ollama/releases/download/v0.4.2/ollama-windows-amd64.zip",
         "BaseAddress": "http://127.0.0.1:11434",
         "DefaultModel": "gemma2:2b"
     },
@@ -25,7 +24,7 @@
         "DefaultModel": "gemma2:2b",
         "DefaultTemperature": 0,
         "CacheTimeoutInSeconds": 3600,
-        "HttpTimeoutInSeconds": 600,
+        "HttpTimeoutInSeconds": 3600,
         "EnableCache": true,
         "EnableTools": false
     },