-
Notifications
You must be signed in to change notification settings - Fork 77
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
80fecb2
commit c894b34
Showing
3 changed files
with
83 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,25 +1,67 @@ | ||
# Chat Models: Ollama | ||
|
||
Text generation with LLMs via Ollama. | ||
|
||
## Description | ||
|
||
Spring AI provides a `ChatClient` abstraction for integrating with LLMs via several providers, including Ollama. | ||
|
||
When using the _Spring AI Ollama Spring Boot Starter_, a `ChatClient` object is autoconfigured for you to use Ollama. | ||
By default, the _llama2_ model is used. | ||
|
||
```java | ||
@RestController | ||
class ChatController { | ||
private final ChatClient chatClient; | ||
|
||
ChatController(ChatClient chatClient) { | ||
this.chatClient = chatClient; | ||
} | ||
|
||
@GetMapping("/ai/chat") | ||
String chat(@RequestParam(defaultValue = "What did Gandalf say to the Balrog?") String message) { | ||
return chatClient.call(message); | ||
} | ||
} | ||
``` | ||
|
||
## Running the application | ||
|
||
The application relies on Ollama for providing LLMs. You can either run Ollama locally on your laptop (macOS or Linux), or rely on the Testcontainers support in Spring Boot to spin up an Ollama service automatically. | ||
|
||
### When using Ollama | ||
|
||
First, make sure you have [Ollama](https://ollama.ai) installed on your laptop (macOS or Linux). | ||
Then, use Ollama to run the _llama2_ large language model. | ||
|
||
```shell | ||
ollama run llama2 | ||
``` | ||
|
||
Finally, run the Spring Boot application. | ||
|
||
```shell | ||
./gradlew bootRun | ||
``` | ||
|
||
### When using Docker/Podman | ||
|
||
The application relies on the native Testcontainers support in Spring Boot to spin up an Ollama service with a _llama2_ model at startup time. | ||
|
||
```shell | ||
./gradlew bootTestRun | ||
``` | ||
|
||
## Calling the application | ||
|
||
You can now call the application that will use Ollama and llama2 to generate text based on a default prompt. | ||
|
||
```shell | ||
http :8080/ai/chat | ||
``` | ||
|
||
Try passing your custom prompt and check the result. | ||
|
||
```shell | ||
http :8080/ai/chat message=="What is the capital of Italy?" | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,23 +1,59 @@ | ||
# Chat Models: OpenAI | ||
|
||
Text generation with LLMs via OpenAI. | ||
|
||
## Description | ||
|
||
Spring AI provides a `ChatClient` abstraction for integrating with LLMs via several providers, including OpenAI. | ||
|
||
When using the _Spring AI OpenAI Spring Boot Starter_, a `ChatClient` object is autoconfigured for you to use OpenAI. | ||
By default, the _gpt-3.5-turbo_ model is used. | ||
|
||
```java | ||
@RestController | ||
class ChatController { | ||
private final ChatClient chatClient; | ||
|
||
ChatController(ChatClient chatClient) { | ||
this.chatClient = chatClient; | ||
} | ||
|
||
@GetMapping("/ai/chat") | ||
String chat(@RequestParam(defaultValue = "What did Gandalf say to the Balrog?") String message) { | ||
return chatClient.call(message); | ||
} | ||
} | ||
``` | ||
|
||
## Running the application | ||
|
||
The application relies on an OpenAI API for providing LLMs. | ||
|
||
### When using OpenAI | ||
|
||
First, make sure you have an OpenAI account. | ||
Then, define an environment variable with the OpenAI API Key associated to your OpenAI account as the value. | ||
|
||
```shell | ||
export SPRING_AI_OPENAI_API_KEY=<INSERT KEY HERE> | ||
``` | ||
|
||
Finally, run the Spring Boot application. | ||
|
||
```shell | ||
./gradlew bootRun | ||
``` | ||
|
||
## Calling the application | ||
|
||
You can now call the application that will use Ollama and llama2 to generate text based on a default prompt. | ||
|
||
```shell | ||
http :8080/ai/chat | ||
``` | ||
|
||
Try passing your custom prompt and check the result. | ||
|
||
```shell | ||
http :8080/ai/chat message=="What's the capital of Italy?" | ||
http :8080/ai/chat message=="What is the capital of Italy?" | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters