Skip to content

Commit

Permalink
docs: update streaming example (#674)
Browse files Browse the repository at this point in the history
direct usage of chat completions API causes users to think its an OK thing to directly call openai client side, which is not a good idea
  • Loading branch information
Yonom authored Aug 18, 2024
1 parent 7d9c1fb commit e17d8b8
Showing 1 changed file with 21 additions and 6 deletions.
27 changes: 21 additions & 6 deletions apps/docs/content/docs/runtimes/custom-rest.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -126,19 +126,34 @@ export default function RootLayout({
Declare the `run` function as an `AsyncGenerator` (`async *run`). This allows you to `yield` the results as they are generated.

```tsx twoslash {2, 11-13} title="@/app/MyRuntimeProvider.tsx"
import { ChatModelAdapter } from "@assistant-ui/react";
import {
ChatModelAdapter,
ThreadMessage,
ModelConfig,
} from "@assistant-ui/react";
import { OpenAI } from "openai";

const openai = new OpenAI();
const backendApi = async ({
messages,
abortSignal,
config,
}: {
messages: ThreadMessage[];
abortSignal: AbortSignal;
config: ModelConfig;
}) => {
return openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: "Say this is a test" }],
stream: true,
});
};

// ---cut---
const MyModelAdapter: ChatModelAdapter = {
async *run({ messages, abortSignal, config }) {
const stream = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: "Say this is a test" }],
stream: true,
});
const stream = await backendApi({ messages, abortSignal, config });

let text = "";
for await (const part of stream) {
Expand Down

0 comments on commit e17d8b8

Please sign in to comment.