-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
21 changed files
with
383 additions
and
40 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,13 +1,6 @@ | ||
# Introduction | ||
|
||
A ChatUI that supports plugins and prioritizes local processing. I like PoeAI, but dislike the subscription model. After using the API, my monthly AI bill decreased by more than 10 times ($20 => $1.43). | ||
Welcome to the official website of NovaChat 👏🏻 | ||
NovaChat is a plugin-supported, local-first ChatUI. You can use it directly in your browser or host it remotely yourself. In the future, it will support fully offline client downloads. | ||
|
||
- Local first | ||
- Plugin support | ||
- Simple self-hosting | ||
- Cross-platform web support | ||
- Open Source & Privacy | ||
|
||
## Thanks | ||
|
||
Among all current ChatUIs, I like PoeAI the most for functionality and HF ChatUI the most for UI design. Since the latter is open-source, I directly copied many components and code from it. | ||
![intro](/images/intro-1.png) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
# Using Plugins | ||
|
||
By default, NovaChat does not support any LLM Providers, but you can easily download the necessary plugins to support them from the [plugin list](https://app.novachat.dev/#/plugins), and then configure them in the [settings](https://app.novachat.dev/#/settings). | ||
|
||
![Plugin Usage](/images/plugin-usage-1.png) | ||
|
||
## OpenAI | ||
|
||
1. Download the OpenAI Provider | ||
2. Open the [NovaChat settings page](https://app.novachat.dev/#/settings) | ||
3. Enter your OpenAI API key in the **API key** field | ||
4. Select the model you want to use in the **Model** dropdown | ||
![openai-1](/images/plugin-usage-openai-1.png) | ||
5. Click **New Chat** to start a new conversation | ||
6. Enter your question and press Enter to send | ||
![openai-2](/images/plugin-usage-openai-2.png) | ||
|
||
## Vertex Anthropic | ||
|
||
1. Download the Vertex Anthropic Provider | ||
2. Open the [NovaChat settings page](https://app.novachat.dev/#/settings) | ||
3. Modify the settings `googleSaClientEmail/googleSaPrivateKey/region/projectId` | ||
4. Select the model | ||
![Vertex Anthropic](/images/plugin-usage-vertex-anthropic-1.png) | ||
5. Start a new conversation | ||
6. Enter your question and press Enter to send | ||
|
||
## Ollama | ||
|
||
1. Run Ollama locally, you must set an environment variable, check if `http://localhost:11434` is accessible, refer to: [How can I allow additional web origins to access Ollama?](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-configure-ollama-server) | ||
`OLLAMA_ORIGINS="https://app.novachat.dev" ollama serve` | ||
2. Download the Ollama Provider | ||
3. Open the [NovaChat settings page](https://app.novachat.dev/#/settings) | ||
![Ollama](/images/plugin-usage-ollama-1.png) | ||
4. Select the model | ||
![Ollama](/images/plugin-usage-ollama-2.png) | ||
5. Start a new conversation | ||
6. Enter your question and press Enter to send |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# Self-Hosting | ||
|
||
Since NovaChat doesn't rely on a backend, it can be easily self-hosted. | ||
|
||
## Running Locally | ||
|
||
> Local requirements: Node.js 20 + pnpm 9 | ||
```sh | ||
git clone https://github.com/rxliuli/novachat.git | ||
pnpm i | ||
pnpm dev | ||
``` | ||
|
||
Open [http://localhost:5173](http://localhost:5173) to access it. | ||
|
||
## Deploying Remotely | ||
|
||
### Using Cloudflare Pages | ||
|
||
1. Build static assets locally with `pnpm build` | ||
2. [Register for a Cloudflare account](https://dash.cloudflare.com/sign-up) | ||
3. Navigate to **Workers & Pages > Overview > Create an application > Pages**, and select **Upload assets** | ||
![self-host-1](/images/self-host-1.png) | ||
4. Enter a project name and create it | ||
![self-host-2](/images/self-host-2.png) | ||
5. Drag and drop the `dist` directory to upload | ||
![self-host-3](/images/self-host-3.png) | ||
6. Now, you've successfully deployed to Cloudflare Pages | ||
![self-host-4](/images/self-host-4.png) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,218 @@ | ||
# Creating a Plugin | ||
|
||
The root directory of a plugin must contain at least two files: | ||
|
||
- plugin.json: A JSON file describing the plugin information | ||
- index.js: A JavaScript script implementing the functionality | ||
|
||
The plugin bundle is a zip archive of these two files. | ||
|
||
## Configuring plugin.json | ||
|
||
| Field | Type | Required | Description | | ||
| ------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------- | | ||
| id | string | ✅ | Unique identifier for the plugin, must consist of numbers, lowercase letters, and dots. | | ||
| name | string | ✅ | Plugin name, no restrictions, but it's recommended to keep it short. | | ||
| version | string | ✅ | Plugin version number, must consist of numbers, lowercase letters, and dots. | | ||
| description | string | - | Plugin description. | | ||
| configuration | object | - | Array of plugin options, this field is used to provide options for users to select or fill in, see `config object`. | | ||
|
||
### config object | ||
|
||
| Field | Type | Required | Description | | ||
| ---------- | ---------------------- | -------- | ----------------------------------------------------------- | | ||
| title | string | ✅ | Category displayed on the configuration page for the plugin | | ||
| properties | Record<string, object> | ✅ | Configuration JSON Schema, example below | | ||
|
||
```json | ||
{ | ||
"title": "OpenAI", | ||
"properties": { | ||
"openai.apiKey": { | ||
"type": "string", | ||
"description": "The API key to use for OpenAI", | ||
"default": "", | ||
"format": "password" | ||
} | ||
} | ||
} | ||
``` | ||
|
||
## Implementing index.js | ||
|
||
As mentioned earlier, index.js is a JavaScript script that needs to export an activate function for NovaChat to initialize, and all plugin-related APIs are imported from `@novachat/plugin`. | ||
|
||
Here's a plugin that doesn't do anything: | ||
|
||
```ts | ||
export async function activate() { | ||
console.log('demo plugin activated') | ||
} | ||
``` | ||
|
||
### Adding LLM Support | ||
|
||
To add support for a new LLM, use `novachat.model.registerProvider` to register a new LLM Provider, which generally provides multiple models. | ||
|
||
Here's an implementation of the OpenAI Provider: | ||
|
||
```ts | ||
import * as novachat from '@novachat/plugin' | ||
import OpenAI from 'openai' | ||
|
||
function convertReq( | ||
req: novachat.QueryRequest, | ||
stream?: boolean, | ||
): OpenAI.ChatCompletionCreateParams { | ||
return { | ||
model: req.model, | ||
messages: req.messages.map((it) => { | ||
if (it.from === 'user' && it.attachments) { | ||
return { | ||
role: 'user', | ||
content: [ | ||
{ | ||
type: 'text', | ||
text: it.content, | ||
}, | ||
...it.attachments.map( | ||
(atta) => | ||
({ | ||
type: 'image_url', | ||
image_url: { url: atta.url }, | ||
} as OpenAI.ChatCompletionContentPartImage), | ||
), | ||
], | ||
} | ||
} | ||
return { | ||
content: it.content, | ||
role: it.from, | ||
} | ||
}), | ||
stream, | ||
} | ||
} | ||
|
||
export async function activate(context: novachat.PluginContext) { | ||
const client = async () => { | ||
return new OpenAI({ | ||
apiKey: await novachat.setting.get('openai.apiKey'), | ||
dangerouslyAllowBrowser: true, | ||
}) | ||
} | ||
await novachat.model.registerProvider({ | ||
name: 'OpenAI', | ||
models: [ | ||
{ id: 'o1-preview', name: 'o1-preview' }, | ||
{ id: 'o1-mini', name: 'o1-mini' }, | ||
{ id: 'gpt-4o', name: 'GPT-4o' }, | ||
{ id: 'gpt-4o-mini', name: 'GPT-4o-mini' }, | ||
{ id: 'gpt-4', name: 'GPT-4' }, | ||
{ id: 'gpt-4-turbo', name: 'GPT-4o-turbo' }, | ||
], | ||
async invoke(query) { | ||
const response = await ( | ||
await client() | ||
).chat.completions.create( | ||
convertReq(query) as OpenAI.ChatCompletionCreateParamsNonStreaming, | ||
) | ||
return { | ||
content: response.choices[0].message.content ?? '', | ||
} | ||
}, | ||
async *stream(query) { | ||
const response = await ( | ||
await client() | ||
).chat.completions.create( | ||
convertReq(query, true) as OpenAI.ChatCompletionCreateParamsStreaming, | ||
) | ||
for await (const it of response) { | ||
yield { | ||
content: it.choices[0].delta.content ?? '', | ||
} | ||
} | ||
}, | ||
}) | ||
} | ||
``` | ||
|
||
The above uses `novachat.setting.get('openai.apiKey')` to get the user-configured API Key, so you also need to add configuration options in plugin.json. | ||
|
||
```json | ||
{ | ||
"configuration": { | ||
"title": "OpenAI", | ||
"properties": { | ||
"openai.apiKey": { | ||
"type": "string", | ||
"description": "The API key to use for OpenAI", | ||
"default": "", | ||
"format": "password" | ||
} | ||
} | ||
} | ||
} | ||
``` | ||
|
||
> Reference: <https://github.com/rxliuli/novachat/tree/main/packages/plugin-openai> | ||
### Implementing a Prompt Bot | ||
|
||
Sometimes you just want to use prompts for role-playing or other purposes, hoping to use the already configured LLM Provider. In this case, you can implement a Prompt Bot. Here's a translation plugin that detects the language of the input content and translates it to English if it matches the user's default language, otherwise translates it to the user's configured language. | ||
|
||
```ts | ||
import * as novachat from '@novachat/plugin' | ||
import { last } from 'lodash-es' | ||
import { franc } from 'franc-min' | ||
|
||
const SYSTEM_MESSAGE = | ||
'You are a professional, authentic machine translation engine.' | ||
const USER_MESSAGE = ` | ||
Translate the following source text to {{to}}, Output translation directly without any additional text. | ||
Source Text: {{text}} | ||
Translated Text: | ||
`.trim() | ||
|
||
export async function activate() { | ||
await novachat.model.registerBot({ | ||
id: 'translator', | ||
name: 'Translator', | ||
async *stream( | ||
query: novachat.QueryRequest, | ||
): AsyncGenerator<novachat.QueryChunkResponse> { | ||
const lastMessage = last(query.messages) | ||
if (!lastMessage) { | ||
throw new Error('No last message') | ||
} | ||
const defaultModel = await novachat.model.getDefault() | ||
if (!defaultModel) { | ||
throw new Error('No default model') | ||
} | ||
const localLanguage = | ||
(await novachat.setting.get('translator.localLanguage')) ?? 'eng' | ||
const language = franc(lastMessage.content) | ||
const toLanguage = language === localLanguage ? 'eng' : localLanguage | ||
const stream = novachat.model.stream({ | ||
messages: [ | ||
{ from: 'system', content: SYSTEM_MESSAGE }, | ||
{ | ||
from: 'user', | ||
content: USER_MESSAGE.replace('{{to}}', toLanguage).replace( | ||
'{{text}}', | ||
lastMessage.content, | ||
), | ||
}, | ||
], | ||
model: defaultModel.id, | ||
}) | ||
for await (const it of stream) { | ||
yield it | ||
} | ||
}, | ||
}) | ||
} | ||
``` | ||
|
||
> Reference: <https://github.com/rxliuli/novachat/tree/main/packages/plugin-translator> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,22 @@ | ||
# Plugin Development | ||
# Developing Plugins | ||
|
||
In NovaChat, plugins are the primary way to extend the application's functionality. They can be used to support new LLMs, create custom Bots, or integrate other features. | ||
|
||
## Principle | ||
|
||
Plugins need to be developed using JavaScript. When using a plugin, NovaChat will run the plugin in a WebWorker and use message communication to call the plugin's functions. | ||
|
||
## Runtime Environment | ||
|
||
It's important to note that plugins run in a WebWorker, so they cannot access the DOM. Additionally, the following APIs are blocked and should not be used: | ||
|
||
- Cache-related: localStorage/sessionStorage/indexedDB/cookie, consider using the settings API instead | ||
- Remote code: including `eval/Function/import/import()`, which are synonymous with insecurity. Please bundle all code into a single file | ||
|
||
## Plugin Types | ||
|
||
Currently, plugins are mainly divided into two categories: LLM Provider and Bot. Although both can be used in NovaChat, the latter doesn't require custom API keys and will only use the configured default LLM Model. | ||
|
||
## Acknowledgements | ||
|
||
The plugin system has drawn inspiration from many excellent designs, such as VSCode for Web, Figma, Joplin, and others. |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
File renamed without changes
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
# 简介 | ||
|
||
欢迎来到 NovaChat 的官方网站 👏🏻 | ||
NovaChat 是一个支持插件、本地优先的 ChatUI。你可以直接在浏览器中使用 NovaChat,也可以自行托管到远端,后续将支持完全离线化的客户端下载。 | ||
NovaChat 是一个支持插件、本地优先的 ChatUI。你可以直接在浏览器中使用它,也可以自行托管到远端,后续将支持完全离线化的客户端下载。 | ||
|
||
![intro](/images/intro.png) | ||
![intro](/images/intro-1.png) |
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.