chat.registerProvider or lm.registerProvider for custom AI providers

[worksofliam]

One of the biggest issues with the AI community right now is that there are multiple vendors re-implementing the GitHub Chat UI in a webview to recreate Copilot but with different providers - either running locally or running in a cloud server - and this is causing some pain for extension maintainers.

I want to support customers that use the GitHub Copilot service, or host their own model locally, or pay for a cloud service, but all through the same UI. Right now, if you want to use your own models, you need a new extension (like Continue) which adds a new UI and has unique standards compared to the Chat UI. The issue is that more vendors are creating their own 'AI services' extensions with their own webview.

What I would really like to see is those vendors, instead of creating a new webview to create their own chat window, I'd love to be able to re-use the VS Code Chat UI/UX with other services/providers that can be registered through my extension. Similar to the language APIs, which allow extensions to implement language-server interfaces so the UI is the same across multiple languages, I would love that for the AI UI (Chat window, asking questions in code, etc) also.

Something like chat.registerChatProvider or lm.registerChatProvider, which would then have callbacks. Then, after your provider has been registered, I might see it show up in the family selection list in the Chat window.

I really don't want to see multiple extensions re-implementing their own chat webviews. I would like for it to be standardized in the UI for extensions to be able to add to. Under 'Should I use a webview?' in the VS Code docs, it says:

they should also be used sparingly and only when VS Code's native API is inadequate

This is an area where the API is inadequate and as more vendors are building extensions, it'd be better if we all standardized (like the language server APIs).

[gjsjohnmurray]

Is the proposed API in src/vscode-dts/vscode.proposed.chatProvider.d.ts relevant?

[worksofliam]

@gjsjohnmurray Yes, that file does look like it's headed in the right direction! What's the eta on that?

[mjbvz]

I think you can use the chat.createChatParticipant apis: https://github.com/microsoft/vscode-extension-samples/tree/main/chat-tutorial

[worksofliam]

@mjbvz Either you didn't read the issue or there is a misunderstanding. I am aware of the Chat API and I have used them. In theory I don't want the Chat API to change. I want to be able to add my own provider/family/model that the Chat API can utilize.

I would appreciate it if the issue was re-opened.

[mjbvz]

If you want to swap out the model, the proposed api @gjsjohnmurray mentioned. This api is still under development and will be finalized when its ready

However the chat participant api is closer to the language-api like abstraction you mention and is what most extensions should build on. The participant api actually also doesn't say anything about which model is used, so you can even use it today to prototype chatting with custom models