chat.registerProvider or lm.registerProvider for custom AI providers

[worksofliam]

One of the biggest issues with the AI community right now is that there are multiple vendors re-implementing the GitHub Chat UI in a webview to recreate Copilot but with different providers - either running locally or running in a cloud server - and this is causing some pain for extension maintainers.

I want to support customers that use the GitHub Copilot service, or host their own model locally, or pay for a cloud service, but all through the same UI. Right now, if you want to use your own models, you need a new extension (like Continue) which adds a new UI and has unique standards compared to the Chat UI. The issue is that more vendors are creating their own 'AI services' extensions with their own webview.

What I would really like to see is those vendors, instead of creating a new webview to create their own chat window, I'd love to be able to re-use the VS Code Chat UI/UX with other services/providers that can be registered through my extension. Similar to the language APIs, which allow extensions to implement language-server interfaces so the UI is the same across multiple languages, I would love that for the AI UI (Chat window, asking questions in code, etc) also.

Something like chat.registerChatProvider or lm.registerChatProvider, which would then have callbacks. Then, after your provider has been registered, I might see it show up in the family selection list in the Chat window.

I really don't want to see multiple extensions re-implementing their own chat webviews. I would like for it to be standardized in the UI for extensions to be able to add to. Under 'Should I use a webview?' in the VS Code docs, it says:

they should also be used sparingly and only when VS Code's native API is inadequate

This is an area where the API is inadequate and as more vendors are building extensions, it'd be better if we all standardized (like the language server APIs).

[gjsjohnmurray]

Is the proposed API in src/vscode-dts/vscode.proposed.chatProvider.d.ts relevant?

[worksofliam]

@gjsjohnmurray Yes, that file does look like it's headed in the right direction! What's the eta on that?