Feature Suggestion: Provide hooks with a way to retrieve LLM output as a string

[rburmorrison]

Hey there!

This may already be possible, but if not it might be a good addition!

The idea is that in a hook, instead of only being able to open a new window or popup, we can get the result of the output directly as well. My use case for this is trying to get The Parrot to generate Neovim/Vim commands and automatically insert them into Vim for me. I'd like to be able to just ask the LLM for the string and then use Neovim Lua to add it to command-mode for me before I confirm it.

Here would be a conceptual, "theoretical" hook with a possible design of having a new "Ask" command with a callback that gives the resulting string:

-- ...
Cmd = function(prt, params)
  local chat_prompt = [[
  You are an expert in creating nvim/vim command-line commands. Based
  on the provided prompt, generate an appropriate command that can be
  entered in the Neovim editor's command line interface. The command
  should match the prompt in functionality.

  Here is the prompt:

  ```
  {{command}}
  ```

  Output the exact nvim/vim command without any additional text or
  explanation.

  For example, for the prompt "Replace all instances of 'one' with
  'two'", the command is:

  ```
  :'<,'>s/one/two/g
  ```

  Ensure that the output is fully formatted and executable in the
  nvim/vim command line.

  Output _only_ the command needed.
  ]]

  local model = prt.get_model("command")
  prt.Ask(params, model, "Input ~", chat_prompt, function(result)
    -- process the resulting text and insert it into cmd mode
  end)
end,
-- ...

I imagine other types of useful hooks that would likely emerge from having a unified interface for requesting an LLM response, and the ability to do some pre/post-processing of the output would also have its own benefits.

I looked through some of the code and tried to find if there's anything directly accessible like this, but wasn't able to find anything. I'm not very familiar with Neovim Lua outside of configuring Neovim, however, so I may have missed it.

Thanks!

[frankroeder]

Dear @rburmorrison,
This is a great suggestion. I already created a local branch and am looking into this. The thing is that everything is async and the output gets piped into the buffer. Let me see how this works with the command input of neovim.

[rburmorrison]

@frankroeder

Thanks for looking into it! I'm not too familiar with building plugins for Neovim, but if you find you don't have enough time to implement this, let me know and I might try my hand at it if I find time!

[frankroeder]

@rburmorrison

You could try because my time currently is very limited.

• You could either try to create a new type of Target (see ui.lua) and add the related routine to the promptmethod (https://github.com/frankroeder/parrot.nvim/blob/main/lua/parrot/chat_handler.lua#L1154)
• Or you can find a way to simply output the string after the API request, but this requires turning off the streaming response and waiting, maybe synchronously for the request to finish. This requires a lot of changes.
• Alternatively, you can find a way to use the, e.g., Target.popup, retrieve the text from it, and close it. Now you can do whatever you want with the yanked text.