CodeGPT is not sending previous messages of chat

[niceapps-ch]

What happened?

Since version 2.3.1 it is not sending the message history of a chat tab, but only the system prompt and the latest message.

This is the request of the second message:

[2024-02-27 15:01:00.695] [INFO] Received POST request to /v1/chat/completions with body: {
  "stream": true,
  "temperature": 0.1,
  "messages": [
    {
      "role": "system",
      "content": "You are an AI programming assistant.\nFollow the user's requirements carefully & to the letter.\nYour responses should be informative and logical.\nYou should always adhere to technical information.\nIf the user asks for code or technical questions, you must provide code suggestions and adhere to technical information.\nIf the question is related to a developer, you must respond with content related to a developer.\nFirst think step-by-step - describe your plan for what to build in pseudocode, written out in great detail.\nThen output the code in a single code block.\nMinimize any other prose.\nKeep your answers short and impersonal.\nUse Markdown formatting in your answers.\nMake sure to include the programming language name at the start of the Markdown code blocks.\nAvoid wrapping the whole response in triple backticks.\nThe user works in the Webstorm IDE built by JetBrains which has a concept for editors with open files, integrated unit test support, and output pane that shows the output of running the code as well as an integrated terminal. The user is working on a project using Angular and typescript.\nYou can only give one reply for each conversation turn."
    },
    {
      "role": "user",
      "content": "why do you do this?"
    }
  ],
  "model": "gpt-4"
}

And here the chat window.

Relevant log output or stack trace

No response

Steps to reproduce

1. Ask for an optimization of code
2. Ask a follow up question

CodeGPT version

2.4.0

Operating System

macOS

[reneleonhardt]

@niceapps-ch Can you still reproduce this bug in 2.5.1?
The behavior is different for me (Ollama / CodeLlama), depending on a new chat or existing history.
The context (selected code) is resent when I ask your follow-up question, but the response is not what I expected (practically the same as the first response) 😅

[niceapps-ch]

@reneleonhardt Yes, still acts the same way in 2.5.1. The thing is, the request for the follow up question will still include the selected code (if you still have it selected in the editor) but does not include the previous messages. So the LLM has no context of the conversation and acts as if you start a new conversation with your "follow up question" and the selected code.

When you connect CodeGPT to LM Studio you can actually see the request in the log window of LM Studio, which helps for debugging.

[niceapps-ch]

@reneleonhardt And also here I have to add that it seems only to be an issue when using the Custom OpenAI Service configuration.

[carlrobertoh]

Duplicate of #384

If this issue is still reproducible with a provider other than LM Studio, then please reopen the ticket.

[smirgol]

I think I have the same issue.

I'm using the "Custom OpenAI" provider to connect to my local CodeLlama32B LLM provided by https://github.com/YellowRoseCx/koboldcpp-rocm, which I'm using for GPU acceleration. It has an OpenAI compatible API, so I think that should work.

Now, I can ask a question and also provide a file for context and receive an answer.
At the end of each answer is this line, though:
Unknown API response. Code: 200, Body:

Now, when I ask a follow up question, the context is lost / not sent to the LLM and thus it can't make any sense of my follow up question.

Not sure what I can do about that...

Edit

I'm now using LM Studio for hosting the LLM, which I wasn't aware of yet and it looks like that this is now working properly.