embedding doesn't work

[jurykor]

I just found out that from version 3.18.5 embedding doesn't work well.

Let's take this as an example of the "twinny" repository.

In version 3.18.4, if I run the explain @extension.global.d.ts, I'm getting a file analysis:

But in versions 3.18.5 and later, I just get "reflections on the topic":

So I had to roll back to 3.18.4...

[rjmacarthy]

Please update to 3.19.0 and let us know.

Many thanks!

[fishshi]

I tried 3.19.0, and it seems that this issue does exist. I haven't looked at the code in detail yet, but I suspect it might be related to the fileContents that was removed in a710b6b.

[jurykor]

in version 3.19.0, the embedding process does not start processing (I waited a long time without recording):

2024-11-13.14-39-52.mp4

[rjmacarthy]

Thanks for the detailed report. It seems related to this issue: #386 Will be investigating soon, since we removed some default ignored file list (in favour of user defined) we have some crashes. Are there any errors in the console? In vscode you can go to Help > Toggle Developer Tools and see the console logs from twinny.

Many thanks!

[jurykor]

I'll take a look now. There is only one moment at once - all the files in the folder (from the video) are less than 100 lines.

[jurykor]

At the moment of clicking the "Embed workspace documents" button:

2024-11-13 15:08:14.755 [error] [Extension Host] Fetch error: TypeError: Cannot read properties of undefined (reading '0')
	at EmbeddingDatabase.getEmbeddingFromResponse (c:\Users\JuriKor\.vscode\extensions\rjmacarthy.twinny-3.19.0-win32-x64\out\index.js:57898:31)
	at onData (c:\Users\JuriKor\.vscode\extensions\rjmacarthy.twinny-3.19.0-win32-x64\out\index.js:57787:24)
	at fetchEmbedding (c:\Users\JuriKor\.vscode\extensions\rjmacarthy.twinny-3.19.0-win32-x64\out\index.js:57697:5)
	at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

PS. It so happens that the latest messages are related to a different issue (and version). Could we move them to a separate thread so as not to confuse them with the current discussion?

[yuzcat01]

It seems that it now only passes the file path to the model instead of the file content, but the model cannot read the file content just by the path, so it's not working.

[rjmacarthy]

Please provide API details of embedding provider API for ollama. Please make sure it's /api/embed as described here it's the new standard it replaced the old API endpoint: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings

[jurykor]

Please provide API details of embedding provider API for ollama. Please make sure it's /api/embed as described here it's the new standard it replaced the old API endpoint: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings

Thanks, it helped. The problem with the embedding process is coming to an end, but the initial problem still remains. However, as far as I understand, this problem will be solved with #389