v2.1.8

[clusterzx]

Fixed #118

Fixed #116

Fixed #115

Fixed #98

Fixed #106 again

Fix #103

Fix #140

Implemented #74
(/app/logs)

Implemented #88

Added a loggerService.js to log all messages to a txt and html file

Full Changelog: v2.1.7...v2.1.8

---

This discussion was created from the release v2.1.8.

[mamema]

some progress with ollama, as the docker log is error free. Unfortunately the reality isn't :)

Even though the docker logs are showing:
[SUCCESS] Updated document 26136 with: {

tags: [ 38 ],

the history and paperless-ngx are empty regrding new tags or correspondents. The odd point here is, the log is stating:
No tags or correspondent found in response from Ollama for Document.

Please review your prompt or switch to OpenAI for better results.

then i deleted the history of this document and was going into manual mode.
And here the odd thing, it gives in one second feedback "sucessfully analyzed", this is too fast. This cannot be true!

In ollama i have this:
time=2025-01-13T18:40:38.522Z level=WARN source=runner.go:129 msg="truncating input prompt" limit=2048 prompt=8169 keep=5 new=2048

truncated input prompt?

this is not good i guess

[paradizelost]

Wondering if this is related to the change removing the num_ctx that was set to 10000 before that i've mentioned should be a configurable setting in the UI for what to send over to ollama

[clusterzx]

What I have integrated back in (what I removed for serveral versions) is that all existing Tags and correspondents are inserted into the prompt, so the LLM can see what existing structure you have and match better.

[clusterzx]

I guess the context got too long now.
But lets be real, I will abandon Ollama and switch to a fully configurable solution where you can define also other OpenAI API compliant services (not only Ollama).
Maybe things get better then.

[mamema]

all existing Tags and correspondents

all of them from paperless-ngx?
At the moment 488 tags and 128 correspondents. This breaks every prompt. :-)

[paradizelost]

I've got 2100 tags and 500+ correspondents

[clusterzx]

I have a 16GB of VRAM and the larger models are not even thinkable for me, they need arround 40-60GB of VRAM. Like LLama3.3 70b

[mamema]

and this cannot be optimized with training and fine tuning?

[clusterzx]

No, thats not possible. I mean yeah you can fine tune models, do a different quantitization, change some preprocessors... but in the End you wont succeed. These trainings are much more intensive as the power to use it is needed.

[mamema]

I have asked openai about this, he disagrees, so i‘m trying it in the coming weeks. Fine tuning for special tasks like ours should be feasible, he says….

[clusterzx]

He lied to you 😂 you need maaaaasive amounts of data that is accurate labeled. I mean MASSIVE, about hundreds of thousand documents. Also if you wanna do it on a existing model you may run into issues when trying to train that (hardware limits).

But I can be wrong 😁 so I wish you really good results and if not you learned something new.

[paradizelost]

I added the num_ctx variable back to the ollamaService.js and set it to the 100000 i had before and my documents processed successfully without truncation errors

[mamema]

my ollama is currently a docker container can you tell me where this file is? So i can put this ouside of the container.

[paradizelost]

i had to run docker exec -it paperless-ai /bin/bash, then edit /app/services/ollamaSerivce.js to add it, then restart the container. I'm only editing it in the existing container so its lost on update.

[mamema]

oh i see there is a new version, where this is already is included. The dev gets lazy, only three versions per day. :-)

[clusterzx]

Dont worry, I pushed a new version 2.1.9 it is in pipeline. 15mins from now. The value is back. Also a new settings option to enable if you want your existing tags and correspondents to be sent with.