Extreme memory usage and slow loading of moderately sized text file

[Isopod]

Out of curiosity I decided to try my usual editor benchmark with cosmic-text-editor: Opening sqlite3.c (a ~9 MB file which you can download here).

Results:

• Even in release mode, it took around a minute to load.
• It uses almost 3 GB of RAM! For a 9 MB file! In other words, it inflates the actual useful data by a factor of 300 (times, not percent).

For comparison:

• Sublime Text 3 loads the file in around 2 seconds and uses 300 MB of memory.
• Vim loads it instantaneously and uses 34 MB of memory (but disables syntax highlighting) .
• Mousepad (GTK-based editor from Xfce) loads it in around 10 seconds and uses 150 MB of memory.

I understand that this project is still in its infancy, but you might want to address performance before you develop this any further.

[jackpot51]

It is likely this is caused by syntax highlighting the entire file at load. I'll be working on lazy loading of files that should help.

[jackpot51]

I've pushed improvements to opening large files.

[jackpot51]

Another improvement drops the memory usage when loading that file to about 80MB.

[Isopod]

Oh, cool, I did not expect an update. It does load faster now and uses less memory. However, one thing I'm not sure about: Does this just cheat by loading only the beginning of the file? How is the performance and memory usage when you jump to the end of the file? Unfortunately, I can't test this because the scrollbar appears to be broken and I haven't found a working shortcut for jumping to the end of the file. But I do notice that the memory usage creeps up to several GB when I hold down the PageDown key, which leads me to suspect that it's just lazy-loading. Unfortunately, that doesn't really fix the issue.

[jackpot51]

There are some more optimizations I would need to come up with to not have to syntax highlight all previous lines before being able to highlight the current scoll location in the file.

[jackpot51]

I've pushed some changes, based on pop-os/cosmic-text#206, that reduce the memory usage when scrolling to the end of that file to about 418MB, while still keeping syntax highlighting enabled.

[jackpot51]

Additionally, it loads instantly and scrolls to the bottom of the file in 5 seconds.

[JoaoDelille]

Cosmic-edit version:
Name : cosmic-edit
Version : 0.1.0~^20240827git8bae2b7
Release : 1.fc40
Architecture : x86_64
Size : 29 M
Source : cosmic-edit-0.1.0~^20240827git8bae2b7-1.fc40.src.rpm
Repository : @System
From repo : copr:copr.fedorainfracloud.org:ryanabx:cosmic-epoch

I was going to submit a new issue because the exact same thing happens to me, but then found this closed thread. Opening a 1gb file uses up all 20 available Gb of ram and crashes the PC. Testing with the sqlite3.c file I get the same 3Gb of RAM usage!

Did the patches revert at some point?