Too much RAM consumption (Surveyor)

[K0rS1z]

We have a server where a large number of people play. Stable online 10-15 players, in some places more or less. There are more than 100 players on the white list. There is also a build of mods on the server and there are many of them.
Over time, I began to notice that the RAM consumption at each startup is getting higher and higher, and the CPU usage is abnormally high, even when there are no players on the server. As a result, this led to the fact that there was simply not enough RAM for normal operation - at least 28 GB was required (!), which is simply abnormal, and the processor was used almost 100% constantly. When I started to figure it out, I noticed that when the Surveyor starts reading its data in the world, all the parameters fly into the sky.

By removing Antique-atlas-2.10.5+1.20.jar and surveyor-0.6.25+1.20.jar this stopped and the use of server resources became normal.
We have Forge 47.3.12.
There are 1816 files in the world - data\surveyor folder.

The process itself is on video - https://youtu.be/wvqm_bEmL58

[sisby-folk]

would you believe "this is not a bug"

like it's not a feature either, we just haven't implemented the thing that would be required to avoid this behaviour because we're not really sure how:

sisby-folk/surveyor#63

[sisby-folk]

actual practical advice for your server - set terrain to NONE/DISABLED in the server's surveyor config - that'll fix it.

You won't get explored area sharing or restoration, but that's rarely as useful as sharing markers and player locations anyway.

[K0rS1z]

Wow, ok, thx.

[K0rS1z]

would you believe "this is not a bug"

like it's not a feature either, we just haven't implemented the thing that would be required to avoid this behaviour because we're not really sure how:

sisby-folk/surveyor#63

Sorry I didn't notice.

[sisby-folk]

nah I just wrote that one up, didn't realise we didn't have it before, you're all good. anyway I'm leaving this open so people know where to go

[sisby-folk]

@K0rS1z if you haven't already deleted the data\surveyor folder and it still has all the files starting with c in it, could you send it our way? You can delete the landmarks file so it only has terrain and structure data - we've just never had someone generate a save file that big and it could be very useful for working on optimizations.

[K0rS1z]

@K0rS1z if you haven't already deleted the data\surveyor folder and it still has all the files starting with c in it, could you send it our way? You can delete the landmarks file so it only has terrain and structure data - we've just never had someone generate a save file that big and it could be very useful for working on optimizations.

I saved the entire surveyor folder, I didn't delete anything, maybe not only files starting with c will be needed.
https://drive.google.com/file/d/12iKmSLh2jCMyEhGtrV2Z3We-dhdFFZZw/view?usp=sharing

[rabbitscreams]

coming in here to say this issue also affects singleplayer worlds with modded biomes implemented, with the same solution of turning off terrain mapping. currently playing wythers overhauled overworld and lingering too long (anywhere between 10 minutes to less than a second) in one of the modded biomes causes the world to cease ticking (frozen mobs, unusable chests, no block drops, etc). this world is brand new (less than 2 cumulative hours old), and ive given it up to 16gb of ram to see if that gets around things

obligatory disclaimer i have a faulty CPU and this probably affects things, but i did want to chime in that this isnt just large multiplayer worlds. (and/or this is a different problem that happens to have the same solution)