Terminal (lack of) performance with big network and a lot of updates

[Hiranus]

Issue type:

• 🐌 Performance issue

---

Short description:

As you see I have lots of items in network. 24 double golden chests from Sophisticated Storage and drawer network from Functional Storage.
When I start autocrafting via Integrated Terminal or I turn on something that pushes items into the network every few ticks, like a very fast smelter or processing of combs from Productive Bees then terminal either barely functions or freezes (to the point that windows shows "java is not responding" window) and I have to wait until server finishes doing whatever it's currently working on and only then I regain control.

Steps to reproduce the problem:

My best guess is to have really large network and then simulate its heavy use. For example, you could setup something that pulls large amount of items into the network and then something that exports these items from the network which would result in huge amount of "items added" and "items removed" updates.

I think that terminal simply struggles with the amount of updates that are generated. Basic solution would be to check how fast updates are generated and if it's too fast then group them and send one bigger update every 2 seconds or so rather than 10 updates every tick.

---

Versions:

• This mod: 1,6,4
• IntegratedDynamics: 1.23.16
• CyclopsCore: 1.25.2-634
• Minecraft: 1,21
• Forge: NeoForge 21,1,74

Profiler output:

This is from me crafting 30 diamond drawer upgrades from Functional Storage. Client froze and I had to wait until crafting was finished.
https://spark.lucko.me/1hgtK9XeTB
This profile is from when I queued 2000 combs to be processed and opened terminal, UI didnt froze, but I had huge lag spikes. Processing of these combs pushed something into the network every ~4 ticks or less, depending on the comb.
https://spark.lucko.me/k2Z1Ktcnnp

[rubensworks]

Thanks for reporting!

[rubensworks]

Note to self:

May be related to CyclopsMC/IntegratedDynamics#1412

https://github.com/CyclopsMC/IntegratedTerminals/blob/master-1.21/src/main/java/org/cyclops/integratedterminals/core/terminalstorage/TerminalStorageTabIngredientComponentClient.java#L407 (why is the diff calc even needed?) (somehow optimize, or move to a different thread)

[rhullah]

Thanks for taking the time to report this @Hiranus. I have a very similar setup with sophisticated storage and functional drawers and any time I craft a big item (Omega beehive upgrade) my client just hangs until it's done. I've been meaning try and setup a test world that exhibited it but just haven't gotten to it.

I agree that this really needs to be investigated and resolved which would solidify this mod in the crafting area. Right now I used it as a stepping stone to get to max AE2 Quantum/Matrix crafting.

[rubensworks]

@Hiranus I've made some changes that should positively impact performance.
Could you try out this dev build of Common Capabilities to see how this performs in your pack?

If it reduces lag, I will include this in the next release.
If not, can you share a new spark log? As I'll need to make some more invasive changes in that case.
(note to self: change raw ingredients datastructure or move off main thread)

[rubensworks]

@rhullah In case @Hiranus does not respond anymore, could you have a look on your end?

[Hiranus]

Sorry for late response. ~5 days ago I switched to using AE2 so I can no longer easily test it.

[rhullah]

@rubensworks I'll try and test this today, thanks!

[rhullah]

@rubensworks: So, my original interaction with it was that I would submit the craft job and it would immediately freeze on me basically until the system was done with the crafting job. Trying the DEV build I at first I though performance was improved because I was able to close the crafting grid right away instead of getting stuck in there. But then I went back into it just to see/feel it's reactiveness, and after a few seconds, it froze until the crafting job was done. So my experience with the DEV build didn't really improve anything.

Let me know if there's more I can test/get for you. I didn't run a spark profile or anything during the job, so if you want one, just let me know.

[rubensworks]

Ok, thanks for checking @rhullah!
Could you help me with one more thing, and provide me with a Spark log like in the first post above?
That would help me pinpoint the performance bottleneck on your machine.

[rhullah]

Let me know if this isn't enough, or you want something more specific: https://spark.lucko.me/FVEhjH0EIG

spark

spark is a performance profiler for Minecraft clients, servers, and proxies.

[rubensworks]

@rhullah I suspect you generated the server spark logs, while I will need the client logs (since the lag occurs client-side while you have the terminal open during a crafting job). Could you generate and share those instead?

Note to self: Duplicate data as map<ingred,instancewrapped>, and regen list on change or vice-versa. Or just a plain collection is sufficient?

Note to self 2: If this doesn't improve perf enough, investigate lowering tick rate of diff packets.

[rhullah]

Yes, that last spark was a Server profile. Here's a client profile crafting the same item:

https://spark.lucko.me/hcwmw0kdvX

spark

spark is a performance profiler for Minecraft clients, servers, and proxies.

[rhullah]

Also, to let you know, right after I start the craft I go into my terminal and scroll my list of items. I will say, this DEV build does seem to make it a little more responsive w/ scrolling my inventory. Before it would spin wait until the craft was finished and I couldn't do anything.

[rubensworks]

Thanks for checking @rhullah!

Note to self: look into the more invasive change.

[rubensworks]

@rhullah I've prepared a new dev build for Intergrated Terminals. Could you try out this one? https://www.dropbox.com/scl/fi/8yimrtor6zb8mpbhmidux/IntegratedTerminals-1.21.1-neoforge-1.6.5-DEV.jar?rlkey=s3x2nsrn79cateod3q2zmc8tq&dl=0

There should definitely be an improvement with this one.
But if needed, there are even more things I could do.
So if performance would still be insufficient, could you share another spark log?

You may see some performance degradations when requesting crafts from JEI/REI/EMI, but these will be resolved once I finalize this issue. (if that's not the case, please let me know)

[rhullah]

Do you want me to continue to use the DEV build of CommonCapabilities too?

[rubensworks]

Up to you, it won’t hurt in any case.

[rhullah]

Wow, that was way better! I was able to browse my terminal while crafting was being done and I'm not even sure how long the craft took to finish because I didn't notice lag difference between crafting and not crafting.

I know you only asked for it if there was still a performance issue, but since I took it anyway here's a client profile, let me know if you'd like a server one as well: https://spark.lucko.me/VnlkJUdtCH

---

You may see some performance degradations when requesting crafts from JEI/REI/EMI, but these will be resolved once I finalize this issue. (if that's not the case, please let me know)

By this, do you mean the Control/Command click to auto craft missing ingredients w/in JEI/REI/EMI? If so, I did not test that aspect of it.

spark

spark is a performance profiler for Minecraft clients, servers, and proxies.

[rubensworks]

Excellent, thanks for all the help @rhullah!
I'll look into polishing things up, and preparing a new release soon.

By this, do you mean the Control/Command click to auto craft missing ingredients w/in JEI/REI/EMI? If so, I did not test that aspect of it.

Indeed, that's what I meant.
But I'll look into that myself in any case.

[rhullah]

Thank you much for this. This is exciting.

[N1ghthauq]

I have been contemplating submitting a bug report for this since ATM 8... If there is anything I can do to help with spark logs, let me know...

I have drawn a correlation to it being a large network with a lot of updates, and items having slightly different NBT. Same item ID, but it seems they contend for position when ordering the inventory of the terminal. When the network updates, they are in different spots between themselves.

for example:

[a,b,c,d,e,f,g] is a range of these similar items

after item x is added to the network outside of the range it swaps the sorting of the range

x,[b,a,g,f,e,c,d]

this happens for every network update.

I have worked around this by using RS or AE2, or removing the items from the network, and performance goes through the roof.

If I should submit a separate bug for that, let me know and I will.

[Jack-McKalling]

If you are not using the dev builds to compare your experience yet, then I suggest you try that first.

Your description doesn't sound like it is the underlying cause of the actual problem, but just an additional potential reason for some of it. So I'm leaning on awaiting the next release first, but feel free to test the above dev builds some more.