Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SEWorldGenPlugin issues #80

Open
WhiteFeatherx69 opened this issue Apr 3, 2021 · 90 comments
Open

SEWorldGenPlugin issues #80

WhiteFeatherx69 opened this issue Apr 3, 2021 · 90 comments
Labels
bug Something isn't working v2.0 Issue with version 2.0

Comments

@WhiteFeatherx69
Copy link

WhiteFeatherx69 commented Apr 3, 2021

Server is crashig 2-6 times each day when SEWorldGenPlugin is enabled, and server is getting massive SS drops when being in Asteroid Fields.

The GPS location updater, updatres like 20-40 times each sec.

01:15:38.5859 [FATAL] Initializer: System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List1.Enumerator.MoveNextRare() at SEWorldGenPlugin.Session.MyEntityTrackerComponent.UpdateBeforeSimulation() at Patched_Sandbox.Game.World.MySessionUpdateComponents_0(Object ) at Patched_Sandbox.Game.World.MySessionUpdate_0(Object , MyTimeSpan ) at Sandbox.MySandboxGame.Update() at Patched_Sandbox.Engine.Platform.GameUpdateInternal_0(Object ) at Patched_Sandbox.Engine.Platform.GameRunSingleFrame_0(Object ) at Patched_Sandbox.Engine.Platform.FixedLoop+<>c__DisplayClass11_0<Run>b__0_0(Object ) at Sandbox.Engine.Platform.GenericLoop.Run(VoidAction tickCallback) at Sandbox.Engine.Platform.Game.RunLoop() at Sandbox.MySandboxGame.Run(Boolean customRenderLoop, Action disposeSplashScreen) at Torch.VRageGame.DoStart() in C:\build\workspace\Torch_Torch_master\Torch\VRageGame.cs:line 288 at Torch.VRageGame.Run() in C:\build\workspace\Torch_Torch_master\Torch\VRageGame.cs:line 123 at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() System.InvalidOperationException: Collection was modified; enumeration operation may not execute. at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource) at System.Collections.Generic.List1.Enumerator.MoveNextRare()
at SEWorldGenPlugin.Session.MyEntityTrackerComponent.UpdateBeforeSimulation()
at Patched_Sandbox.Game.World.MySessionUpdateComponents_0(Object )
at Patched_Sandbox.Game.World.MySessionUpdate_0(Object , MyTimeSpan )
at Sandbox.MySandboxGame.Update()
at Patched_Sandbox.Engine.Platform.GameUpdateInternal_0(Object )
at Patched_Sandbox.Engine.Platform.GameRunSingleFrame_0(Object )
at Patched_Sandbox.Engine.Platform.FixedLoop+<>c__DisplayClass11_0b__0_0(Object )
at Sandbox.Engine.Platform.GenericLoop.Run(VoidAction tickCallback)
at Sandbox.Engine.Platform.Game.RunLoop()
at Sandbox.MySandboxGame.Run(Boolean customRenderLoop, Action disposeSplashScreen)
at Torch.VRageGame.DoStart() in C:\build\workspace\Torch_Torch_master\Torch\VRageGame.cs:line 288
at Torch.VRageGame.Run() in C:\build\workspace\Torch_Torch_master\Torch\VRageGame.cs:line 123
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
01:15:38.5859 [INFO] Initializer: Generating minidump at C:\UDInstanceAlpha\Instance\Minidump.dmp
01:15:38.5859 [ERROR] Initializer: Keen broke the minidump, sorry.

@WhiteFeatherx69
Copy link
Author

00:02:42.4083 [WARN] Keen: SEWorldGenPlugin - WARNING Couldnt register asteroid object providers for assembly ProtoBuf.Net.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
00:02:42.5333 [WARN] Keen: SEWorldGenPlugin - WARNING Couldnt register asteroid object providers for assembly Steamworks.NET, Version=15.0.0.0, Culture=neutral, PublicKeyToken=null
00:02:42.8145 [WARN] Keen: SEWorldGenPlugin - WARNING Couldnt register Types for assembly ProtoBuf.Net.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
00:02:42.8614 [WARN] Keen: SEWorldGenPlugin - WARNING Couldnt register Types for assembly Steamworks.NET, Version=15.0.0.0, Culture=neutral, PublicKeyToken=null

@thorwin99
Copy link
Owner

This second log is just warnings, thats nothing. It searches for custom asteroid object providers, there are 2 currently, asteroid ring and sphere. As for the other issue, i will need to investigate a bit and fix it

@thorwin99
Copy link
Owner

Sim speed loss when being in asteroid belts can be solved by reducing the density. Remember, the plugin roughly uses 4 to 6 times the density of default SE. While SE might generate 5 asteroids around a player, this plugin by default generates hundred in a belt.

@thorwin99
Copy link
Owner

Published a pre release for you to test: https://github.com/thorwin99/SEWorldGenPlugin/releases/tag/v2.0.5.3pre1
This should fix the crash log you posted, as for the other issues, liket he one with the gps updater, can you elaborate more on that please? The GPS updater checks if any gps needs to be deleted, which it does before simulation

@WhiteFeatherx69
Copy link
Author

WhiteFeatherx69 commented Apr 6, 2021

thanks.. seems to work.. but i get weird reboot issues now.. it loads up all my ram and freezes. and dies after 5 mins stuck..
the GPS locator for Asteroid belts.. updates it location for the belt like 20 times a sec.. and jumps around you when close to the belt.. when looking under GPS, you can see the Asteroid belt GPS updates like there is no day tomorrow.
.

@thorwin99
Copy link
Owner

If it eats up the ram while in asteroids reduce the density. 0.1 is the vanilla equivalent. There is some issue with asteroids, that doesnt clean up the ram, but i dont know why thats happening. The plugin itself removes all references to them, so it should be SE.
Updating is correct just not that it jumps around and refreshes the GPS list. I will check that

@thorwin99
Copy link
Owner

I am sorry, but i cant replicaet that GPS issue, when getting close to an asteroid belt, that yellow dynamic gps goes away. Or do you have persistent gps activated for belts? If not, could you post the xml files for the plugin for the world here?

@thorwin99 thorwin99 added bug Something isn't working v2.0 Issue with version 2.0 labels Apr 9, 2021
@WhiteFeatherx69
Copy link
Author

i'll try it out soon.. we are having some other issues. some is about to be fixed.. i keep you updated.. all i know is, when the GPS marker, while looking at it under GPS window, you can see it update the coords like 20-30 times per sec.. a bit to fast. since when getting close to the belt, the gps begins to jump all over the place to tell you where the belt is closes to..

@thorwin99
Copy link
Owner

Just to clarify, do you mean asteroid rings or asteroid belts. For asteroid belts, i could not replicate the jumping around. If it is for belts, are you far away from the center (ca. >2ls)?

@WhiteFeatherx69
Copy link
Author

okay.. i have been told by many player-s that the asteroid belt, is always loaded. and can be seen from really far away.. maybe that's why it takes up so much memory.. i managed to use 20+ gb memory with seworldgenplugin.

@thorwin99
Copy link
Owner

There are 2 things, if there are grids in there, they load asteroids. If not, there still might be the memory leak that somehow doesnt clean up asteroids, or they were edited. Normally it unloads asteroids, as i already tested that in singleplayer while watching the entity list. What i am not sure is, if the clientside cleans them up, but i cant affect that side with the current solution

@WhiteFeatherx69
Copy link
Author

well.. im still wondering about the Gps for the asteroid rings locator. everytime you move, unger K then GPS, select the asteroid ring GPS, you can see it's updating it's location, like 100 times a sec.. im wondering if that can cause instability, when having 40+ players, that it will update for all of them everytime they move?

@thorwin99
Copy link
Owner

The thing is, it updates that location before simulation so it updates it as often as simulation runs. I am not sure if i can make that slower without putting it on another thread that would need to synchronize back to the main thread. Where do you even see it updating? the GPS list should not even refresh as an update is not the same as removing and adding it. What i could do is set a timer that has to run to 0 before updating GPSs again, so they would update once per second, but i have to see how i would go about it

@WhiteFeatherx69
Copy link
Author

WhiteFeatherx69 commented May 15, 2021 via email

@thorwin99
Copy link
Owner

Yeah, i would like to see that. I will implement a timer, so it only updates them once a second

@WhiteFeatherx69
Copy link
Author

@thorwin99
Copy link
Owner

Ah, okay yeah makes sense. I thought you mean the list itself updates, which would cause that you can't select a gps

@WhiteFeatherx69
Copy link
Author

oh. an antoher issue.. sadly.. players comments about the gps disappears and reappears when a player joins the game/world.

@thorwin99
Copy link
Owner

Do you mean persistent or discovery gpss

@WhiteFeatherx69
Copy link
Author

Players just reported, they were far away, but could see the gps marker in space.. then it just Disappears, when a new player joins the server, the gps marker in space reappears.

@thorwin99
Copy link
Owner

When you move away from the ring it should disappear. Does it get deleted from the gps list?

@WhiteFeatherx69
Copy link
Author

WhiteFeatherx69 commented May 15, 2021

yes when you are to far away. the GPS vanish as it should. and reappear when coming in range.
But the plugin is making some hits on the server.. 35 players, and players reporting some lags spikes.. so sadly, i have to take off
the plugin again, sorry.. so that's all i have for now..

@WhiteFeatherx69
Copy link
Author

WhiteFeatherx69 commented May 15, 2021

MyProceduralGeneratorComponent' took 1.84ms/f main, 0.00ms/f parallel (total 917.90ms) that's a lot sadly for one belt
Maybe im doing something wrong.

@thorwin99
Copy link
Owner

Yeah it goes through the whole system and searches the belts, that for each player. I will make it once per second or try to put it on a different thread

@WhiteFeatherx69
Copy link
Author

used 32Gb of ram. 20+ for your plugin alone.. damn

@thorwin99
Copy link
Owner

So there was a memory leak that got fixed 2 updates ago. However if you can provide more details where it used those 20 Gb it would be appreciated. It can simply be the asteroid generator, since the amount of asteroids generated is quite large and there seems to be some memory leak regarding those, which is a known issue

@WhiteFeatherx69
Copy link
Author

@WhiteFeatherx69
Copy link
Author

yeah. and it's UD:. we have like 20-46 players on every day.

@WhiteFeatherx69
Copy link
Author

well. thanks for the help..

@WhiteFeatherx69
Copy link
Author

can't you make it so it used Vanilla asteroids to spawn around the moon.. so they behave the same

@thorwin99
Copy link
Owner

No. Again, in vanilla, the server does not generate the asteroids, the clients do. With the plugin the server does it. The plugin would have to be on EVERY client and the server to work that way, and that's the only way. But that's what mods should be, not really a plugin, since it is magnitudes more powerful than mods. However, mods can't generate asteroids, without implementing that whole stuff myself.

@thorwin99
Copy link
Owner

@WhiteFeatherx69 Since you are a server owner, i want your opinion on this matter. I thought about solving the performance problem and the came up with one solution.
A world setting for a dedicated server mode. This mode can be enabled or disabled. When disabled, the plugin works as is. When enabled, asteroid generation will work like the vanilla one, however each and every client would need to have the plugin installed. I ask about your opinion on this since you seem to run a server and forcing each client to have the plugin would be the only way to move asteroid generation and synchronization from the server and improve performance for server and clients. Again, this will be a separate setting which can be enabled, that way existing servers / worlds dont break and servers that want to force that could enable it.

@WhiteFeatherx69
Copy link
Author

Can't force players to download a plugin, since we have so many new players coming and going..

for the last few months, we have manually spawned asteroids around Earth's Moon, and since that's the only thing we need.

Since you are saying, each client discover an asteroid, the server implement it to the world, where it will be handled like every other asteroid, so when a player gets far enough away, it de-renders from world.

Your plugin is adding asteroids that can't process the same way, (fixed, always rendered) for everyone to see no matter the range.

option 1, is there a way to conceal them when no players are around,

option 2, somehow make your plugin not generate, but rather repaste asteroids in the same order around the moon.

option 3. mod it. somehow.

@thorwin99
Copy link
Owner

Yeah thought so, that's why i wanted your opinion. The problem is, asteroids are not replicated. they don't show up on clients at all when generated on a server. So the way the plugin forces the server to update the clients that an asteroid was spawned is to set it as modified. A modified asteroid lies server side and gets replicated to all clients. Even in vanilla SE that will happen, but to a lesser extends since not all asteroids are mined.
I dont know whats possible with the modding API and what is not, but the last time i tried, it was nearly impossible to spawn an asteroid with it, once i figure that out, a mod could be the solution.
Manually placing them does set them as modified so they are saved, i guess. I can look whether or not the asteroids dont correctly unload, that could be a possibility

@WhiteFeatherx69
Copy link
Author

okay.. cause that's the only issue.. if you can make the asteroid rings behave like every other asteroid, that would be perfect.

i don't mind the ram usages,

but the effect it has on all players, it tanks the server, cause the server keeps them loaded for every player no matter the range..

can't you replicate the vanilla style into the plugin somehow. so the behave the same.

without having to force every player to get the plugin.

@thorwin99
Copy link
Owner

No. Again, vanilla generates them clientside so the code has to run clientside

@WhiteFeatherx69
Copy link
Author

then let a player handle the plugin, then it's one player that gonna have the issue,

@thorwin99
Copy link
Owner

How is that supposed to work?

@WhiteFeatherx69
Copy link
Author

WhiteFeatherx69 commented Aug 19, 2021 via email

@WhiteFeatherx69
Copy link
Author

i have no idea.. enable the plugin on the server. with nothing created.. then the player with the plugin enters and create the field around the moon.. maybe..

i suggested a new idea to the team..

we just disabled the plugin.. it was taking up 35gb of memory alone from your plugin sadly...

my other idea was. to password the server.. and let a single player fly around the moon while i have increased the asteroid density, then it would generate a vanilla field around the moon.. then save the asteroids so next time we do wipes.. we can just paste in the field again.. vanilla style..

@thorwin99
Copy link
Owner

The thing with the client as the generator wouldnt work, it would still need to sync the asteroids to all players and the result would be the same. For the idea to save each and every asteroid, i dont know how many you currently have, but those would be thousands of asteroids saved.

@WhiteFeatherx69
Copy link
Author

why we wipe all asteroids.. then make the field.. then save... then open the server for all

@thorwin99
Copy link
Owner

I know what you mean. But to save those asteroids, it will create thousands of savefiles for those asteroids. Thats what i mean

@WhiteFeatherx69
Copy link
Author

well. we are running out of ideas..

the staff is telling me to just create a small field around the moon.. and leave it at that

@thorwin99
Copy link
Owner

I think thats the best idea, at least currently sadly. Creating the ring is possible, saving it would be too if i modify the plugin a bit, but thats just so many asteroids, i dont think it is worth it, especially since SE streams saved asteroids regardless like the plugin does. The plugin works for multiplayer but for 40 player servers, it will reach its limit.

@WhiteFeatherx69
Copy link
Author

well. i let the staff try a few things.. i let you know if they have some ideas..

@thorwin99
Copy link
Owner

Issue will now be handled on the MultiplayerSupport branch. There might be a solution to the asteroid generation performance impact in multiplayer by creating a mod, that generates the asteroids for the plugin on the client. That mod would communicate with the plugin, to generate the asteroids. This would remove the streaming issues, since streaming thousands of asteroids is heavy on performance of client and server. However, i dont know if that will work yet, since the Mod API is really restrictive.

@WhiteFeatherx69
Copy link
Author

Let me know, if you get a breakthrough, we would like to have it reinstated. as part of "Upside Down"

thorwin99 added a commit that referenced this issue Apr 20, 2022
thorwin99 added a commit that referenced this issue Apr 21, 2022
@thorwin99
Copy link
Owner

@WhiteFeatherx69 Sadly, it doesn't look good... I have implemented a mod that could work, and already tested it. It communicates with the plugin to decide where to generate asteroids. However, the ModAPI doesn't allow the client to generate asteroids, if it isn't the server. The client can't generate its own asteroids with a mod, unless i would implement the complete voxel engine of SE into the mod itself, and therefore completely replicate it. but i dont know of that would even work, since many things are restricted by the modAPI.

However, there is one thing that can be done. The ModAPI does allow the generation of prefab asteroids on the client. So technically the mod could generate asteroids, however there are only 10 or so different prefab asteroids in SE, so variety wouldn't be high and one could quickly notice the repeating asteroids...

@WhiteFeatherx69
Copy link
Author

could it work with custom made asteroids. since we are making those ourself as well.. and we would like o get the asteroid belt back around the moon..

@thorwin99
Copy link
Owner

You mean modded asteroids? I think so, if they show up in the prefab spawn menu for asteroids they will work

@WhiteFeatherx69
Copy link
Author

Should ask if we could use the source code of yours???

@thorwin99
Copy link
Owner

Sure thing. If you plan to modify the plugin you can always just fork it here on GitHub.
Otherwise, it's still open source, so no problem in copying parts of the code.
If its major parts of the code and not forked, i would still appreciate to at least be mentioned in a comment or something.
Thanks for asking either way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working v2.0 Issue with version 2.0
Projects
None yet
Development

No branches or pull requests

3 participants