Unity crash when finding closest point in update

[DriesVRBase]

Hi,

I think I've discovered a bug in the point picking tool for V3.

I'm trying to make a annotation tool for my point cloud but Unity always crashes.
It works for a while but after a few seconds unity freezes.
A possible guess is that there are too many threads being started?

You can test it by changing the Update function in the 'PointPickerV3' script.
Change the 'Input.GetMouseButtonDown' to 'Input.GetMouseButton'. Then go into play mode and hold the left mouse button down while moving around the point cloud.
After a while Unity should freeze and crash without a crash report screen. In the editor log, this is the last log I get.

---

CLR: Managed code called FailFast, saying "An exception was not handled in an AsyncLocal notification callback."
(v3) Picked point at :(1.71, 1.54, 2.25)
UnityEngine.Debug:ExtractStackTraceNoAlloc (byte*,int,string)
UnityEngine.StackTraceUtility:ExtractStackTrace ()
UnityEngine.DebugLogHandler:Internal_Log (UnityEngine.LogType,UnityEngine.LogOption,string,UnityEngine.Object)
UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
UnityEngine.Logger:Log (UnityEngine.LogType,object)
UnityEngine.Debug:Log (object)
PointCloudExtras.PointPickerV3:PointSelected (UnityEngine.Vector3) (at Assets/Project/BIM/features/PointCloudTools/PointCloudViewerDX11/Scripts/Common/dev/PointPickerV3.cs:99)
unitycodercom_PointCloudBinaryViewer.PointCloudViewerTilesDX11:PointCallBack (object) (at Assets/Project/BIM/features/PointCloudTools/PointCloudViewerDX11/Scripts/PointCloudViewerTilesDX11.cs:1931)
UnityLibrary.MainThread/CallInfo:Execute () (at Assets/Project/BIM/features/PointCloudTools/PointCloudViewerDX11/Scripts/Common/MainThread.cs:25)
UnityLibrary.MainThread/d__12:MoveNext () (at Assets/Project/BIM/features/PointCloudTools/PointCloudViewerDX11/Scripts/Common/MainThread.cs:103)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)

(Filename: Assets/Project/BIM/features/PointCloudTools/PointCloudViewerDX11/Scripts/Common/dev/PointPickerV3.cs Line: 99)

Unhandled Exception:
System.Threading.ThreadAbortException: Thread was being aborted.
at System.Threading.ExecutionContext.OnAsyncLocalContextChanged (System.Threading.ExecutionContext previous, System.Threading.ExecutionContext current) [0x000ce] in :0

I'm not really sure where to start looking but I'll update in case I find something.

[DriesVRBase]

Edit 1:

I've managed to stop Unity from crashing. But it's not performant at all at the moment.

I have a CancelThreads method that blocks the main (calling) thread until the point picking threads are done. The method looks something like this:

void CancelThreads()
        {
            if (pointPickingThread != null && pointPickingThread.IsAlive)
            {
                pointPickingThread.Join();
            }

            if (pointPickingThread2 != null && pointPickingThread2.IsAlive)
            {
                pointPickingThread2.Join();
            }
        }

And gets called right before starting the threads in the 'RunPointPickingThread' method. It's far from perfect but it stops unity from crashing.

I think a better way to fix this would be to use Tasks and a thread pool? I'll try that after I do some more tests with this.

[unitycoder]

Ok, yeah i would except that to cause problems... (original intended use was to click and wait for results).
So you basically want to "hover" points constantly?

Does it need to return distance, or just highlight the closest point(s) or something else?

[DriesDeRidder]

Doesn’t really matter. Just a vector3 of the nearest point is enough. The usecase is it’s like a laser that is hitting points ln the point cloud and when i press a the mouse or button on the controller i start drawing points with a line renderer

[unitycoder]

Wouldn't it cause many "bad hits" on points that are behind your 2 nearby points?

Say you try to draw line between door frame nearby,
but in the middle (while drawing) it hits some random point 50 meters behind the door?
(if your use case is to connect points)

[DriesVRBase]

Yes I see what you mean. It's a valid point but I'll add some safeties in place. fe. Distance between points can only be smaller than 1m....
It doesn't have to be very precise. and I'll keep the ray quite short (20-30m) it's not for drawing at long distances.

[unitycoder]

Ok, I'll post here if i find some alternative solutions.

How large point clouds you estimate you might have? (since the current picker gets slower on larger clouds).

Some options that come to mind:

• using depth map (but not so accurate always) like https://github.com/staggartcreations/Graphics-Raycast
• using GPU (its possible to get closest points in shader and then pass data to script for sorting, but it had some slow down issues when last tested, might need to look again)

[DriesVRBase]

We're working with clients that provide their point clouds so it can range from a single room to a building or a large ship. The largest point clouds I had were around 30gb. But mostly they are around 10gb.

So unfortunately they can get quite large.

Thanks for also looking into it. I'm also going to be trying to make a multithreaded picker but I don't have a lot of experience regarding this so it'll be a learning thing.

[unitycoder]

in the meanwhile,
other workaround ideas:

• keep view distance very low during drawing lines (since you don't want to pick far away points), this should make picking faster
• instead of holding mouse button down, could do the start-and-end click, with GPU highlight effect for the closest point (so user knows what point is hovered and gets picked, before they click) *this requires modifying the point cloud shader: take mousepos or mouseray and adjust point color based on distance to that (but to make it better, might require collecting those nearby points and sorting, to get actual closest one)

[DriesVRBase]

Yep great points, will try to implement them

[DriesVRBase]

Will post some updates here as well just FYI.
Have been doing some test with the current implementation on the measuring v3 demo sample you've provided with the mouseButtonDown.
Currently this takes around 1.3ms. Of which 1ms is starting the threads. I'll edit this comment once I have better results with a continious thread that's not starting and stopping constantly.

[unitycoder]

another idea to think about:

• if you initially draw fake brush line on screen (where user drags mouse from-to)
• then on mouseup, start processing to find actual points that would be hit (within that line), could do this in multiple threads

in theory that would give similar effect like in drawing tools where the freehand line gets "optimized" into as straighter line (after user releases mouse button).

Or,
using your continuous worker thread(s),
while user is drawing, it would add mouse positions in threadsafe queue (for the processing to catch up, and only if mouse moved x pixels amount) and then as results are coming out into another queue/list, it would draw the connected lines.

[DriesVRBase]

I see what you're getting at. It definitely has a usecase but I'm also using it in VR and I think it might look a bit weird like that. We're also targeting inexperienced VR users so instant user feedback is quite important. But it definitely has usecases for some cases where mainly pc or mobile is targeted.

So the continious thread is almost finished. Getting very good results. on the sample project (the old paper scroll text) it doesn't even come up in the profiler. Going to test a bit larger model now
(Edit: Okay I just read that other threads won't show up in the profiler so yeah that explains it but I don't see a performance impact so I guess it's good) :)

As you can see, in the beginning I'm drawing and at the end I stop drawing and there is no change in the profiler

Edit2: Just tested this on a smaller model (1.2M points instead of 2.4M) but it's a much denser model. And here the points aren't updating as smooth. No performance impact ofcourse but it's not a smooth annotation anymore