Controller flickering when moving with camera position #1686
Comments
|
A bit of extra detail, I'm updating the camera rig position in a GVRDrawFrameListener.onDrawFrame() callback. I can see from looking in GearCursorController.java that it's posting a message to the EventHandlerThread to call the handleControllerEvent() function that will update the controller position relative to the Camera rig. So that update is getting done out of sync with the other updates that running on the GL Thread which would explain the flickering. |
|
The controller will issue an event every frame. You can update the camera rig right after you handle the controller event and avoid the flickering. |
|
We are looking into enabling a thread-safe way for you to modify the camera transform and avoid flickering. It doesn't work in the draw frame listener because it needs to be done right after the head sensors are read. I might be able to add something for you this week - I need to consult with a colleague to figure out how to do it. |
|
I have added a feature to allow you to update the camera rig safely. It is in PR #1646. The ICameraEvents interface defines the onViewChange function which is not called IMMEDIATELY after the camera rig is updated from the sensor EVERY FRAME. I wrote a test to verify the handler is called. I did not verify it fixes the flickering. Let me know if it still flickers. ICameraEvents myHandler = new ICameraEvents() |
|
@SteveGreatApe Please help me reproduce. Made some tweaks to gvr-controller in https://github.com/liaxim/GearVRf-Demos/tree/test1686. Is this what you have in mind? Thanks. |
|
I've updated my GVRf Keyboard project at https://github.com/SteveGreatApe/AvrKeyboard/tree/GVRfv4x to allow you to move around the environment with the controller. To use it checkout the GVRfv4x branch and update the following path in build.gradle to point to your GearVRf folder: Once you've got it up and running use the controller touchpad to navigate, swipe left and right to rotate, and press down on the touchpad to move in a direction corresponding to your touch position. |
|
Ok, it is not exactly flicker and it is not exactly a thread-safety problem from what I can tell thus far. Your app is rendering at ~30fps. If continuously pressing the button, the controller model kind of gets left behind. I see a lag of 30+ ms from the time the camera rig is updated to the time the handleControllerEvent runs. Pretty much at the same time the camera rig jumps to the next location but the model is at the one that is 30+ ms old. Every once in a while the stars align and handleControllerEvent gets much more recent camera rig position, the model jumps to a more correct location but then it starts lagging again. Still trying to comprehend the problem. To me it seems it is all about the lag right now. |
|
@SteveGreatApe Please instead of calling directly updateCameraRig from onDrawFrameListener do this: I find a great improvement. I see it as more of a workaround though. |
|
Should GearVRf emit the controller events post-render too? |
|
I removed ICameraEvents since this approach was flawed |
|
Regarding controller events from post-render - undecided. I rather look into what are the problems first. |
|
Is there any reason the Picker can't simply be attached to the Camera Rig like it was in 4.0? That worked with no problems and avoided the problem with it not rotating with the camera as well as this flicker issue. |
|
You can attach a picker to the camera any time you want. This still works. You can ignore the picker attached to the controller if you want to. The camera picker will produce pick events (IPickEvents) but not touch events (ITouchEvents). Each controller has a picker so it can generate touch events. In the case of the Gaze controller, the picker is attached to the camera rig. For the Gear controller. the picker is attached to the controller model in the scene so it will pick from the controller instead of the eye. |
|
I think I see some confusion, I've been using the term Picker to refer the Controller & Picker combined. So what I should actually be saying is I want the Controller attached to the camera rig so it moves and rotates with it. At the moment you're manually moving it in sync with the camera rig, but not rotating it with the rig. Whereas if the controller was attached to the camera rig it would automatically move and rotate with it, you'd just need to update the relative position and rotation from the received ControllerEvent's. |
|
The controller's position and orientation is reported to us by the system (either Oculus or Daydream). If the controller was attached to the camera rig, we would still need to compute the inverse of the camera rig's transform to maintain a global position that is relative to the camera. The problem would not go away, it would just move somewhere else. I made a small change to my pull request to help synchronize the picking better. It performs the picking part of controller event processing in the GL thread. See if it fixes your problem. |
|
It still looks flickery to me when moving around. I've come up with a change that fixes the orientation issue, this matches the controller orientation with the camera rig. Add the "Steve's Mod" section into GearCursorController.java |
|
I am looking at your modification to see if what effect it has on the rest of our sample applications. I will include this mod if it doesn't break stuff. |
|
Thanks, I've been using it since posting it above and it's working fine for dealing with rotating the controller with the camera rig. But the flicker and lag issue still exists, is there a reason for not updating the controller postion in GearCursorController.onDrawFrame() itself, rather than posting the update to the EventHandlerThread? |
|
I agree that the code which updates the controller model should be in onDrawFrame. The event generation can stay in the handler thread. I will make this change to PR #1731 |
|
I have eliminated the event handler thread entirely for the Gear controller. I am reviewing whether it is needed for the other controllers. You should not see flickering now. If this fix works for your app we will merge it in. |
|
Hi, I've tried using your NolaDonato:controllerfix branch to test this, but I can't get it start up now. It's crashing after getting an error binding a texture. The last log line on thread that dies is as below, followed by the Fatal Signal 11 error. I sometimes get log lines from other threads in between. But it seems this pair consistently go together for the thread that is crashing. The crash happens after the call to my MainActivity onCreate function has completed including a call to setMain() with my GVRMain derived class. But it doesn't reach my GVRMain onInit function before it dies. So I've done very little work in my app that could have triggered the problem. Here's the full log for the thread that crashed, this includes an extra log line of the form "ZZZ glBindTexture %d" to log whenever it's binding a texture, and STOP_ON_ERROR defined in gvr_log.h so it stops straight after the error with "Fatal signal 6". |
|
I rebased my branch with master to prepare for merging. I will look into this. |
|
@SteveGreatApe Do you get the error at the exact same texture #12 each time you launch? How about after modifying some files of your code and relaunch? SIGSEGV is segmentation fault. Wondering if corrupt memory somehow changed the values of parameters to glBindTexture() thus throwing GL_INVALID_ENUM. |
|
I just rebased this branch and applied some fixes from another PR which should take care of this crashing. Please try again and let me know if it still crashes for you. |
|
Thanks, it's working now. I found a couple of issues that I've sorted out. There was still the problem I mentioned before where the controller orientation wasn't getting adjusted properly as you rotate the camera rig. I've fixed that by swapping the order of the multiplication between mTempPivotMtx and q. There was still a lag of a frame between updating the camera position and updating the controller position. This is because in OvrViewManager.onDrawFrame it was calling mGearController.onDrawFrame() after actually drawing the frame, whereas usual onDrawFrame callbacks in the application are called just before drawing the frame. I've swapped the order around and it's working okay for me, not sure if there was a reason for doing it the old way around that will makes this more complicated, but seems fine for me at least. |
|
Thanks for your testing! I will test your changes against our samples to make sure they don't break anything. |
|
Hi, I've just pulled from the Master branch and found it's got the fix merged in, but it's missing the two changes I suggested above. |
|
My bad. Wasn't paying attention when merging. Try PR #1775 |
|
Thanks, that seems to fix it |
|
@SteveGreatApe Planning to close this issue soon. |
This is using the same controller code as in #1685, using the latest pull of the framework code. I've based my picker code on that found in the gvr-controller sample, using inputManager.selectController(), to access GVRPicker I've added a call to newController.getPicker().
I attach the picker to the camera with:
When I move the camera rig the controller moves with the camera okay, but it displays a very bad flicker, there's two very clear flickering instances of the picker visible whenever the camera position is moving.
The text was updated successfully, but these errors were encountered: