Problem with Rotating Picker attached to Camera

[SteveGreatApe]

Hi, I've converted some code over to the latest pull of the framework code to try the picker changes, I've based my picker code on that found n the gvr-controller sample, using inputManager.selectController(), to access GVRPicker I've added a call to newController.getPicker().

    inputManager.selectController(new GVRInputManager.ICursorControllerSelectListener() {
        public void onCursorControllerSelected(GVRCursorController newController, GVRCursorController oldController) {
            GVRPicker picker = newController.getPicker();
            EnumSet<GVRPicker.EventOptions> eventOptions = picker.getEventOptions();
            eventOptions.add(GVRPicker.EventOptions.SEND_TO_HIT_OBJECT);
            ...

I've got it working okay so far when not attached to a moving camera. But when I attach the picker to the camera with:

        cameraRig.getHeadTransformObject().attachComponent(picker);

Then I rotate the camera with the following code.

        cameraRig.getTransform().setRotationByAxis((float)playerAngle, 0.0f, 1.0f, 0.0f);

The Picker doesn't rotate with the camera, even though it does move position with the camera okay when setting the transform position. (Except for an issue with flicker, see following report).

Is there a problem with the way I'm doing this, or is it a framework issue? Are there any demo projects that show the picker attached to the camera?

[NolaDonato]

You don't have to attach the picker to the camera. If you are using the gaze controller, it will do this automatically for you. If you are using the Gear controller, the picker is attached the the controller in the scene, not the camera. In your gvr.xml file, you should have a line that looks like this:
useControllerTypes='gaze,controller'
This tells it to enable the gaze or Gear controller for your application. If it detects a connected Gear controller, the picker is attached to that and you use the controller for picking. Otherwise it defaults to the gaze controller and attaches the picker to the camera.

[SteveGreatApe]

Okay, I've removed the code attaching the picker to the camera rig, I can see it wasn't required now as it does move with the camera regardless. But the problem is still the same, it still doesn't rotate with the camera.

[NolaDonato]

What do you mean by "the picker does not rotate with the camera?" GVRPicker is a component that is attached to the GVRSceneObject. GVRCameraRig is also a component attached to the same GVRSceneObject. The orientation of the camera rig comes from the headset. If you rotate the camera rig, the headset sensors will overwrite that every frame.

I am not sure what you are trying to achieve by rotating the camera. Do you not want it to track the headset?

[SteveGreatApe]

The way I rotate the camera rig is I use GVRContext registerDrawFrameListener() call to set up a callback per frame, in that frame I do the following.

    @Override
    public synchronized void onDrawFrame(float frameTime) {
...
// Use trackpad input to update player position and orientation by setting variables mCameraRigPos. & mPlayerAngle
...
// Set the players position and orientation in the scene by setting the camera rig transform
            mCameraRig.getTransform().setPosition(mCameraRigPos.x, mCameraRigPos.y, mCameraRigPos.z);
            mCameraRig.getTransform().setRotationByAxis(mPlayerAngle, 0.0f, 1.0f, 0.0f);
    }

This is always used to work fine with older versions with the way I attached the picker to the camera rig. It now still works okay (except for flickering) with regards to the camera position, but no longer adjusts the picker when the camera rig is rotated.

[NolaDonato]

Are you using the Gear controller or the gaze controller? If you are using the Gear controller, the picker gets its orientation from the controller, not the camera.

[NolaDonato]

You can still attach your own picker to the camera rig and ignore the picker and ignore the controller if you want. It will work exactly the way it did before. To do that, remove "useControllerTypes" from your gvr.xml file and don't call GVRInputManager.selectController. Then your app will be responsible for making the picker, attaching the cursor, etc. like before.

I am curious what you are trying to achieve by changing the camera rig orientation. Are you trying to override the information from the head tracking sensors? If I understand better what you are trying to do I might be able to suggest an alternative that would work better in GearVRf. If you would be willing to share a section of your application I would be happy to look it over and see if there are more efficient ways to achieve what you want.

[SteveGreatApe]

This is with the Gear controller, it's nothing strange that I'm doing. I'm just using the camera rig orientation to reflect the direction the user's body is facing, then the usual head tracking works to reflect the direction they're looking relative to the direction their body is facing, I'd expect the picker orientation to also be relative to that.

A simple use case is something like allowing the user to wander around a room facing and walking in any direction they like? Are you expecting them to physically turn themselves in the real world to match they direction they want to walk in the program? That's not how I'd expect most apps to work unless they wanted an excessive level of immersion. It's much easier for the user to be able to sit down facing one way then use some gesture on the controller such as left/right swipes or clicks on the touchpad to rotate their virtual selves in the VR environment.

[NolaDonato]

PR #1646 should fix this with ICameraEvents.onViewChange. Let me know if it works for you now.

[SteveGreatApe]

Thanks, I've given this a go, but I'm having a problem receiving the onViewChange events. What I've done is:

        ICameraEvents cameraEvents = new ICameraEvents() {
            public void onViewChange(GVRCameraRig rig) {
                Log.d(TAG, "onViewChange:");
                updateCameraRig(rig);
            }
        };
        scene.getGVRContext().getEventReceiver().addListener(cameraEvents);

Occasionally the log shows a short burst of onViewChange events right at the start, but then they stop arriving. More often I simply never receive any onViewChange events.

[NolaDonato]

@SteveGreatApe can you tell me the status of this bug? I know onViewChange did not work and we removed it. Are you still having issues with the camera orientation and the controller?

[SteveGreatApe]

The mod I posted in #1686 is working fine for dealing with controller orientation, guess I should have posted that under this thread not that one.

[NolaDonato]

I made a pull request #1731 that is a more thread-safe version of your mod (It accesses the model matrix once as opposed to getting the rotation and then the translation in case the GL thread interrupts between these two). I have tested it with our demos which exercise the controller and it seems to work fine. Please test this PR with your application to make sure I got it right.