[Bug] camera extrinsics

[Yangr116]

Prerequisite

• I have searched Issues and Discussions but cannot get the expected help.
• I have read the FAQ documentation but cannot get the expected help.
• The bug has not been fixed in the latest version (dev-1.x) or latest version (dev-1.0).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

torch

Reproduces the problem - code sample

Background:

I would like to convert the bbox of your great dataset into the format of Omni3D. So, I need to convert get the rotation matrix of a bbox (from the initial bbox coordinate into the camera coordinate).

My camera coordinate is: X towards right, Y towards down, Z towards into the screen. I think this is aligned with the camera coordinate in this dataset. (

EmbodiedScan/embodiedscan/structures/bbox_3d/box_3d_mode.py

Lines 30 to 41 in fe26e4b

	Coordinates in Camera:
	.. code-block:: none
	z front
	/
	/
	0 ------> x right
	|
	|
	v
	down y

)

I find that the bbox in this dataset lies in the world coordinate according to this code:

EmbodiedScan/embodiedscan/visualization/img_drawer.py

Line 107 in fe26e4b

corners_img = intrinsic @ extrinsic_w2c @ corners.transpose()

When I visualize the box using my visualization code. I find the top face (blue-green) of the box is to the left:

I print the 3d bbox info of a data point in the scannet:

explorer.show_image('scannet/scene0415_00', '00000', render_box=True)

And I add the below code into the line in

EmbodiedScan/embodiedscan/explorer.py

Line 546 in fe26e4b

instance = select['instances'][i]

to check the bbox information:

                        import math
                        from scipy.spatial.transform import Rotation
                        print(f"#### {self.classes[self.id_to_index[instance['bbox_label_3d']]]}####")
                        print(f"bbox_3d in ann: {instance['bbox_3d']}")
                        angle = [1, 1, 1]
                        angle[:3] = map(lambda x: x*180 / math.pi, instance['bbox_3d'][-3:])
                        print(f"angle {angle}")
                        print(f"extrinsic: {extrinsic}")
                        extrinsic_w2c = np.linalg.inv(extrinsic)
                        rotmat = Rotation.from_matrix(extrinsic_w2c[:3, :3])
                        euler_angle = rotmat.as_euler('xyz', degrees=True).tolist()
                        print(f"extrinsic angle: {euler_angle}")

For a bounding box:

toilet####

bbox_3d in ann: [-0.42515089947718376, 0.31674408904254436, 0.3837619480912089, 0.7353889074944258, 0.4506813515427113, 0.8246120158582926, -0.0, 0.0, -0.0]
angle [-0.0, 0.0, -0.0]
extrinsic: [[-0.03071581 0.58103032 -0.81330209 0.29357407]
[ 0.9991814 0.03927344 -0.00967998 0.18009791]
[ 0.026316 -0.812934 -0.581761 1.199998 ]
[ 0. 0. 0. 1. ]]
extrinsic angle: [-179.04680239144434, 54.41982902742494, 93.02611718726739]

This shows that the Z-axis is rotated 90.

Why Z-axis is rotated 90?

Reproduces the problem - command or script

see before

Reproduces the problem - error message

rotation problem

Additional information

No response