Questions about camera extrinsics

[linxuewu]

Thank you for the valuable open-source data. I have some confusion regarding the content of the data:

1. How is the world coordinate system defined? Specifically, where is the origin and what are the positive directions of the axes?
2. Why do the extrinsic parameters of the fixed camera vary across different episodes within the same scene?
3. When using the camera's intrinsic and extrinsic parameters to project cartesian_position onto images, I noticed some bias. What could be the reason?

[linxuewu]

vis_imgs.mp4

This is a data visualization with projection bias; the data path is '1.0.1/ILIAD/success/2023-06-11/Sun_Jun_11_14:56:46_2023'.

[linxuewu]

vis_imgs.1.mp4

And, this is a data visualization without projection bias; the data path is '1.0.1/ILIAD/success/2023-06-11/Sun_Jun_11_15:52:37_2023'.

This data was recorded only one minute apart from the previous one, and the camera position does not appear to have changed based on the video; however, there is a notable difference in the extrinsic parameters.

[AniruthSuresh]

3. When using the camera's intrinsic and extrinsic parameters to project `cartesian_position` onto images, I noticed some bias. What could be the reason?

Can you please share on how you extracted the intrinsics of the camera ? I cant seem to find it . Can you share where its present in the dataset or a script to extract this . Thanks

[mluogh]

IDK about 3 but for

1. I'm fairly certain they're defined with the robot base as origin. If I place cameras in MuJoCo using the extrinsics with robot at world origin, it produces the same view as the IRL videos (+- some coordinate convention differences)

2. Do they? I haven't seen this, only the fixed camera intrinsics actually changing because the camera shakes irl.