Disrepancy in IoU calculation with LiDAR and Camera Mask

[seamie6]

I noticed in your config you seem to only calculate IoU wrt the camera mask, and do not use the LiDAR mask:

val_evaluator = dict(
    type='OccMetric',
    num_classes=18,
    use_lidar_mask=False,
    use_image_mask=True)

However, in GaussianOcc (and OccNeRF) it appears they do use this in nusc_dataset.py with the logical AND:

label = np.load(label_path)
semantics_3d, mask_camera_3d, mask_lidar_3d = label['semantics'], label['mask_camera'], label['mask_lidar']
inputs['semantics_3d'], inputs['mask_camera_3d'] = semantics_3d, np.logical_and(mask_camera_3d, mask_lidar_3d)

Am I misinterpreting this?

[npurson]

Thank you for bringing this to our attention!

Our implementation is based on FlashOcc and FBOcc, both of which use camera masks, as seen in nuscenes_dataset_occ.py#L128 and nuscenes_dataset.py#L698.

After reviewing relevant works following your observation, we found that most other approaches remain use camera masks, including SelfOcc (eval_iou.py#L236), SparseOcc (old_metrics.py#L17), and LangOcc (nuscenes_dataset_occ.py#L75).

We are currently unaware of the rationale behind GaussianOcc's implementation and the performance implications. We may investigate this in the future.

Let me know if you have further questions.