SGGpoint

Official implementation of "Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysis", CVPR 2021

[Paper] [Supp.] [Video]

Figure. Our proposed 3D point-based scene graph generation (SGG_point) framework consisting of three sequential stages, namely, scene graph construction, reasoning, and inference.

Dataset

A quick glance at some features of our cleaned 3DSSG-O27R16 dataset (compared to the original 3DSSG dataset):

• dense point cloud representation with color and normal vector info. encoded - see Sec. A - Point Cloud Sampling;
• with same scene-level split applied on 3DSSG - but with FullScenes (i.e., original graphs) instead of SubScenes (subgraphs of 4-9 nodes in 3DSSG);
• with small / partial scenes of low quality excluded - see this list (officially announced in 3DSSG's FAQ Page);
• with object-level class imbalance alleviated - see Sec. B1 - Node (object) Remapping;
• with edge-wise comparative relationships (e.g., more-comfortable-than) filtered out - we focus on structural relationships instead;
• reformulate the edge predictions from a multi-label classification problem to a multi-class one - see Sec. B2 - Edge (Relationship) Relabelling;

To obtain our preprocessed 3DSSG-O27R16 dataset, please follow the instructions in our project page - or you could also derive these preprocessed data yourselves by following this step-by-step preprocessing guidance with scripts provided.

Code

This repo. also contains Pytorch implementation of the following modules:

• Preprocessing A: 10dimPoints & batch script;
• Preprocessing B: SceneGraphAnnotation.json & Prep. Script;
• dataloader's instructions (might be updated later here);
• SubNetworks.py: Backbones (PointNet & DGCNN), Tails (NodeMLP & EdgeMLP), edge feats. initialization func.;
• EdgeGCN.py: CoreNetwork with two twinning attentions;

Citation

If you find our data or project useful in your research, please cite:

@InProceedings{SGGpoint,
   author    = {Zhang, Chaoyi and Yu, Jianhui and Song, Yang and Cai, Weidong},
   title     = {Exploiting Edge-Oriented Reasoning for 3D Point-Based Scene Graph Analysis},
   booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
   month     = {June},
   year      = {2021},
   pages     = {9705-9715}
}