Object-Compositional Neural Implicit Surfaces

Project Page | Paper

This repository contains the official implementation of the ECCV2022 paper:
Object-Compositional Neural Implicit Surfaces.
Qianyi Wu, Xian Liu, Yuedong Chen, Kejie Li, Chuanxia Zheng, Jianfei Cai, Jianmin Zheng.
The paper introduces ObjectSDF: a volume rendering framework for object-compositional implicit neural surfaces, allowing to learn high fidelity geometry of each object from a sparse set of input images and the corresponding semantic segmentation maps.

Setup

Installation Requirmenets

The code is compatible with python 3.9 and pytorch 1.10.1. In addition, the following packages are required: numpy, pyhocon, plotly, scikit-image, trimesh, imageio, opencv, torchvision.

You can create an anaconda environment called objsdf with the required dependencies by running:

conda env create -f environment.yml
conda activate objsdf

Data

We provide the installation guidence to use our code in Toydesk dataset. At first, you need to download Toydesk dataset and put it in the './data' folder. Then

cd data
bash process_toydesk.sh

We require the RGB images with the corresponding semantic segmentation map for model training. After running the above script, you will get the corresponding file for two desk scenes.

Usage

For example, if you would like to train ObjectSDF on Toydesk 2, please run:

cd ./code
python training/exp_runner.py --conf confs/toydesk2.conf --train_type objsdf

Citation

If you use this project for your research, please cite our paper.

@inproceedings{wu2022object,
  title={Object-compositional neural implicit surfaces},
  author={Wu, Qianyi and Liu, Xian and Chen, Yuedong and Li, Kejie and Zheng, Chuanxia and Cai, Jianfei and Zheng, Jianmin},
  booktitle={European Conference on Computer Vision},
  pages={197--213},
  year={2022},
  organization={Springer}
}

Related Links

If you are interested in NeRF / neural implicit representions + semantic map, we would also like to recommend you to check out our other related works:

• Neural implicit generative model, Sem2NeRF.

• Digital speaking portrait animation, SSPNeRF.

Acknowledgments

Our implementation was mainly inspired by VolSDF.