TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes

By Mutian Xu*, Pei Chen*, Haolin Liu, and Xiaoguang Han

Introduction

This repository is built for:

TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes (ECCV2022 Oral) [arXiv]

If you find our work useful in your research, please consider citing:

@inproceedings{xu2022toscene,
  title={TO-Scene: A Large-scale Dataset for Understanding 3D Tabletop Scenes},
  author={Xu, Mutian and Chen, Pei and Liu, Haolin and Han, Xiaoguang},
  booktitle={ECCV},
  year={2022}
}

Dataset

TO-Scene contains 20,740 scenes with three different variants which are TO_Vanilla, TO_Crowd, TO_ScanNet.

1. Full TO-Scene data:

• You can download our dataset with the corresponding variants from Google Drive:

Format	TO_Vanilla	TO_Crowd	TO_ScanNet
ply (point cloud)	Download (4.3GB)	Download (2.1GB)	Download (4.3GB)
npz (xyz, color, semantic_label, instance_label, bbox)	Download (6.2GB)	Download (2.8GB)	Download (6.5GB)

• Alternatively, for mainland China users, we also provide Baiduyun link:

Format	TO_Vanilla	TO_Crowd	TO_ScanNet
ply (point cloud)	Download (4.3GB)	Download (2.1GB)	Download (4.3GB)
npz (xyz, color, semantic_label, instance_label, bbox)	Download (6.2GB)	Download (2.8GB)	Download (6.5GB)

• Additionally, you can download our TO-Real data we mentioned in the paper,

2. Meta data:

You may also need to download meta_data at Google Drive or Baiduyun, including train/val/test split in format of .txt and pre-calculated mean_size_arr (mean size of objects of different classes).

3. Build your own dataset:

We have released the original CAD placement file (json file, describing the position, scale, and orientation of the small objects). You can run your own rendering and reconstruction based on this original annotation file, with parameters (e.g. point density) in your control.

Benchmark on TO-Scene

We have provided the code implementations for running 3D semantic segmentation and 3D object detection on our dataset, with the corresponding instructions.

Note that TO-Scene dataset contains 60,174 tabletop object instances from 52 common classes. For reference, we show the classes of these small tabletop objects below, which can be downloaded here as well.

Big furniture

Class	Semantic	Class	Semantic	Class	Semantic
3	cabinet	9	window	24	refrigerator
4	bed	10	bookshelf	28	showercurtain
5	chair	11	picture	33	toilet
6	sofa	12	counter	34	sink
7	table	14	desk	36	bathtub
8	door	16	curtain	39	garbagebin

Small tabletop objects

Class	Semantic	Class	Semantic	Class	Semantic	Class	Semantic
41	bag	54	laptop	67	chessboard	80	mirror
42	bottle	55	microphone	68	coffee_machine	81	notebook
43	bowl	56	microwave	69	comb	82	pencil
44	camera	57	mug	70	cutting_board	83	plant
45	can	58	printer	71	dishes	84	plate
46	cap	59	remote_control	72	doll	85	radio
47	clock	60	phone	73	eraser	86	ruler
48	keyboard	61	alarm	74	eye_glasses	87	saucepan
49	display	62	book	75	file_box	88	spoon
50	earphone	63	cake	76	fork	89	tea_pot
51	jar	64	calculator	77	fruit	90	toaster
52	knife	65	candle	78	globe	92	vase
53	lamp	66	charger	79	hat	93	vegetables

Contact

If you have any questions, please contact Mutian Xu ([email protected]) or Pei Chen ([email protected]).

News

The dataset challenge will be released soon, via a complete website!