BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection

Lei Yang · Kaicheng Yu · Tao Tang · Jun Li · Kun Yuan · Li Wang · Xinyu Zhang · Peng Chen

CVPR 2023

BEVHeight is a new vision-based 3D object detector specially designed for roadside scenario. BEVHeight surpasses BEVDepth base- line by a margin of 4.85% and 4.43% on DAIR-V2X-I and Rope3D benchmarks under the traditional clean settings, and by 26.88% on robust settings where external camera parameters changes. We hope our work can shed light on studying more effective feature representation on roadside perception.

News

• [2024/05] BEVHeight is integrated into NVIDIA DeepStream-3D for sensor fusion.
• [2023/03/15] Both arXiv and codebase are released!
• [2023/02/27] BEVHeight got accepted to CVPR 2023!

Incoming

• Release the pretrained models
• Support train and test on a custom dataset

Table of Contents

1. Getting Started
2. Acknowledgment
3. Citation

Getting Started

• Installation
• Prepare Dataset

Train BEVHeight with 8 GPUs

python [EXP_PATH] --amp_backend native -b 8 --gpus 8

Eval BEVHeight with 8 GPUs

python [EXP_PATH] --ckpt_path [CKPT_PATH] -e -b 8 --gpus 8

Experimental Results

• DAIR-V2X-I Dataset

Method	Config File	Range	Car			Pedestrain			Cyclist			model pth
			[email protected]			[email protected]			[email protected]
			Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
BEVHeight	R50_102	[0, 102.4]	77.48	65.46	65.53	26.86	25.53	25.66	51.18	52.43	53.07	model
	R50_140	[0, 140.8]	80.80	75.23	75.31	28.13	26.73	26.88	49.63	52.27	52.98	model
	R101_102	[0, 102.4]	78.06	65.94	65.99	40.45	38.70	38.82	57.61	59.90	60.39	model
	R101_140	[0, 140.8]	81.80	76.19	76.26	38.79	37.94	38.26	58.22	60.49	61.03	model

• Rope3D Dataset

hom_train.pkl

hom_val.pkl

Method	Config File	Range	Car | [email protected]			Big Vehicle | [email protected]			Car | [email protected]			Big Vehicle | [email protected]			model pth
			Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
BEVHeight	R50_102	[0, 102.4]	83.49	72.46	70.17	50.73	47.81	47.80	48.12	42.45	42.34	24.58	26.25	26.28	model
	R50_140	[0, 140.8]	85.46	79.15	79.06	64.38	65.75	65.77	46.39	42.85	42.71	27.21	33.99	34.03	model

Acknowledgment

This project is not possible without the following codebases.

• BEVDepth
• DAIR-V2X
• pypcd

Citation

If you use BEVHeight in your research, please cite our work by using the following BibTeX entry:

@inproceedings{yang2023bevheight,
    title={BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detection},
    author={Yang, Lei and Yu, Kaicheng and Tang, Tao and Li, Jun and Yuan, Kun and Wang, Li and Zhang, Xinyu and Chen, Peng},
    booktitle={IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR)},
    month = mar,
    year={2023}
}