Awesome Radar Perception

Overview

• Surveys
• Datasets
• Representations
  • ADC Signal
    • Classification/Motion Recognition
    • Object Dection
    • Vital Sign
  • Radar Tensor
    • Classification
    • Detection
    • Segmentation
    • Tracking
    • Multi-Task
  • Point Cloud
    • Classification
    • Detection
    • Segmentation
    • Tracking
    • Scene Flow
    • Odometry
    • Gait Recognition
  • Grid Map
    • Detection
    • Segmentation
  • Micro-Doppler Signature
    • Motion (Gait/Gestures/Activity) Classification
• Calibration
• Citation

Surveys

• 2023 - Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Review arXiv [Paper] [Website] [GitHub]
• 2023 - Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review TIV [Paper] [Website] [GitHub]
• 2023 - Reviewing 3D Object Detectors in the Context of High-Resolution 3+1D Radar CVPR Workshop [Paper]
• 2023 - Radars for Autonomous Driving: A Review of Deep Learning Methods and Challenges ACCESS [Paper]
• 2023 - Vehicle Detection for Autonomous Driving: A Review of Algorithms and Datasets TITS [Paper]
• 2023 - An Overview of Algorithms for Contactless Cardiac Feature Extraction from Radar Signals: Advances and Challenges TIM [Paper]
• 2023 - 4D Millimeter-Wave Radar in Autonomous Driving: A Survey arXiv [Paper]

Datasets

Id	Name	Year	Task	Data Representation	Link
1	nuScenes	2019	Object Detection Object Tracking	Point Cloud	Paper Website Github
2	Astyx	2019	Object Detection	Point Cloud	Website
3	SeeingThroughFog	2020	Object Detection	Point Cloud	Website
4	CARRADA	2020	Object Detection Semantic Segmentation Object Tracking Trajectory Prediction	Range-Doppler Tensor Range-Azimuth Tensor	Website
5	HawkEye	2020	Semantic Segmentation	Point Cloud	Website
6	Zendar	2020	Object Detection Mapping Localization	Range-Doppler Tensor Range-Azimuth Tensor Point Cloud	Website
7	RADIATE	2020	Object Detection Object Tracking SLAM Scene Understanding	Range-Azimuth Tensor	Website
8	AIODrive	2020	Object Detection Object Tracking Semantic Segmentation Trajectory Prediction Depth Estimation	Point Cloud	Website
9	MulRan	2020	Place Recognition	Radar Tensor	Website
10	Oxford Radar RobotCar	2020	Object Detection Odometry	Radar Tensor Grid Map	Website
11	SCORP	2020	Semantic Segmentation	Radar Tensor Grid Map	Paper
12	CRUW	2021	Object Detection	Range-Azimuth Tensor	Website
13	RaDICaL	2021	Object Detection	ADC Signal	Website
14	RadarScenes	2021	Object Detection Semantic Segmentation Object Tracking	Point Cloud	Website
15	RADDet	2021	Object Detection	Range-Azimuth-Doppler Tensor	Github
16	FloW	2021	Object Detection	Range-Doppler Tensor Point Cloud	Website Github
17	RADIal	2021	Object Detection Semantic Segmentation	ADC Signal Range-Azimuth-Doppler Tensor Range-Azimuth Tensor Range-Doppler Tensor Point Cloud	Github
18	Pixset	2021	Object Detection Object Tracking	Point Cloud	Website
19	Open Radar Datasets	2021	Object Classification	Micro-Doppler Signature	Github
20	VoD	2022	Object Detection	4D Point Cloud	Website
21	Boreas	2022	Object Detection Localization Odometry	Range-Azimuth Tensor	Website
22	TJ4DRadSet	2022	Object Detection Object Tracking	4D Point Cloud	Website
23	K-Radar	2022	Object Detection Object Tracking SLAM	4D Range-Azimuth-Doppler Tensor	Github
24	ColoRadar	2022	Localization	Radar Tensor Point Cloud	Paper Website
25	aiMotive	2022	Object Detection	Point cloud	Website
26	WaterScenes	2023	Instance Segmentation Semantic Segmentation Free-space Segmentation Waterline Segmentation Panoptic Perception	4D Point cloud	Paper Website GitHub
27	ThermRad	2023	Object Detection	4D Point Cloud	Paper
28	NTU4DRadLM	2023	SLAM	4D Point Cloud	Paper GitHub
29	Dual Radar	2023	Object Detection Object Tracking	4D Point Cloud	Paper GitHub
30	MiliPoint	2024	Activity Recognition	Point Cloud	Paper GitHub
31	OORD	2024	Place Recognition	Range-Azimuth Tensor	Paper GitHub
32	V2X-Radar	2024	Object Detection	4D Point cloud	Paper GitHub

Representations

ADC Signal

Classification/Motion Recognition

• 2020 - Radar Image Reconstruction from Raw ADC Data using Parametric Variational Autoencoder with Domain Adaptation ICPR [Paper]
• 2021 - Improved Target Detection and Feature Extraction using a Complex-Valued Adaptive Sine Filter on Radar Time Domain Data EUSIPCO [Paper]
• 2021 - Data-Driven Radar Processing Using a Parametric Convolutional Neural Network for Human Activity Classification IEEE Sensors [Paper]
• 2021 - Spiking Neural Network-Based Radar Gesture Recognition System Using Raw ADC Data IEEE Sensors Letters [Paper]
• 2022 - Detection of Human Breathing in Non-Line-of-Sight Region by Using mmWave FMCW Radar TIM [Paper]
• 2023 - CubeLearn: End-to-End Learning for Human Motion Recognition From Raw mmWave Radar Signals IEEE IOT [Paper]

Object Dection

• 2023 - ADCNet: End-to-end perception with raw radar ADC data arXiv [RADIal] [Paper]
• 2023 - T-FFTRadNet: Object Detection with Swin Vision Transformers from Raw ADC Radar Signals arXiv [RADIal] [Paper]
• 2023 - Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion NeurIPS [RADIal][Paper]
• 2023 - Azimuth Super-Resolution for FMCW Radar in Autonomous Driving CVPR [RADIal] [Paper]

Vital Sign

• 2016 - RF-based child occupation detection in the vehicle interior IRS [Paper]
• 2019 - A Theoretical Investigation of the Detection of Vital Signs in Presence of Car Vibrations and RADAR-Based Passenger Classification TVT [Paper]
• 2020 - Non-Contact Vital Signs Monitoring for Multiple Subjects Using a Millimeter Wave FMCW Automotive Radar IMS [Paper]
• 2023 - Sparsity-Based Multi-Person Non-Contact Vital Signs Monitoring via FMCW Radar JBHI [Paper]
• 2023 - Radar-Based Monitoring of Vital Signs: A Tutorial Overview JPROC [Paper]

---

Radar Tensor

Classification

• 2019 - Deep Learning-based Object Classification on Automotive Radar Spectra RadarConf [Paper]
• 2021 - DeepHybrid: Deep learning on automotive radar spectra and reflections for object classification ITSC [Paper]

Detection

• 2019 - Experiments with mmWave Automotive Radar Test-bed RA ACSSC [Paper]
• 2019 - Vehicle Detection With Automotive Radar Using Deep Learning on Range-Azimuth-Doppler Tensors RAD ICCV Workshop [Paper]
• 2020 - Probabilistic oriented object detection in automotive radar RA CVPR Workshop [Paper]
• 2020 - RODNet: Radar Object Detection Using Cross-Modal Supervision RA WACV [CRUW] [Paper]
• 2020 - RODNet: A Real-Time Radar Object Detection Network Cross-Supervised by Camera-Radar Fused Object 3D Localization RA JSTSP [CRUW] [Paper]
• 2020 - Range-Doppler Detection in Automotive Radar with Deep Learning RD IJCNN [Paper]
• 2020 - RAMP-CNN: A Novel Neural Network for Enhanced Automotive Radar Object Recognition RAD IEEE Sensors [Paper]
• 2020 - CNN Based Road User Detection Using the 3D Radar Cube RAD RAL [Paper]
• 2021 - [GTR-Net] Graph Convolutional Networks for 3D Object Detection on Radar Data RAD ICCV Workshop [Paper]
• 2021 - RADDet: Range-Azimuth-Doppler based Radar Object Detection for Dynamic Road Users RAD CRV [RADDet][Paper] [Code]
• 2022 - DAROD: A Deep Automotive Radar Object Detector on Range-Doppler maps RD IV [CARRADA RADDet] [Paper]
• 2022 - A recurrent CNN for online object detection on raw radar frames [CARRADA CRUW] [Paper]
• 2022 - Radatron: Accurate Detection Using Multi-resolution Cascaded MIMO Radar ECCV [Paper]
• 2022 - K-Radar: 4D Radar Object Detection for Autonomous Driving in Various Weather Conditions RADE NeurIPS [K-Radar][Paper] [GitHub]
• 2023 - Enhanced K-Radar: Optimal Density Reduction to Improve Detection Performance and Accessibility of 4D Radar Tensor-based Object Detection RADE arXiv [K-Radar][Paper]
• 2023 - YOLO-ORE: A Deep Learning-Aided Object Recognition Approach for Radar Systems TVT [Paper]

Segmentation

• 2020 - RSS-Net: Weakly-supervised multi-class semantic segmentation with FMCW radar RAD IV [Paper]
• 2020 - Deep Open Space Segmentation using Automotive Radar RAD ICMIM [Paper]
• 2021 - PolarNet: Accelerated Deep Open Space Segmentation using Automotive Radar in Polar Domain RAD VEHITS [Paper]
• 2021 - Multi-view Radar Semantic Segmentation RAD ICCV [Paper] [GitHub]
• 2024 - TransRadar: Adaptive-Directional Transformer for Real-Time Multi-View Radar Semantic Segmentation RAD WACV [Paper] [GitHub]

Tracking

• 2022 - Exploiting Temporal Relations on Radar Perception for Autonomous Driving CVPR [Paper]

Multi-Task

• 2022 - [FFT-RadNet] Raw High-Definition Radar for Multi-Task Learning CVPR [RADIal] [Paper]
• 2023 - Cross-Modal Supervision-Based Multitask Learning With Automotive Radar Raw Data RD TIV [RADIal][Paper]

---

Point Cloud

Classification

• 2017 - Comparison of random forest and long short-term memory network performances in classification tasks using radar SDF [Paper]
• 2018 - Radar-based Feature Design and Multiclass Classification for Road User Recognition IV [Paper]
• 2020 - Off-the-shelf sensor vs. experimental radar - How much resolution is necessary in automotive radar classification? FUSION [Paper]
• 2022 - Radar-PointGNN: Graph Based Object Recognition for Unstructured Radar Point-cloud Data RadarConf [Paper]

Detection

• 2019 - 2D Car Detection in Radar Data with PointNets ITSC [Paper]
• 2020 - Detection and Tracking on Automotive Radar Data with Deep Learning FUSION [Paper]
• 2020 - Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radar CVPR [Paper]
• 2021 - RPFA-Net: a 4D RaDAR Pillar Feature Attention Network for 3D Object Detection ITSC [Paper] [Code]
• 2021 - Comparison of Different Approaches for Identification of Radar Ghost Detections in Automotive Scenarios RadarConf [Paper]
• 2022 - Contrastive Learning for Automotive mmWave Radar Detection Points Based Instance Segmentation ITSC [Paper]
• 2023 - 3-D Object Detection for Multiframe 4-D Automotive Millimeter-Wave Radar Point Cloud IEEE Sensors [TJ4DRadSet][Paper]
• 2023 - SMURF: Spatial Multi-Representation Fusion for 3D Object Detection with 4D Imaging Radar TIV [VoD TJ4DRadSet][Paper]
• 2023 - MVFAN: Multi-View Feature Assisted Network for 4D Radar Object Detection ICONIP [Astyx VoD] [Paper]
• 2023 - See Beyond Seeing: Robust 3D Object Detection from Point Clouds via Cross-Modal Hallucination [VoD] [Paper]
• 2024 - DenserRadar: A 4D millimeter-wave radar point cloud detector based on dense LiDAR point clouds [K-Radar] [Paper]
• 2024 - Exploring Domain Shift on Radar-Based 3D Object Detection Amidst Diverse Environmental Conditions ITSC [K-Radar] [Paper]
• 2024 - RadarPillars: Efficient Object Detection from 4D Radar Point Clouds ITSC [VoD] [Paper]
• 2024 - L4DR: LiDAR-4DRadar Fusion for Weather-Robust 3D Object Detection arXiv [VoD, K-Radar] [Paper]
• 2024 - Feature-enhanced PointPillars for 3D Millimeter-wave Object Detection TAES [Paper]

Segmentation

• 2018 - Semantic Segmentation on Radar Point Clouds FUSION [Paper]
• 2018 - Supervised Clustering for Radar Applications: On the Way to Radar Instance Segmentation ICMIM [Paper]
• 2019 - 2D Car Detection in Radar Data with PointNets ITSC [Paper]
• 2020 - RSS-Net: Weakly-Supervised Multi-Class Semantic Segmentation with FMCW Radar IV [Paper]
• 2022 - Panoptic Segmentation for Automotive Radar Point Cloud RadarConf [Paper]
• 2023 - Deep Instance Segmentation With Automotive Radar Detection Points TIV [RadarScenes] [Paper]

Tracking

• 2020 - Detection and Tracking on Automotive Radar Data with Deep Learning FUSION [Paper]
• 2021 - Long-Range Multi-Object Tracking at Traffic Intersections on Low-Power Devices TITS [Paper]
• 2023 - Which Framework is Suitable for Online 3D Multi-Object Tracking for Autonomous Driving with Automotive 4D Imaging Radar? arXiv [Paper]
• 2024 - RaTrack: Moving Object Detection and Tracking with 4D Radar Point Cloud ICRA [VoD] [Paper] [Code]

Scene Flow

• 2022 - Self-Supervised Scene Flow Estimation With 4-D Automotive Radar RAL [Paper Code]
• 2023 - Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision-Supplementary Material CVPR [VoD] [Paper Code]

Odometry

• 2023 - Efficient Deep-Learning 4D Automotive Radar Odometry Method TIV [Paper]
• 2023 - DRIO: Robust Radar-Inertial Odometry in Dynamic Environments RAL [Paper]

Gait Recognition

• 2021 - Person Reidentification Based on Automotive Radar Point Clouds TGRS [Paper]
• 2020 - Gait Recognition for Co-Existing Multiple People Using Millimeter Wave Sensing AAAI [Paper]

---

Grid Map

Detection

• 2015 - Automotive Radar Gridmap Representations ICMIM [Paper]
• 2015 - Detection of Arbitrarily Rotated Parked Cars Based on Radar Sensors IRS [Paper]
• 2016 - 3D Occupancy Grid Mapping Using Statistical Radar Models IV [Paper]
• 2017 - Semantic Radar Grids IV [Paper]
• 2018 - Adaptions for Automotive Radar Based Occupancy Gridmaps ICMIM [Paper]
• 2018 - High Resolution Radar-based Occupancy Grid Mapping and Free Space Detection VEHITS [Paper]
• 2019 - Semantic Segmentation on Automotive Radar Maps IV [Paper]
• 2019 - Occupancy Grids Generation Using Deep Radar Network for Autonomous Driving ITSC [Paper]
• 2020 - Semantic Segmentation on 3D Occupancy Grids for Automotive Radar IEEE ACCESS [Paper]
• 2023 - NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for Autonomous Driving RA ICRA [Paper]

Segmentation

• 2019 - Road Scene Understanding by Occupancy Grid Learning from Sparse Radar Clusters using Semantic Segmentation ICCV Workshop [Paper]
• CNN based road course estimation on automotive radar data with various gridmaps ICMIM [Paper]
• 2020 - Scene Understanding With Automotive Radar TIV [Paper]
• 2023 - Semantic Segmentation-Based Occupancy Grid Map Learning With Automotive Radar Raw Data TIV [RADIal][Paper]

---

Micro-Doppler Signature

Motion (Gait/Gestures/Activity) Classification

• 2016 - Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks IEEE Geoscience and Remote Sensing Letters [Paper]
• 2017 - New Analysis of Radar Micro-Doppler Gait Signatures for Rehabilitation and Assisted Living ICASSP [Paper]
• 2018 - Human Motion Classification with Micro-Doppler Radar and Bayesian-Optimized Convolutional Neural Networks ICASSP [Paper]
• 2018 - Radar-Based Human-Motion Recognition With Deep Learning: Promising Applications for Indoor Monitoring IEEE Signal Processing Magazine [Paper]
• 2019 - Radar-Based Human Gait Recognition Using Dual-Channel Deep Convolutional Neural Network TGRS [Paper]
• 2019 - Experiments with mmWave Automotive Radar Test-bed ACSSC [Paper]
• 2022 - Attention-Based Dual-Stream Vision Transformer for Radar Gait Recognition ICASSP [Paper]

Calibration

• 2019 - Extrinsic 6DoF calibration of a radar–LiDAR–camera system enhanced by radar cross section estimates evaluation [Paper]
• 2021 - A Joint Extrinsic Calibration Tool for Radar, Camera and Lidar TIV [Paper]
• 2021 - 3D Detection and Tracking for On-road Vehicles with a Monovision Camera and Dual Low-cost 4D mmWave Radars ITSC [Paper]
• 2022 - 3DRadar2ThermalCalib: Accurate Extrinsic Calibration between a 3D mmWave Radar and a Thermal Camera Using a Spherical-Trihedral ITSC [Paper]
• 2022 - K-Radar: 4D Radar Object Detection for Autonomous Driving in Various Weather Conditions NeurIPS [Paper] [GitHub]

Citation

Please use the following citation when referencing

@misc{yao2023radarperception,
      title={Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Review}, 
      author={Shanliang Yao and Runwei Guan and Zitian Peng and Chenhang Xu and Yilu Shi and Weiping Ding and Eng Gee Lim and Yong Yue and Hyungjoon Seo and Ka Lok Man and Jieming Ma and Xiaohui Zhu and Yutao Yue},
      year={2023},
      eprint={2312.04861},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}