UniAD is a Unified Autonomous Driving algorithm framework which follows end-to-end manners, taking multi-view vision input and could output planning results directly. UniAD achieves SOTA performance in many autonomous driving tasks especially on planning task.
(Image taken from the UniAD repo)
This repo demonstrates how to deploy UniAD on NVIDIA Drive Orin platform using TensorRT. Specifically, we trained a tiny version of UniAD (UniAD-tiny) and provide step by step workflow including model training, ONNX model export, and inference with a sample C++ application.
The project for UniAD deployment can be created based on the UniAD repo and BEVFormer_tensorrt repo, on top of them, you will need to copy our prepared tool/config/helper files and apply several patches to make the project environment compatible and deployable. Please follow the instructions at Project Setup to set up the project code base from multiple sources.
We provide a dockfile for your convenience to prepare the environment for both training and deployment, please follow the instructions at Environment Preparation to setup the environment needed in the following steps.
In order to prepare data used for exporting to ONNX and inference with TensorRT, please follow the instructions at Data Preparation to prepare the data for the project. For preparing training data, please refer to the instructions from UniAD.
For efficiency when deploying a UniAD model on DRIVE platform, we trained a tiny version of UniAD(UniAD-tiny), with a smaller ResNet backbone and reduced image size & bev size. After training, the model needs to be exported from Pytorch to ONNX format. Please see Model Training and Exportation for details.
The inference is showcased with a C++ sample application, it loads raw images and other data as input, runs inference with a built TensorRT engine, and outputs the results of tracking and planning with visualization. Please follow the instructions at Inference Application on how to build the TensorRT engine, compile and run the inference application.
The inference application will generate output visualizations to showcase the planning trajectory and dynamic object detection. Notice that post-processing such as collusion correction is not implemented in the current sample. Raw planning trajectory and object detection results are visualized.
In the visualization, green lines depict planning trajectories of the ego car in green bounding box. Detection of other objects are visualized in bounding boxes with different colors, and with white heading and confidence scores marked. The lines starting from the center of those bounding boxes indicate the velocities of the objects.