-
Notifications
You must be signed in to change notification settings - Fork 17
Home
This project contains instructions, code and other artifacts to rebuild the Nvidia Redtail project with a drone running the Arducopter flight stack and the latest Nvidia Jetpack 4.2.x on a TX2. I am using a stereo camera as default since it provides a more sophisticated depth calculation compared to mono cameras as used in the original Redtail project.
This wiki will not cover the build instructions and setup for a drone since this is covered in detail in the original Redtail wiki. Focus is on the changes made to make this project run on the Arducopter flightstack and on the Jetpack 4.2.x release.
In order to rebuild the project, the following components are needed:
- A Jetson TX2 and a small size carrier board, such as the Auvidia J120 to mount the TX2 on a drone
- A drone with enough thrust and size to carry all the needed hardware components, eg a Tarot 650 frame
- A Pixhawk cube with Arducopter installed
- Needed sensors: an optical flow sensor to navigate w/o GPS, a Lidar ground distance sensor. I am using the original PX4Flow along with a Garmin LidarLite.
- A ZED Stereo camera
- A joystick eg Logitech 710
- A laptop running Ubuntu 16.04 or 18.04 is a convenient way to run GCS software like QGroundcontrol as well to control the drone using a Joystick.
The NVIDIA Jetson platform is used to run most of the components, such as DNN inference, the controller, and video streaming. Install Jetpack 4.2.x by using the Nvidia SDKmanager. If you are using the Auvidea J120 breakout board, you need to apply some kernel patches as described here
Important: you need to have apsync installed in order to have all drone communication set up correctly. Code and instructions can be found here
The Arducopter wiki describes how to connect the Jetson with the Pixhawk here and how to configure Arducopter to communicate with the TX2.
Do not install the TX2 image file referenced in the Arducopter wiki since its outdated.
apsync creates a WiFi access point with both SSID and password set as ardupilot on the TX2. Instead of using the bulky wifi antennas supplied with the TX2 development board, you may add a small WiFi antenna to your Jetson board, such as a Taoglas Limited FXP522.A.07.A.001 dual channel 3dBi antenna which is very small and can lay flat on one part of your drone's surface and also is powerful enough to be able to connect to Jetson module using WiFi from several meters away.
Connect to this WiFi access point using a computer you can SSH to Jetson module from. Apsync creates an apysync username for Jetson and sets it as it's default login, asking you to set the password for this account during installation. It also sets the default IP address of the module to 10.0.1.128 static address. You can either use the IP address to SSH to the module or add apsync 10.0.1.128 to your known hosts in /etc/hosts and use the hostname for SSH:
$ ssh [email protected]If you are using a Tarot frame, this repo contains the 3D CAD files to build a one-axis gimbal for the ZED Stereo camera.
To install Redtail, please follow the steps as described here.
This shell script downloads and installs all the dependencies as well as building the code for this project. Make sure that the apsync wifi access point is set up and the Jetson module has internet access through LAN cable connected to the J120 board. The script needs a small interaction such as accepting the licences and entering super user's password for installing dependencies such as ROS. This script installs all the dependencies, clones a copy of this repository, creates a Catkin workspace in ~/catkin_ws using Python Catkin tools and build all the required codes.
In order to communicate to mavlink through mavros, make sure you have opened a UDP port in your mavlink-router. Edit ~/start_mavlink-router/mavlink-router.conf and add the following lines at the end:
[UdpEndpoint to_ros]
Mode = Normal
Address = 127.0.0.1
Port = 14855
First, install ROS on your PC. For Ubuntu 16.04 (Xenial) install ROS Kinetic Desktop. For instructions see here. For Ubuntu 18.04, install ROS Melodic Desktop, see here
Then install the required joystick driver for your ROS version (kinetic or melodic): http://wiki.ros.org/joy and test if it works as described here.
You need to set the following environment variables and add them to your .bashrc.
$ export ROS_MASTER_URI=http://10.0.1.128:11311 #the TX2 IP address should be 10.0.1.128 if you are using apsync
$ export ROS_IP=10.0.1.xxx # enter here the IP of your host PC to find out, use ifconfig
Start the joystick node on the host PC:
rosrun joy joy_node _dev:=/dev/input/js0 & # check the correct address of your joystick with ls -l /dev/input
To test if the joystick is operating correctly with ROS and communicating with the ROS master on the TX2: execute the following commands in a terminal window on the TX2:
roscore &
rostopic echo /joy
Alternatively, you can install the joystick support in a docker container as described in the original Nvidia Redtail project on your host PC.
Note: It seems that Jetpack 4.2.x has an issue with the ROS joystick drivers. I was not able to get it running. Therefore I recommend to connect the joystick to the host PC as described above.
The project's AI that enables autonomous navigation is based on a deep neural network (DNN) which can be trained from scratch using publicly available data. A few pre-trained DNNs are also available as a part of this project including Trailnet, YOLO, and a few TensorRT stereo-DNNs for use with the ZED camera, including NVTiny, NVsmall, resnet18 and resnet18_2D.
Please see the original Nvidia Redteil repo for a more detailed description of the different models and instructions on how to train new models.
Thorough testing of the end2end setup is mandatory. Follow these steps. Before you start testing the code, please verify that your drone flies properly without the TX2 connected. Please refer to the Arducopter wiki on how to tune and maiden the copter.
Once the hardware and software setup steps are complete, it's time to take off! Follow these steps to fly the drone.