Event-Orin drone

Overview repository for the Event-Orin drone.

git clone --recurse-submodules [email protected]:tudelft/event_orin_drone.git

Or without --recurse-submodules if you don't need PX4 etc.

Repository structure

Airborne stuff:

• fc: flight controller and software (PX4)
• drone: drone parts and setup
• orin: Jetson Orin setup and config
• sensors: sensor setup and config

Ground stuff:

• gs: ground station software (QGroundControl)
• sim: extras for PX4 simulation

Communication stuff:

• comms: communication setup (MAVLink Router, uXRCE agent)
• wifi: wifi setup
• radio: radio transmitter setup

ROS:

• ros: ROS2 setup and config

Misc:

• scripts: some useful scripts for real-world flight
• prints: 3D prints for the drone

If a folder/link doesn't exist, there's nothing there (yet).

Tested on:

• Ubuntu 22.04 (WSL) on Windows 11 with NVIDIA GPU
• Jetson Orin NX 16 GB with JetPack 6.0 and Jetson Linux 36.3

System overview

Components

FC

• Holybro Kakute H7 Mini flight controller
• Disable ESC beeping/add a song:
  • BLHeli Suite (.zip file for windows)
  • Follow this guide roughly
    • No more flashing: eol
  • Add music: follow this
    • Imperial march
  • Disable inactivity beep: turn beacon strenth to zero as here
• Flash Kakute H7 Mini flight controller with PX4 firmware
  • Initially has Betaflight, so use Betaflight Configurator to flash the bootloader following this
    • Doesn't always work?
    • No DFU mode, no expert mode, .hex bootloader file
  • Build firmware: make holybro_kakuteh7mini_default (file here)
    • Note: this will fail if the correct tags are not checked out in the PX4 repo; to do so, run git remote add upstream [email protected]:PX4/PX4-Autopilot.git && git fetch upstream && git fetch upstream --tags && git push --tags following this
  • Flash with QGroundControl
  • Upload parameter file

Drone

Component	Product	Mass [g]	Approx. power [W]
Frame	Armattan Marmotte 5 inch	455	200
Motors	Emax ECO II Series 2306
Propellers	Ethix S5 5 inch
Flight controller	Holybro Kakute H7 Mini
Optical flow & range sensor	MicoAir MTF-01
ESC	Holybro Tekko32 F4 4in1 mini 50A BL32
Battery	iFlight Fullsend 4S 3000mAh Li-Ion	208	-
On-board compute	NVIDIA Jetson Orin NX 16GB & DAMIAO v1.1 carrier board	62	9 (jtop)
Event camera	iniVation DVXplorer Micro	22	max 0.7
Stereo camera	Intel RealSense D435i	75	max 3.5
Total	-	822	213.2

Orin

General

TODO: add dts/dtb files

• Jetson Orin NX 16GB module with DAMIAO v1.1 carrier board
  • Careful installing the heat sink!
• Other hardware:
  • SSD: Corsair MP600 Core Mini
  • Wifi: Intel AX200 (add some antennas)
  • RTC battery: CR1220
  • Good XT30 power supply (used the one from A603 carrier board)
• Use SDK manager to install JetPack 6.0
  • If not seen, make sure that it is in recovery mode (lsusb should show NVIDIA Corp. APX)
  • Select everything for installation
  • Select Jetson Orin NX 16GB with developer kit (even though it isn't)
  • mavlab/mavlab for login
  • If it doesn't work: try different cable! I had to retry a few times
    • Too hot? USB timeout? Cable?
• Update everything in GUI and with apt update (but don't yet autoremove)
• hostnamectl set-hostname event-orin-drone
• Configure wifi using nmtui or connect over ethernet for now (make sure to set to 'share to local' on Ubuntu)
• Configure orin:
  • Run configure_orin.sh as cd orin && sudo ./configure_orin.sh
  • Fixes wifi
    • Better version of this?
  • Sets fan to max on boot and MAXN power mode (0, [best performance[(https://docs.nvidia.com/jetson/archives/r36.3/DeveloperGuide/SD/PlatformPowerAndPerformance/JetsonOrinNanoSeriesJetsonOrinNxSeriesAndJetsonAgxOrinSeries.html#supported-modes-and-power-efficiency)])
• Enable extra UART following these posts:
  • Decompile dtb to dts with sudo dtc -I dtb -O dts A.dtb > A.dts, modify, then recompile with sudo dtc -I dts -O dtb A.dts > A.dtb
  • Put in /boot/dtb, then specify the file in /boot/extlinux/extlinux.conf as in the posts, then reboot
• DONT install jtop (instructions), this messes with the max fan above
• Check RTC: here
• UARTs:
  • FC telem1 (connector on FC) -> /dev/ttyTHS1: UART closest to carrier board edge
    • MAVLink Router
  • FC telem2 (soldered on FC) -> /dev/ttyTHS3: UART closest to center of carrier board
    • uXRCE agent to ROS
  • For telem2, we'll use a baudrate of 3000000 (PX4 max, see here); for telem1, we'll use 921600 (else we got issues when controlling offboard)
    • No need to set it here, but be sure that the UART port/wires can support this
    • If too much, use something like 921600

jetson-containers

• Install jetson-containers
• Build base image with e.g. jetson-containers build --name event-orin cuda:12.2 pytorch:2.4 openai-triton:3.0.0
• Check versions of installed packages, so you can get pre-built binaries from here
• When running a container, it's nice to create a user with the same UID/GID inside, and use that, so not all folders will end up being owned by root: jetson-containers run -v .:/workspace --user $(id -u):$(id -g) <container_name>
• Example script for drone flight: see here

Sensors

General

• If you want to access sensors without root (whether inside or outside a container), we need to set some udev rules
• Use sensors/unsafe_usb_udev.sh (UNSAFE? but fine for Orin?) to open all USB ports for regular users
  • Test if this works
• On WSL, you need to share the USB device between Windows and WSL using usbipd-win
  • You can check if it's being detected in WSL with lsusb

MicoAir MTF-01

• Check that flow scale is set to 100 using MicoAssistant

Intel RealSense D435i

• Udev rules: here from here
• Build librealsense from source in Docker container following here
  • Before, we had realsense in jetson-containers, but that still needed two fixes to allow realsense to build (from here).
  • Furthermore, librealsense libs should be loaded before any ROS2 ones, so we prepend to the LD_LIBRARY_PATH in the entrypoint.sh script following this comment.
• Run from Docker container using realsense-ros
• Check if working with rs-enumerate-devices
• Could only get 15 Hz at 640x480 (even though 30 Hz should work?)

Configuration:

• Use depth quality tool to tune
  • Spatial, temporal and decimation filters with defaults
  • 640x40, 30 fps does work here, even with USB2.1
  • Both image and depth to auto-exposure
  • Align depth to color is nice
  • Further tips here

iniVation DVXplorer Micro

General

• Udev rules: here from here
• Run from Docker container using libcaer_driver ROS packages
• Check if working with ros2 launch libcaer_driver driver_node.launch.py
• Works well, but seems to saturate for full resolution?

Camera calibration

• Could be done with a combination of simple_image_recon and camera_calibration, but it's easier to use DV-GUI for this.
• Parameters that came out for our DVXplorer with 3.6mm lens can be found here

GS

Tested on Windows 11 with WSL (Ubuntu 22.04).

General

• QGroundControl (stable, v4.4.2)
  • Install in Windows, else you can't flash
• Configure link to WSL: see here

Containers

• Install Docker
• Install NVIDIA Container Toolkit
• Install jetson-containers (up to and including 'Docker default runtime')
  • On aarch64, jetson-containers automatically mounts USB devices, but this doesn't happen on x86_64, so add --device /dev/bus/usb.

Sim

If you want to run simulations with PX4 and Gazebo and ROS2:

• Install PX4 following this
• ROS2 Humble following this
• ROS2-Gazebo bridge for Gazebo Garden following this
• Micro XRCE-DDS Agent following this using snap with the --edge tag
  • Run with snap run micro-xrce-dds-agent udp4 -p 8888
• Connect radio as joystick, calibrate in Windows/QGroundControl
  • Joystick buttons: some issue here
• Simulation with depth camera messages sent to ROS2: sim/px4_gazebo_sim_example.sh
• If you need it to work with a container, set --ipc=host following this

Comms

MAVLink Router

• For telemetry to ground station and logging (because PX4 cannot log to flash, and Kakute H7 Mini doesn't have SD...)
• Install mavlink-router
  • Make sure to add user to dialout and tty as here
• Put config (from here) in /etc/mavlink-router/main.conf
• Run as mavlink-routerd
• Make sure to set up a TCP comms link in QGroundControl pointing to <orin-ip-address:5760
• Use sufficient baudrate (but not max PX4, this gives issues); we found 921600 to work well

uXRCE agent

• For communication between PX4 and ROS2
• Install following this using snap
• Run as micro-xrce-dds-agent serial --dev /dev/ttyTHS3 -b 3000000 (match highest baudrate on PX4)

PX4-ROS2

• PX4-ROS2 interface lib as here
• Works great
• Custom modes don't show up in QGroundControl v4.4.2 but you can still select them by overwriting PX4 default modes
  • In PX4 shell: commander status to see if registered, then commander takeoff and commander mode auto:mission to switch to the custom mode overwriting the PX4 'Mission' flight mode

Wifi

• Antenna sticking out of the drone for better range
• Use Tailscale in combination with a hotspot
  • Fixed some problems: this
  • eduroam seems too high latency?
• Key-only SSH, no root login
• Hotspot on laptop or phone (Windows: set to 2.4 GHz to maximize range)

TODO: allow WSL to connect to drone without Tailscale (mirrored networking mode?), so we can have the OptiTrack client running on the laptop (else we need Zenoh bridge)

Radio

• ELRS, Radiomaster Pocket with EdgeTX, Radiomaster RP2
• Setup:
  • Update EdgeTx with buddy, and doing reinstall in SD card content to get clean install
  • Go through all radio settings with these posts
  • Then continue here https://www.expresslrs.org/quick-start/transmitters/tx-prep/#radio-settings
  • Then update ELRS Tx firmware here
    • Better instructions here
  • Configurator here
  • Bind phrase: mavlab is cool
  • Receiver: follow this
  • Packet rate
• Don't use MAVLink telemetry, just use CRSF protocol (link mode normal), packet rate 250Hz I think, switch mode wide, telem ratio std
  • Also check receiver via 'other devices' (?)
• Disable auto wifi interval: see this (annoying)

ROS

• Contains ROS2 workspace, and script build_ros2.sh to build ROS2 packages
• See sim for PX4-ROS2 setup

Scripts

• drone_flight.sh: start all necessary windows/services for real flight
• move_logs_to_drive.sh: mount USB stick, move logs, unmount

Examples

Collecting data while flying

• Make sure containers, ROS, etc. are all built on the Orin
• Run drone_flight.sh on the Orin (connect over SSH) to start all necessary services
• Connect to your local QGroundControl by setting up a comms link in QGroundControl (under application settings -> comm links) pointing to <orin-ip-address>:5760 over TCP
• Make sure flight modes are set correctly in QGroundControl
• Then do ros2 launch eo_drone drone_flight_recorder.launch.py on the Orin to start recording events and flight controller stuff to a rosbag
• Fly around (in position or stabilized mode) and land
• Stop the rosbag recording, close all windows
• Put USB stick in Orin, check its UUID, then modify/run move_logs_to_drive.sh to move the logs to the USB stick
• Read using Foxglove or convert to H5

Testing external flight modes in simulation

• Make sure containers, ROS, etc. are all built locally
• Run sim/px4_gazebo_sim_external_modes.sh to start PX4 and Gazebo simulation
  • If you don't end up with 4 tmux windows, something went wrong, so check the commands in normal terminals to see the error messages
  • If there's something about timeout while waiting for FMU, increase the sleep in the script
  • The sim might be very slow; if so, try disabling the image bridge in the script (then the corresponding depth seeker won't work of course)
• You can connect a joystick with QGroundControl but it's not necessary, set up comms link to WSL as explained here
• In the PX4 terminal:
  • commander status to see if custom modes are registered
  • commander takeoff to take off
  • commander mode ext1 to start the depth seeker based on a depth camera, which should avoid the walls
  • commander mode auto:loiter to hover again, or commander mode posctl to control using the sticks
  • commander mode ext2 to start flying a square autonomously

Using a neural network for estimation/control

• Same as above, but now the external control node takes the output of a Python node running a PyTorch neural network
  • The network mimics a depth estimation network, taking an image and predicting a depth map
  • It is untrained, however, so it won't work very well (just to illustrate the concept)
• Run sim/px4_gazebo_sim_network_control.sh to start PX4 and Gazebo simulation
• In the PX4 terminal:
  • commander mode ext1 to start the depth seeker based on a depth estimation network, which shouldn't work well (because untrained)