Remote ID Development Coordination

[junwoo091400]

About

As hamish started effort on documenting Remote ID setup here: PX4/PX4-user_guide#2605, I have browsed through PX4 PRs / discord discussions / Ardupilot implementations to find that we have some holes to fill before making remote ID fully functional. This Issue aims to coordinate development efforts on that.

NOTE: This won't go into v1.14, but the release after, since we currently don't have agreement on what exactly needs to be implemented between manufacturers and users.

What is missing

Mandatory

• OPEN_DRONE_ID_SYSTEM values not getting cached within PX4, when received from GCS (operator location type, category eu, operator lat/lon, etc)
• GCS GPS showing red error in QGC. Potentially low rate of OPEN_DRONE_ID_LOCATION getting sent by PX4
  • Seems like maybe the updated condition isn't getting satisfied every 1 second (minimum required update rate for LOCATION message):

    PX4-Autopilot/src/modules/mavlink/streams/OPEN_DRONE_ID_LOCATION.hpp

    Lines 264 to 265 in be38633

    	if (updated) {
    	mavlink_msg_open_drone_id_location_send_struct(_mavlink->get_channel(), &msg);

• Heartbeat from Remote ID: If not received for certain time interval, declare malfunction in Remote ID system, and indicate in status field (as MAV_ODID_STATUS_REMOTE_ID_SYSTEM_FAILURE) of the LOCATION message.
• OPEN_DRONE_ID_ARM_STATUS based arming authorization. Example: https://github.com/ArduPilot/ardupilot/blob/5fa8b887a2df8a85822f84f41eec64bef5066895/libraries/AP_OpenDroneID/AP_OpenDroneID.cpp#L196C1-L198
• OPEN_DRONE_ID_ARM_STATUS message indicating failure in flight: LOCATION message's status field should be set to emergency or system failure. Relevant discussion: Add Open Drone ID arming check #21652 (comment)

Optional

• OPEN_DRONE_ID_BASIC_ID ID format is not in correct format: MAV_ODID_ID_TYPE_SERIAL_NUMBER needs to be ANSI/CTA-2063 format.
• OPEN_DRONE_ID_OPERATOR_ID is not getting cached within PX4, when received from GCS
• OPEN_DRONE_ID_SELF_ID values not getting cached within PX4, when received from GCS

Uncertain

• Streaming of OPEN_DRONE_ID_ARM_STATUS out of PX4 itself? -> https://github.com/PX4/PX4-Autopilot/pull/21647/files#r1246114592 | Relevant discussion: Remote ID in PX4 v1.14 PX4-user_guide#2627 (comment)
• Selective streaming of Open Drone ID MAVLink messages, as currently we are streaming it by default.

Related PRs

Pending

• Open Drone ID Live GNSS and Arm status prototype  #21647

Merged

• Add Open Drone ID arming check #21652

Resources

• User testing report on Open Drone ID Live GNSS and Arm status prototype  #21647: https://discord.com/channels/1022170275984457759/1038284900081614879/1115609787317628998
• Initial Open Drone ID support in PX4: mavlink: initial OPEN_DRONE_ID_BASIC_ID/OPEN_DRONE_ID_LOCATION support #20348
• QGC Remote ID support: Include support for Remote ID mavlink/qgroundcontrol#10600
• Initial Ardupilot Open Drone ID support PR (for reference): Wip/opendroneid ArduPilot/ardupilot#21075

[junwoo091400]

@dirksavage88 your PR #21647 doesn't seem to include the changes from #21652, could you rebase your PR to the main branch to take those changes in?

[dirksavage88]

@dirksavage88 your PR #21647 doesn't seem to include the changes from #21652, could you rebase your PR to the main branch to take those changes in?

It has been rebased to main. Also the operator location was set back to takeoff, but perhaps a regional parameter could be used to implement fixed or takeoff position. Also a parameter to allow live gnss for operators not in a fixed location (BVLOS?).

The details of these rulings are difficult to interpret. I think abstracting a lot of the legalese with just setting a regional parameter could work: it's simpler than giving the user the ability to do fixed vs takeoff vs live....given they don't have an indepth understanding of remote ID

[junwoo091400]

I also find some architectural parts of detection of open drone id system's presence & health a bit weird.

Currently:

• TelemetryStatus message has heartbeat_type_open_drone_id flag, indicating having valid heartbeat within 2.5 seconds
• VehicleStatus message has open_drone_id_system_present and open_drone_id_system_healthy flags
• Commander keeps track of the telemetry.heartbeat_type_open_drone_id, and has it's own timestamp (_datalink_last_heartbeat_open_drone_id_system) of when the flag was true
• If 3 seconds elapse from the last valid timestamp of heartbeat (_datalink_last_heartbeat_open_drone_id_system), Open Drone ID system is considered lost.

What I find odd about this is that:

• We have a decoupled notion of system present & system healthy. Is this decoupling really necessary?

It seems that the only case where system is present, but would be not healthy is when the system status of the heartbeat is not standby or active

PX4-Autopilot/src/modules/mavlink/mavlink_receiver.cpp

Lines 2126 to 2127 in a6d2c2c

	_mavlink->telemetry_status().open_drone_id_system_healthy =
	(hb.system_status == MAV_STATE_STANDBY) || (hb.system_status == MAV_STATE_ACTIVE);

PX4-Autopilot/src/modules/commander/Commander.cpp

Lines 2590 to 2605 in a6d2c2c

	if (telemetry.heartbeat_type_open_drone_id) {
	if (_open_drone_id_system_lost) {
	_open_drone_id_system_lost = false;
	if (_datalink_last_heartbeat_open_drone_id_system != 0) {
	mavlink_log_info(&_mavlink_log_pub, "OpenDroneID system regained\t");
	events::send(events::ID("commander_open_drone_id_regained"), events::Log::Info, "OpenDroneID system regained");
	}
	}
	bool healthy = telemetry.open_drone_id_system_healthy;
	_datalink_last_heartbeat_open_drone_id_system = telemetry.timestamp;
	_vehicle_status.open_drone_id_system_present = true;
	_vehicle_status.open_drone_id_system_healthy = healthy;
	}

---

But now that I wrote down these things & thought about it, it does seem like all the components have their purposes 🤦‍♂️

[junwoo091400]

It has been rebased to main. Also the operator location was set back to takeoff, but perhaps a regional parameter could be used to implement fixed or takeoff position. Also a parameter to allow live gnss for operators not in a fixed location (BVLOS?).

For reference, Ardupilot seems to directly send out the system message received from GCS, which includes operator location:

https://github.com/ArduPilot/ardupilot/blob/5fa8b887a2df8a85822f84f41eec64bef5066895/libraries/AP_OpenDroneID/AP_OpenDroneID.cpp#L529-L534

This is where the incoming message is captured:
https://github.com/ArduPilot/ardupilot/blob/5fa8b887a2df8a85822f84f41eec64bef5066895/libraries/AP_OpenDroneID/AP_OpenDroneID.cpp#L769

With this, I am wondering maybe we can have default beahvior be to report home position, but if we get a valid operator location through system message via GCS, then we send out that instead?

[junwoo091400]

@bkueng @katzfey As hamish suggested, It does seem like the open drone id heartbeat 3 seconds timeout information should be available to the LOCATION message stream at least by some means, to communicate to Remote ID hardware about that information.

And thus, https://mavlink.io/en/messages/common.html#MAV_SYS_STATUS_SENSOR_EXTENDED seems to be a proper place to publish this information (and make it available system wide). Thoughts on this?

[junwoo091400]

Regarding the OPEN_DRONE_ID_BASIC_ID, the MAV_ODID_ID_TYPE_SERIAL_NUMBER which needs to be ANSI/CTA-2063 format seems to be stored as a persistent parameter in Ardupilot implementation.

https://github.com/ArduPilot/ardupilot/blob/5fa8b887a2df8a85822f84f41eec64bef5066895/libraries/AP_OpenDroneID/AP_OpenDroneID.cpp#L118

And it seems like it's set via Misison Planner by the user manually: https://docs.cubepilot.org/user-guides/cube-id/cube-id

https://github.com/ArduPilot/MissionPlanner/blob/5448bfe8d8f46ff114bece7d9322906497b1a108/Plugins/OpenDroneID2/OpenDroneID_UI.Designer.cs#L103-L112

However, I wasn't able to find how a user obtains such ID, @hendjoshsr71 do you have a comment on this?

[BluemarkInnovations]

However, I wasn't able to find how a user obtains such ID, @hendjoshsr71 do you have a comment on this?

I can answer that. If a user buys our module we provide a valid Serial Number/ID. UAV manufacturers needs to us their own Serial Number range. The first part is the ICAO code that those manufacturers need to apply for.

edit: normal end-users will not get an ICAO code and therefore will not be able generate their own SN.

[BluemarkInnovations]

• OPEN_DRONE_ID_ARM_STATUS message indicating failure in flight: LOCATION message's status field should be set to emergency or system failure. Relevant discussion: Add Open Drone ID arming check #21652 (comment)

Easiest for now would be to go for the manual assertion of an emergency by the pilot. In QGroundControl the user can set an emergency. Manual assertion of an emergency is enough to be compliant.

[BluemarkInnovations]

It has been rebased to main. Also the operator location was set back to takeoff, but perhaps a regional parameter could be used to implement fixed or takeoff position. Also a parameter to allow live gnss for operators not in a fixed location (BVLOS?).

In the USA, take off position is not allowed for standard Remote ID. Only dynamic or fixed. In my opinion the GCS software should do this such as the QGroundControl implementation. Not sure if PX4 needs to do extra checks. Also I asked the FAA about standard Remote ID vs Remote ID broadcast module. Standard Remote ID is only allowed for new drones after September.

[BluemarkInnovations]

With this, I am wondering maybe we can have default beahvior be to report home position, but if we get a valid operator location through system message via GCS, then we send out that instead?

It is not defined by the FAA, so it is a nice to have, but PX4 should not allow to arm if it hasn't the operator location.

Also there is the OPEN_DRONE_ID_SYSTEM_UPDATE. Not sure if GCS software is streaming this light version of they system message (that save bandwidth), but it means that PX4 should be able to receive and parse this message type also.

[DronecodeBot]

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/px4-maintainers-call-july-04-2023/32943/1

[dirksavage88]

However, I wasn't able to find how a user obtains such ID, @hendjoshsr71 do you have a comment on this?

I can answer that. If a user buys our module we provide a valid Serial Number/ID. UAV manufacturers needs to us their own Serial Number range. The first part is the ICAO code that those manufacturers need to apply for.

edit: normal end-users will not get an ICAO code and therefore will not be able generate their own SN.

So what is the rationale for forcing everyone under ANSI/CTA-2063-A rules then? Why is QGC not allowing take-off position under FAA rules?? Perhaps decoupling the software from the end-product is the true solution here. Vendors are free to fork this autopilot stack and make it RID compliant for standard remote ID rules (Auterion has their own implementation IIRC). I simply don't understand the reason behind locking things down except for legality purposes. Broadcast module support is probably what I would implement primarily...then leave the standard remote ID compliance to the people selling fully built drones.

[DronecodeBot]

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/remote-id-coordination-july-11th/33191/1

[mrpollo]

Hey @BluemarkInnovations, I seem to remember a comment that I think was from you regarding the OPEN_DRONE_ID_BASIC_ID ID format not being in the correct format (ANSI/CTA-2063) as far as I can see QGC should be handling this validation, is this not the case?

[mrpollo]

For OPEN_DRONE_ID_ARM_STATUS we need to include a check on the openDroneIDCheck.cpp#checkAndReport function to make sure the arm status is good else prevent arming

[mrpollo]

We need to understand what the FAA mandates for the "Operator dynamic position" can we set this to fixed or takeoff? Or do we need a GPS on the ground station?

💥To unblock development we are going to assume FIXED position is OK for both FAA and EU regulation💥

[mrpollo]

for [OPEN_DRONE_ID_LOCATION](https://mavlink.io/en/messages/common.html#OPEN_DRONE_ID_LOCATION) we have this right now and it should be enough we need to double check

		configure_stream_local("OPEN_DRONE_ID_LOCATION", 1.f);

[BluemarkInnovations]

So what is the rationale for forcing everyone under ANSI/CTA-2063-A rules then?

The ANSI/CTA-2063-A rule is similar to IP addresses. For public IP addresses you need to register those and are not allowed to pick one your self. The ANSI/CTA-2063-A basically says that the first part of the SN number is a manufacturer code that is given to a company by the ICAO organization. The remainder of the SN code is basically defined by the manufacturer itself (except for the length indicator).

Why is QGC not allowing take-off position under FAA rules??

Well because QGC implements the FAA rules. Of course as a hack, you can set the region to EU in QGC to use a fixed position.

Broadcast module support is probably what I would implement primarily...then leave the standard remote ID compliance to the people selling fully built drones.

But broadcast module support is a stand-alone product that does not need integration with PX4. (Of course it is nice to know the Remote ID status in QGC.) Only standard Remote-ID products need that.

[BluemarkInnovations]

Hey @BluemarkInnovations, I seem to remember a comment that I think was from you regarding the OPEN_DRONE_ID_BASIC_ID ID format not being in the correct format (ANSI/CTA-2063) as far as I can see QGC should be handling this validation, is this not the case?

At the moment, I'm unsure. I don't think that QGC is checking the SN number yet. I did make a PR for ArduRemoteID devices ArduPilot/ArduRemoteID#101 Having said this, QGC is broadcasting the OPEN_DRONE_ID_BASIC_ID to PX4 that relays it to the Remote ID hardware. Given the OPEN_DRONE_ID_ARM_STATUS message, it seems to me logical that the Remote ID hardware is checking the SN number. Same for the GCS (not allowing the user to enter non-valid SN numbers). I don't see any reason for checking the SN number by PX4. How is PX4 going to report a misconfigured SN number to GCS or Remote ID hardware?

[BluemarkInnovations]

We need to understand what the FAA mandates for the "Operator dynamic position" can we set this to fixed or takeoff? Or do we need a GPS on the ground station?

boomTo unblock development we are going to assume FIXED position is OK for both FAA and EU regulationboom

See this table on the OpenDroneID website: Comparison

The F3586 rule mandates the following on Operator position (section 5.1.3.6)

"For Standard Remote ID, the Operator Location/Altitude Type in the System Message shall be either “1. Dynamic” and use a system that accurately reports the location of the ground control station (GCS); or “2. Fixed” where the location is automatically programmed into the UAS. The GCS location must correspond with the location of the operator."

So yes dynamic and fixed is allowed. For using fixed location there are additional requirements (see above). To me it seems that in most situations dynamic location is the best choice.

[BluemarkInnovations]

So the FAA regulation states that standard Remote ID is mandatory except for (https://www.ecfr.gov/current/title-14/chapter-I/subchapter-F/part-89/subpart-F):

"Except for unmanned aircraft designed and produced to be standard remote identification unmanned aircraft, this subpart does not apply to the design or production of:

(1) Home-built unmanned aircraft.

(2) Unmanned aircraft of the United States Government.

(3) Unmanned aircraft that weigh 0.55 pounds or less on takeoff, including everything that is on board or otherwise attached to the aircraft.

(4) Unmanned aircraft designed or produced exclusively for the purpose of aeronautical research or to show compliance with regulations."

I do assume that if your home-built unmanned aircraft flies outside a FAA-Recognized Identification Areas, you still need a Remote ID broadcast module. (Otherwise you can't register your drone at the FAA website.)

[junwoo091400]

From this discussion: it seems like we are streaming Open Drone ID MAVLink messages regardless of whether user is actively using Remote ID or not (so it's working in the background).

Question: Shall we stream Open Drone ID messages selectively, depending on e.g. a central 'Remote ID enable' parameter, or whether 'COM_ARM_ODID' (currently used for enabling arming prevention via Remote ID) is set by the user (which would implicitly mean that user is using Remote ID 🤔)?

@dagar @dirksavage88 @Davidsastresas

[Davidsastresas]

@junwoo091400 I am not familiarized with PX4, but in QGC it works based on a setting. We have a setting in application settings->general settings->misc settings, it is a tickbox and if not enabled, RID won't be active at all in QGC.

About PX4 I am not familiar enough to contribute to the discussion.

[DronecodeBot]

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/what-do-we-need-to-be-ready-for-droneid-mandate/33557/2