voice recognition controller issue

[bitdeckai]

Hi，everyone:

AI and robots are very popular recently. I want to make a voice recognition deck board based on chatgpt or deepseek. It can control the takeoff, landing, forward, backward, left and right movement of CF2.1, similar to the control buttons in Flight Control Tab after installing flow deck, but replaced with voice recognition. At present, I have decided to use ESP32S3 to do this. I want to ask how to send control commands to the aircraft after my MCU recognizes the commands. I hope to use UART interface. How should the specific control commands be packaged and sent?

Another uncertain issue is whether noise during flight will affect speech recognition. We will make some attempts later. If you have any experience, please share it with us.

[ToveRumar]

Hi!
This sounds like a really cool project!

I suggest you look into how to use CPX and CPX over UART.
This documentation overviews the CPX protocol;
https://www.bitcraze.io/documentation/repository/crazyflie-firmware/master/functional-areas/cpx/

We also have an app that uses UART to communicate, so it could be used to figure out how to use UART for this.
https://github.com/knmcguire/crazyflie-firmware/tree/bolt-pos-through-uart/examples/app_uart

I hope this documentation helps you get started with your project!

[bitdeckai]

Hi，

I know the example, but I'm not sure how to implement the instructions for take off, land, shift left and shift right

[ToveRumar]

Hi!

I would use the high level commander in the lib to send CRTP packets over CPX.
If you set the function field of the packet to 3 (CRTP) this will be routed correctly as a crtp packet to the crazyflie.

If you want up/down/left/right you could create crtp packet for GO_TO() in the "high level commander" and specify the step length and direction by setting a point x length from your current point.
This documentation is good to read on CRTP; https://www.bitcraze.io/documentation/repository/crazyflie-firmware/master/functional-areas/crtp/.
https://github.com/bitcraze/crazyflie-firmware/tree/master/docs/functional-areas/crtp

You could look at how the python lib sends these messages using the high_level_commander.py file. So you could check how it is packaged there if you dont want to use the lib directly.

I hope this helps you forward!

[bitdeckai]

Hi，

Can you provide an example, for example, if I need to send an up cmd, what hexadecimal data is needed, because I don't really understand the protocol，Thanks！

[ToveRumar]

Hi!
I will try and point you to how you would do this in the firmware.

From what I understand is you create a struct like this for the position;

 struct positionPacket_s {
   float x;     // Position in m. (current pos + step length)
   float y;
   float z;
   float yaw;   // Orientation in degrees
 } __attribute__((packed));

Then pack it in a CRTPpacket ;

typedef struct _CRTPPacket
{
  uint8_t size;                         //< Size of data
  union {
    struct {
      union {
        uint8_t header;                 //< Header selecting channel and port
        struct {
          uint8_t channel ;      //< Channel should be SET_SETPOINT_CHANNEL
          uint8_t reserved;
          uint8_t port ;      //< Port should be CRTP_PORT_SETPOINT_GENERIC
        };
      };
      uint8_t data[CRTP_MAX_DATA_SIZE]; //< the positionPacket_s packet goes here
    };
    uint8_t raw[CRTP_MAX_DATA_SIZE+1];  //< The full packet "raw"
  };
} __attribute__((packed)) CRTPPacket;

Please look at the positionDecoder() to see how the position packet would be decoded.

Then pack this struct into a CPX packet CPXPacket_t;

typedef struct {
  CPXRouting_t route; //Routing packet, see below
  uint16_t dataLength; //Size of your crtp packet
  uint8_t data[CPX_MAX_PAYLOAD_SIZE]; (crtp packet)
} CPXPacket_t;

together with a CPXRouting_t packet that should be something along the lines of

typedef struct {
  CPXTarget_t destination; //STM32
  CPXTarget_t source; //your board
  bool lastPacket; //is this the last or not
  CPXFunction_t function; //CPX_F_CRTP
  uint8_t version; //CPX_VERSION
} CPXRouting_t;

I understand this explanation might feel a bit convoluted, however I think you should have all the pointers you need to get started.