(ESP32-S3 / BLE 5.0 Feature) Sending and receiving chained extended advertisements using Bluedroid stack. (IDFGH-11053)

[B-Perederei]

Hi everyone who will read this post,
Currently, I am trying to use extended advertisements for sending and receiving 1650 bytes of data from one ESP32-S3 to another.

For the sender, I am using code from this example:
https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/bluedroid/ble_50/peroidic_adv/main/periodic_adv_demo.c
commenting out periodic advertisement logic and generating a 1650-byte array for an adv packet.
For the receiver, I am using code from this example:
https://github.com/espressif/esp-idf/blob/master/examples/bluetooth/bluedroid/ble_50/peroidic_sync/main/periodic_sync_demo.c
In this code, I added ESP_GAP_BLE_EXT_ADV_REPORT_EVT case in gap_event_handler for processing extended advertisement packets and printing caught data into logs.

As a result, I was able to receive chained extended advertisements in the following way:

I (13526) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 229 ----
I (13526) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (13526) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (13536) EXTENDED_ADVERTISEMENT: Advertising Data: 0102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F808182838485868788898A8B8C8D8E8F909192939495969798999A9B9C9D9E9FA0A1A2A3A4A5A6A7A8A9AAABACADAEAFB0B1B2B3B4B5B6B7B8B9BABBBCBDBEBFC0C1C2C3C4C5C6C7C8C9CACBCCCDCECFD0D1D2D3D4D5D6D7D8D9DADBDCDDDEDFE0E1E2E3E4E5
I (13586) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 13 ----
I (13586) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (13596) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (13606) EXTENDED_ADVERTISEMENT: Advertising Data: E6E7E8E9EAEBECEDEEEFF0F1F2
I (13616) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 229 ----
I (13616) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (13626) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (13636) EXTENDED_ADVERTISEMENT: Advertising Data: F3F4F5F6F7F8F9FAFBFCFDFEFF0102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F808182838485868788898A8B8C8D8E8F909192939495969798999A9B9C9D9E9FA0A1A2A3A4A5A6A7A8A9AAABACADAEAFB0B1B2B3B4B5B6B7B8B9BABBBCBDBEBFC0C1C2C3C4C5C6C7C8C9CACBCCCDCECFD0D1D2D3D4D5D6D7D8
I (13676) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 19 ----
I (13686) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (13686) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (13696) EXTENDED_ADVERTISEMENT: Advertising Data: D9DADBDCDDDEDFE0E1E2E3E4E5E6E7E8E9EAEB
I (13706) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 229 ----
I (13716) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (13716) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (13726) EXTENDED_ADVERTISEMENT: Advertising Data: ECEDEEEFF0F1F2F3F4F5F6F7F8F9FAFBFCFDFEFF0102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F808182838485868788898A8B8C8D8E8F909192939495969798999A9B9C9D9E9FA0A1A2A3A4A5A6A7A8A9AAABACADAEAFB0B1B2B3B4B5B6B7B8B9BABBBCBDBEBFC0C1C2C3C4C5C6C7C8C9CACBCCCDCECFD0D1
I (13776) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 19 ----
I (13776) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (13786) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (13796) EXTENDED_ADVERTISEMENT: Advertising Data: D2D3D4D5D6D7D8D9DADBDCDDDEDFE0E1E2E3E4
I (13806) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 229 ----
I (13806) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (13816) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (13826) EXTENDED_ADVERTISEMENT: Advertising Data: E5E6E7E8E9EAEBECEDEEEFF0F1F2F3F4F5F6F7F8F9FAFBFCFDFEFF0102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F808182838485868788898A8B8C8D8E8F909192939495969798999A9B9C9D9E9FA0A1A2A3A4A5A6A7A8A9AAABACADAEAFB0B1B2B3B4B5B6B7B8B9BABBBCBDBEBFC0C1C2C3C4C5C6C7C8C9CA
I (13866) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 19 ----
I (13876) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (13886) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (13886) EXTENDED_ADVERTISEMENT: Advertising Data: CBCCCDCECFD0D1D2D3D4D5D6D7D8D9DADBDCDD
I (13896) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 229 ----
I (13906) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (13906) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (13916) EXTENDED_ADVERTISEMENT: Advertising Data: DEDFE0E1E2E3E4E5E6E7E8E9EAEBECEDEEEFF0F1F2F3F4F5F6F7F8F9FAFBFCFDFEFF0102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F808182838485868788898A8B8C8D8E8F909192939495969798999A9B9C9D9E9FA0A1A2A3A4A5A6A7A8A9AAABACADAEAFB0B1B2B3B4B5B6B7B8B9BABBBCBDBEBFC0C1C2C3
I (13966) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 19 ----
I (13966) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (13976) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (13986) EXTENDED_ADVERTISEMENT: Advertising Data: C4C5C6C7C8C9CACBCCCDCECFD0D1D2D3D4D5D6
I (13996) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 229 ----
I (13996) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (14006) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (14016) EXTENDED_ADVERTISEMENT: Advertising Data: D7D8D9DADBDCDDDEDFE0E1E2E3E4E5E6E7E8E9EAEBECEDEEEFF0F1F2F3F4F5F6F7F8F9FAFBFCFDFEFF0102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F808182838485868788898A8B8C8D8E8F909192939495969798999A9B9C9D9E9FA0A1A2A3A4A5A6A7A8A9AAABACADAEAFB0B1B2B3B4B5B6B7B8B9BABBBC
I (14056) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 19 ----
I (14066) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (14076) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 1
I (14076) EXTENDED_ADVERTISEMENT: Advertising Data: BDBEBFC0C1C2C3C4C5C6C7C8C9CACBCCCDCECF
I (14086) EXTENDED_ADVERTISEMENT: ---- Packet caught, size: 168 ----
I (14096) EXTENDED_ADVERTISEMENT: esp_ble_gap_adv_type_t: 0
I (14096) EXTENDED_ADVERTISEMENT: esp_ble_gap_ext_adv_data_status_t: 0
I (14106) EXTENDED_ADVERTISEMENT: Advertising Data: D0D1D2D3D4D5D6D7D8D9DADBDCDDDEDFE0E1E2E3E4E5E6E7E8E9EAEBECEDEEEFF0F1F2F3F4F5F6F7F8F9FAFBFCFDFEFF0102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778

I spent some time searching the web, and reading BLE 5.0 core documentation and documentation on ESP32 Bluedroid, but haven’t found anything about how I need to format and structure my 1650 packet. From reading the source code in this file:
https://github.com/espressif/esp-idf/blob/master/components/bt/host/bluedroid/api/include/api/esp_gap_ble_api.h
and BLE 5.0 chained extended advertisements documentation I concluded, that the logic of receiving several packets and composing them together into a final one after receiving all chained packages is supported neither on the controller nor on the host level.

Still, I don’t understand why I have such strange packet sizes. Firstly – why maximum size of one extended advertisement packet I managed to reach is 229, but not 251 as it is allocated in esp_ble_gap_ext_adv_reprot_t.adv_data (esp_gap_ble_api.h). Secondly – why do I have small packets with not consistent size at the start and bigger ones mixed with them? For example, the first packet is 229, then 13, then 229 again, then 19, and so on.

My current guesses are that I didn’t configure properly esp_ble_gap_ext_adv_params_t in the sender code or/and extended advertisement packets need to be properly formatted. Also, this behavior may be related to specifics of Bluedroid implementation in the ESP32 library.

Could you provide me with any information that can clarify, why I am getting such a behavior and how I can send and receive chained extended advertisement packets properly using Bluedroid stack?

[esp-zhp]

@B-Perederei
you can refer to：
7.7.65.13 LE Extended Advertising Report event in
BLUETOOTH CORE SPECIFICATION Version 5.4 | Vol 4, Part E page 2264

[B-Perederei]

@zhp0406
Thank you for pointing out that in the documentation.
From the information provided in 7.7.65.13 LE Extended Advertising Report event about Event_Type in the Description section (p. 2259) and unsuccessful attempt finding any other Extended Advertising events I made a conclusion, that composing back chained extended advertising can be achieved only by receiving several Extended Advertising Reports separately and filtering them using sender address and Event_Type.

Also, could you provide a documentation reference that explains smaller packets creation sized from 13 to 19 bytes (Maybe this behavior is explained in the documentation Controller section)

[esp-zhp]

@B-Perederei

the two images below mey help you solve the issue：

[esp-zhp]

@B-Perederei
2.1 Packet format for the LE Uncoded PHYs in
BLUETOOTH CORE SPECIFICATION Version 5.4 | Vol 6, Part B page 2704

[B-Perederei]

@zhp0406
Thank you again for pointing this out. As I understood, the reason for this is that we can't put 242 bytes because of HCI specification, and the other 242 - 229 = 13 bytes are put into the next packet separately.

Last two questions:

1. Can I somehow configure the controller, HCI, or Host to change that logic and receive data on the Host level packed into 229 packets only?
2. How can I assemble received chained packets back? Currently, I can only catch extended advertisement report events separately, filter packets using the sender address and Event_Type complete flag, and compose them back by myself. Is there any API, event, or configuration I missed out on that I can use to assemble chained extended advertisement back automatically on the Host level?

[esp-zhp]

@B-Perederei
May I ask if directly report the complete data (length 1650) to the application layer would meet your requirements?

[B-Perederei]

@zhp0406
Yes, It will meet my requirements. I am generally interested in sending and receiving chained extended advertisements sized more than one 229 byte packet and I try to find the best way to do it.

[esp-zhp]

@B-Perederei

This is our latest feature, which has not been updated to GitHub yet.
note: complete data (max1650) is reported in the ESP_GAP_BLE_LONG_EXT_ADV_REPORT_EVT event.

You can apply this feature using the following patch:

0001-feat-bt-bluedroid-Support-adv-report-event-construct.patch

[B-Perederei]

@zhp0406
Thank you very much for providing this patch, it works perfectly. I hope that you will add this feature to your next releases.

[B-Perederei]

@zhp0406
Hi again. I have one more question regarding correct way of sending chained extended advertisement packets. Currently I just use code from this example , putting 1650 bytes into a packet. As mentioned before in this conversation, actual maximum size of 1 packet we can send is 229 bytes, and, as a result of this limitation, I have such strange logic of packets decompostion into sizes 229 13 229 19 229 19 ...
Is there any way (maybe changing configurations of Bluedroid Host or Host Controller Interface) to fix this issue and only send full 229 bytes packets without smaller packets between them?

[esp-zhp]

@B-Perederei
not support yet,What impact does our current approach have on you?

[B-Perederei]

@zhp0406
There is only concern on higher power consumption on sender/receiver side utilizing small separate packets, also possible problems with frequency of receiving full not truncated 1650 bytes packets, if, for instance, the distance between devices are high.
However, at this point it is just my assumptions and your implementation works fine. It will be great, if you make your implementation not to divide packet in such a way in future, but currently it satisfies my needs.
Thank you for your answer.