Problem when using esp_lcd_panel_init (for ST7701s) with Wi-Fi provisioning using Homekit? (IDFGH-12112)

[puddletowntom]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

When i use an LCD display (ST7701s https://github.com/wireless-tag-com/ZX2D10GE01R-V4848) with the Homekit (esp-homekit-sdk), the pairing stops working when using a simple example such as the lightbulb example. Running the Homekit example by itself works fine for Wi-Fi provisioning and pairing. In addition, the display works fine by itself also. But when i try to use the 2 modules at the same time it prevents connection and pairing. It seems like i have narrowed the problem down to esp_lcd_panel_init() because if i remove this, the problem goes away but obviously that is required for using the display. I have tried playing around with the Wi-Fi configurations and the LCD configurations but to no avail. I am starting to assume there is some kind of memory issue where the two modules are interfering with one another with shared resources. I have noticed that there is support in the IDF for the display st7789 (esp_lcd_panel_st7789.c ). I am wondering is the initialisation from the code attached enough to work with Wi-Fi provisioning as the ST7789 seems more developed. Are any of the developers for the IDF aware of issues around LCD panel initialisation conflicting with Wi-Fi provisioning? The LCD is using double frame buffers, it uses octal 8MB of PSRAM. It would be nice if there was added support in the IDF for the ST7701s.

void qmsd_rgb_init(esp_lcd_rgb_panel_config_t *panel_config)
{
    static lv_disp_drv_t disp_drv;
    int buffer_size;
    void *buf1 = NULL;
    void *buf2 = NULL;
	static lv_disp_draw_buf_t draw_buf;

    lv_init();

    ESP_ERROR_CHECK(esp_lcd_new_rgb_panel(panel_config, &g_panel_handle));
    ESP_ERROR_CHECK(esp_lcd_panel_reset(g_panel_handle));
    //ESP_ERROR_CHECK(esp_lcd_panel_init(g_panel_handle));

    // buffer_size = panel_config->timings.h_res * panel_config->timings.v_res;
    // esp_lcd_rgb_panel_get_frame_buffer(g_panel_handle, 2, &buf1, &buf2);
    // lv_disp_draw_buf_init(&draw_buf, buf1, buf2, buffer_size);

    // lv_disp_drv_init(&disp_drv);         
    // disp_drv.flush_cb = __qsmd_rgb_disp_flush;
    // disp_drv.draw_buf = &draw_buf;
    // disp_drv.hor_res = panel_config->timings.h_res;
    // disp_drv.ver_res = panel_config->timings.v_res;
    // lv_disp_drv_register(&disp_drv);
}

The panel configuration from the link looks like this,

esp_lcd_rgb_panel_config_t panel_config = {
        .data_width = 16,
        .psram_trans_align = 64,
        .pclk_gpio_num = LCD_PCLK_GPIO,
        .vsync_gpio_num = LCD_VSYNC_GPIO,
        .hsync_gpio_num = LCD_HSYNC_GPIO,
        .de_gpio_num = LCD_DE_GPIO,
        .disp_gpio_num = LCD_DISP_EN_GPIO,
        .data_gpio_nums = {
            LCD_DATA0_GPIO,
            LCD_DATA1_GPIO,
            LCD_DATA2_GPIO,
            LCD_DATA3_GPIO,
            LCD_DATA4_GPIO,
            LCD_DATA5_GPIO,
            LCD_DATA6_GPIO,
            LCD_DATA7_GPIO,
            LCD_DATA8_GPIO,
            LCD_DATA9_GPIO,
            LCD_DATA10_GPIO,
            LCD_DATA11_GPIO,
            LCD_DATA12_GPIO,
            LCD_DATA13_GPIO,
            LCD_DATA14_GPIO,
            LCD_DATA15_GPIO,
        },
        .timings = {
            .pclk_hz = 15000000,
            .h_res = 480,
            .v_res = 480,
            .hsync_pulse_width = 10,
            .hsync_back_porch = 10,
            .hsync_front_porch = 10, 
            .vsync_pulse_width = 2,
            .vsync_back_porch = 12,
            .vsync_front_porch = 14,
        },
        .flags.fb_in_psram = 1,
        .flags.double_fb = 1,
        .flags.refresh_on_demand = 0,   // Mannually control refresh operation
        .bounce_buffer_size_px = 0,
        .clk_src = LCD_CLK_SRC_PLL160M,
    };

    qmsd_rgb_init(&panel_config);

Using,

ESP-IDF: v5.1.2
LVGL: 8.3.3

[suda-morris]

If PSRAM is used and WiFi memory is prefered to allocate in PSRAM first
--- esp_wifi/README.md

Yes, both the wifi and RGB LCD drivers are accessing the PSRAM.

What's the behavior of the LCD when it's doing the wifi connection? Does CONFIG_SPIRAM_FETCH_INSTRUCTIONS=y
CONFIG_SPIRAM_RODATA=y help?

BTW, my colleague told me we have a ST7701 driver on the component registry: https://components.espressif.com/components/espressif/esp_lcd_st7701

[Lzw655]

Hi @puddletowntom, according to your test, there might be an issue with memory shortage. Please refer to the following code to periodically print the current remaining space by using a task:

#include <stdio.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_heap_caps.h"

    static char buffer[128];    /* Make sure buffer is enough for `sprintf` */
    while (1) {
        sprintf(buffer, "   Biggest /     Free /    Total\n"
                "\t  SRAM : [%8d / %8d / %8d]\n"
                "\t PSRAM : [%8d / %8d / %8d]",
                heap_caps_get_largest_free_block(MALLOC_CAP_INTERNAL),
                heap_caps_get_free_size(MALLOC_CAP_INTERNAL),
                heap_caps_get_total_size(MALLOC_CAP_INTERNAL),
                heap_caps_get_largest_free_block(MALLOC_CAP_SPIRAM),
                heap_caps_get_free_size(MALLOC_CAP_SPIRAM),
                heap_caps_get_total_size(MALLOC_CAP_SPIRAM));
        ESP_LOGI("MEM", "%s", buffer);

        vTaskDelay(pdMS_TO_TICKS(2000));
    }

[puddletowntom]

@suda-morris I didnt realise that there was a more developed ST7701 driver from the components registry as you posted so thank you for pointing that out to me. I went ahead and tried it out instead of the display code I had been using because i wanted to see if there was any improvement in performance. I had to adjust the commands to match my display but got the display going with this esp_lcd_st7701 driver. Unfortunately this didnt change the behaviour of of the Wi-Fi application. CONFIG_SPIRAM_FETCH_INSTRUCTIONS and CONFIG_SPIRAM_RODATA are both enabled but this doesnt change the behaviour of Wi-Fi either.

@Lzw655 I am not sure if there is a memory shortage. Here is the result of the print out. For this i am only initialising the display while running the Wi-Fi application. Just to reiterate, Wi-Fi connects and pairs when removing the display initialisation.

I (10994) app_wifi: Disconnected. Connecting to the AP again...
I (12044) MEM:    Biggest /     Free /    Total
          SRAM : [   31744 /    80355 /   342583]
         PSRAM : [ 6422528 /  6549096 /  7012352]
I (13404) app_wifi: Disconnected. Connecting to the AP again...
I (14044) MEM:    Biggest /     Free /    Total
          SRAM : [   31744 /    80363 /   342583]
         PSRAM : [ 6422528 /  6549096 /  7012352]
I (15814) app_wifi: Disconnected. Connecting to the AP again...
I (16044) MEM:    Biggest /     Free /    Total
          SRAM : [   31744 /    80191 /   342583]
         PSRAM : [ 6422528 /  6549096 /  7012352]
I (18044) MEM:    Biggest /     Free /    Total
          SRAM : [   31744 /    80371 /   342583]
         PSRAM : [ 6422528 /  6549096 /  7012352]

Thank you for the response.

[puddletowntom]

So I think i have found a work around. I havent fully implemented it yet but i noticed from the esp-iot-solutions repo that there is a wifi lvgl example found here, https://github.com/espressif/esp-iot-solution/tree/master/examples/hmi/lvgl_wificonfig

In this example, they seem to protect wifi commands from lvgl with the following,

lvgl_acquire();
start_wifi_scan();
lvgl_release();

lvgl_acquire();
wifi_connect_success();
lvgl_release();

I noticed that i was able to connect to Wi-Fi when I disengage the panel. eg using the following,

void disconnect_panel(void){
    esp_lcd_panel_io_del(io_handle);
    esp_lcd_panel_del(panel_handle);
}

However, I cannot really do this in the middle of a task because i would obviously have to reinitialise the panel for the display. Its a little messy but if i can use the acquire/release system from the example this might work.

[mrdean05]

@puddletowntom, Have you found a solution to the problem ? I am currently facing the same issue. Connection to wifi crashes when using st7701s but without st7701 initialization, wifi connection is successful.

[puddletowntom]

I gave up on it a while back. A conclusion that I came to was that the design of the ESP32-S3 was more intended for single purpose IoT applications. The marketing team might have advertised HMI applications but didn't put a lot of emphasis on HMI + WiFi applications. It is just a microcontroller after all. So it would make more sense to use 2 microcontrollers instead, an ESP32-S3 dedicated for display and an ESP32-C6 dedicated for Wi-Fi. The advantage of this would be for several reasons.

• Not having a single point of failure as you do with a SoC (WiFi problems interfering with display),
• Easier to maintain code (less use of RTOS means simpler code because WiFi has its own processor),
• More reliable because you no longer have tasks interfering with one another (separate processors for separate tasks)

When you look at the Espressif website they have a new ESP32-P4 which is aimed at being used with displays. Interestingly they removed the RF component so there isnt Wi-Fi on the chip and they recommend using a companion chip such as the ESP32-C6 in conjunction with the ESP32-P4. This aligns with my assumptions that it is probably bad design to try use WiFi and display on the same chip.

[mrdean05]

Thanks @puddletowntom for the insight. I'm likely going with your recommendation (ESP32-S3 + ESP32-C6 combo) as that will be the best solution to the issue.
While checking the Espressif website, I noticed that the ESP32-P4 module/chip hasn't been commercially launched yet, although the dev board is available. Do you have any idea when the module/chip will be publicly available ?

[puddletowntom]

I don't know when it is available, I think I remember hearing it was available this quarter a few months back. The ESP32-P4 can be used with MIPI displays so it might be overkill. But the schematic of the evaluation board shows ESP32-P4 connected to the ESP32-C6 so that might be useful. The ESP32-S3 probably has more support for development since its around longer.

[puddletowntom]

I was thinking about this problem recently and wanted to add some final thoughts. When you consider the internal architecture of the ESP32, the PSRAM is interfaced using SPI/QSPI. This is a single access communication. Where as regular SRAM is directly part of the CPU. This means that SRAM allows for concurrent access while PSRAM does not allow for concurrent access. So when you consider that both the display driver (ST7701) and WiFi (library) are both using PSRAM at the same time, this probably isnt possible given that PSRAM is single access only. This would also explain why the problem had nothing to do with a shortage of memory when looking at the heap earlier. So even though ESP32-S3 is dual core and even though you apply an RTOS solution with multi-tasking, this still doesnt overcome the problem of PSRAM being single access only. If you were to use a display without PSRAM, you could probably use WiFi at the same time. I think i remember testing the Arduino GFX library with ST7701 with WiFi and that worked but with some screen tearing on the display. However, I dont know if the Arduino WiFi library or LCD display library were excluding the use of PSRAM. But I do know that in the IDF framework both WiFi library and the LCD library were using PSRAM at the same time. So you have to be careful with the type of multi tasking you are doing with ESP32. This is probably why the ESP32-P4 excluded RF when the focus was put on its use for displays.

[mrdean05]

Thanks for the insight. I tried the code you sent me, and it worked. Is it possible to configure the ST7701 to use PSRAM and the WiFi library to use SRAM, instead of both using PSRAM?