The refresh rate of the RGB matrix is a bit slow

[gustavoalara]

I'm sorry to be a bother, but I'm conducting tests with the library and have encountered some additional issues. When I send the framebuffer with an image or a series of frames from a video converted to the format understood by ZeDMD through the RenderRGB24 command, I'm noticing that the rendering on the LED matrix is a bit slow. It performs a top-to-bottom sweep that, in animations, makes them not completely smooth (I'm also using m_streaming set to on and not performing a ClearScreen between frames).

I'm not sure if I'm not using something correctly or if it's still something you're working on.

Thanks in advanced

[mkalkbrenner]

RGB24 is the worst mode. The limit the bad USB to serial converter that is used on ESP32.
We already consider to move to ESP32 S3 to increase the data transmission speed.

Mybe you should consider to share your work in progress, so I can take a look at it.

Are you using ZeDMD or ZeDMD HD? Are you using 3.4.1?

[gustavoalara]

Thanks mkalkbrenner

I'm using a ZeDMD with the last 3.4.1 firmware.

Let me upload my code to my GitHub repo and then you can test it (I'm not a developer so probably is full of mistakes)

[gustavoalara]

A question, I understand that data transfer may be slower when using RGB24 images due to their larger size (I'm not sure if they could be compressed before sending them to ZeDMD; I thought I saw that you have the ability to compress data). However, what I observe is that when the image is displayed in the matrix, it doesn't appear immediately (while your logo does). So, I'm not sure if it's an issue with my code or the subsequent handling of the image by ZeDMD.

[mkalkbrenner]

Could you join https://discord.com/channels/1053573768536150017/1088882361225195690

[gustavoalara]

Hi Markus,

No, I can't.. the server refuses me

[mkalkbrenner]

@gustavoalara how can we get in contact?
Is there any server I can join?

[gustavoalara]

Hi Markus,

Once the issues with the new versions were resolved, I recompiled my program and tested image transmission in RGB24. As you can see in the attached video, the image doesn't appear instantly; instead, it undergoes a top-to-bottom sweep. This behavior, when sending frames of an animation, leads to suboptimal "refresh," and I believe it's not related to USB communication.

VID_20240123_222051.mp4

[gustavoalara]

A sample video

VID_20240123_224314.mp4

[mkalkbrenner]

@gustavoalara you use ZeDMD HD. Like mentioned above, the serial interface of the ESP32 is very limited. Even with the smart algorithm and compression, you can't go faster as 5 frames per second if you have a "movie scene".
If it is typical ROM content where parts of the picture remain unchanged, the algorithm is smart enough to support much higher frame rates.
For ZeDMD with a resolution of 128x32 you can get 13 fps for "movie scenes" using the ESP32:
https://www.youtube.com/watch?v=lQ25nDCFcrg

Again, with typical ROM content, the small ZeDMD is much faster.

I'm already working on the ESP32 S3 version. It's native USB interface is expected to be 10 times faster. And the ESP32 has more memory that will allow double buffering for smoother animations.
The ESP32 S3 is around 10$. So still very affordable. It should be an easy update for you as soon as I release the first version.

[gustavoalara]

Thanks @mkalkbrenner

I'll buy the S3 version to give a chance with your new code

One more question, do we have the same limitations when using the wifi communication? (Sorry, but I haven't tested the Wi-Fi functions you've added to the code yet.)

[mkalkbrenner]

WiFi has a higher bandwidth but more latency. So it depends on your use-case. Is it just a movie or backgrounf or do you need "real time".
But one issue remains. Due to memory limitations of the ESP32, single pixels need to be refreshed. The S3 has more memory, so we could use double buffering which will result in smoother animations and quicker drawing. At least I expect that.

[gustavoalara]

Hi @mkalkbrenner

My new couple of ESP32 S3 just arrived today, so I'm very exciting to check your firmware for this board. I see there is no version yet on the ZeDMD repo, do you have any beta to check?

[mkalkbrenner]

@gustavoalara be aware that the pinout of the S3 is different. It would be nice if you provide feedback.

https://github.com/PPUC/libzedmd/tree/s3
https://github.com/PPUC/ZeDMD/tree/s3

https://github.com/PPUC/libzedmd/actions/runs/7756962146
https://github.com/PPUC/ZeDMD/actions/runs/7756868850

[mkalkbrenner]

@gustavoalara To flash the S3 you must use the UART USB header. To run ZeDMD you need to use the native USB port.
I didn't optimize the code for the S3 yet, just increased the data tranfer rate. But there's more to come, for example double buffering.
I hope you bought a N16R8. Otherwise you need to adjust the partition table for the ZeDMD image. And PSRAM support won't work on most other variants.

[gustavoalara]

Hi @mkalkbrenner

Sorry for not writing earlier, due to family illnesses I couldn't take these tests. Just to catch up, does the latest version 0.7.0 also include support for S3 and version 3.5.2 of ZeDMD? (By the way, I bought the N16R8 of the S3).

[mkalkbrenner]

@gustavoalara the S3 basically works. But the firmware still misses the essential part to leverage the native USB communication.
The development is postponed a bit since we focus on VPX Standlone and batocera development at the moment.
BTW batocera 40 will include full ZeDMD support for VPX and EmuStation.

[mkalkbrenner]

And it is worth to mention that we focus on RGB565 / RGB16 encoding to improve the speed. libzedmd offers a method to encode RGB24 as RGB16 before sending to ZeDMD.

[gustavoalara]

Nice to hear about this changes. I'll give a try with the non S3 ESP

So, any plans to migrate these integration on Batocera Linux for Emulation station to the windows version or another Frontends like Launchbox or Attract-Mode?

[mkalkbrenner]

@gustavoalara latest versions became pretty fast:
https://www.youtube.com/watch?v=x-7imm9X13E

[gustavoalara]

Nice!

Currently I'm working with Ojacques in the DOF2DMD server and the Attract-Mode plugin, but we're using the flexdmd library to communicate with the ZeDMD firmware, so I guess Flexdmd need to be updated with your new library.

[gustavoalara]

Ah, I just seen Flexdmd uses DMD Extensions that today have been updated with your library too!

[francisdb]

What kind of DMD is that front one in the video?

[mkalkbrenner]

ZeDMD HD S3 in WiFi mode.