rpi-connect wayvnc out of memory error

[george-ivanov]

Following an update and reset of the bootloader to the latest version (7th of December) the connect service stopped working with the following error in the logs:

Dec 11 17:04:19 raspberrypi systemd[772]: Reloading.
Dec 11 17:04:20 raspberrypi systemd[772]: Started rpi-connect-wayvnc-watcher.path.
Dec 11 17:04:20 raspberrypi systemd[772]: Started rpi-connect.service - Raspberry Pi Connect.
Dec 11 17:04:21 raspberrypi rpi-connect[2901]: fatal error: runtime: out of memory
Dec 11 17:04:21 raspberrypi kernel: __vm_enough_memory: pid: 2909, comm: rpi-connectd, not enough memory for the allocation
Dec 11 17:04:21 raspberrypi rpi-connect[2901]: runtime stack:
Dec 11 17:04:21 raspberrypi rpi-connect[2901]: runtime.throw({0x68256c?, 0x255d8?})
Dec 11 17:04:21 raspberrypi rpi-connect[2901]:         runtime/panic.go:1067 +0x38 fp=0x40001dbdf0 sp=0x40001dbdc0 pc=0x7f3f8
...
Dec 11 17:04:21 raspberrypi systemd[772]: rpi-connect.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 11 17:04:21 raspberrypi systemd[772]: rpi-connect.service: Failed with result 'exit-code'.
Dec 11 17:04:23 raspberrypi systemd[772]: rpi-connect.service: Scheduled restart job, restart counter is at 1.
Dec 11 17:04:23 raspberrypi systemd[772]: Stopped rpi-connect-wayvnc-watcher.path.
Dec 11 17:04:23 raspberrypi systemd[772]: Stopping rpi-connect-wayvnc-watcher.path...
Dec 11 17:04:23 raspberrypi systemd[772]: Started rpi-connect-wayvnc-watcher.path.
Dec 11 17:04:23 raspberrypi systemd[772]: Stopping rpi-connect-wayvnc.service - WayVNC process used by Raspberry Pi Connect...
Dec 11 17:04:23 raspberrypi systemd[772]: Stopped rpi-connect.service - Raspberry Pi Connect.
Dec 11 17:04:23 raspberrypi systemd[772]: Started rpi-connect.service - Raspberry Pi Connect.
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[1446]: DEBUG: ../src/ctl-server.c: 717: New connection
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[1446]: Info: New control socket client connected: 0x557db80660
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[2940]: ERROR: ../src/ctl-client.c: 284: Disconnected
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[1446]: DEBUG: ../src/ctl-server.c: 307: << {"method":"wayvnc-exit","params":{},"id":1}
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[1446]: Info: Dispatching control client command 'wayvnc-exit'
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[1446]: Warning: ../src/main.c: 715: Shutting down via control socket command
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[1446]: Info: Enqueueing response: OK (0)
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[1446]: DEBUG: ../src/ctl-server.c: 567: Response data: (null)
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[1446]: Info: Exiting...
Dec 11 17:04:23 raspberrypi systemd[772]: Stopped rpi-connect-wayvnc.service - WayVNC process used by Raspberry Pi Connect.
Dec 11 17:04:23 raspberrypi systemd[772]: Starting rpi-connect-wayvnc-watcher.service - Reload rpi-connect when rpi-connect-wayvnc changes...
Dec 11 17:04:23 raspberrypi systemd[772]: Starting rpi-connect-wayvnc.service - WayVNC process used by Raspberry Pi Connect...
Dec 11 17:04:23 raspberrypi systemd[772]: Started rpi-connect-wayvnc.service - WayVNC process used by Raspberry Pi Connect.
Dec 11 17:04:23 raspberrypi systemd[772]: Reloading rpi-connect.service - Raspberry Pi Connect...
Dec 11 17:04:23 raspberrypi rpi-connect[2941]: 2024/12/11 17:04:23 INFO No VNC server found, reloading without screen sharing support
Dec 11 17:04:23 raspberrypi rpi-connect[2941]: fatal error: runtime: out of memory
Dec 11 17:04:23 raspberrypi kernel: __vm_enough_memory: pid: 2949, comm: rpi-connectd, not enough memory for the allocation
...
Dec 11 17:04:23 raspberrypi systemd[772]: rpi-connect.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 11 17:04:23 raspberrypi rpi-connect[2959]: ✗ unexpected EOF
Dec 11 17:04:23 raspberrypi systemd[772]: rpi-connect.service: Control process exited, code=exited, status=1/FAILURE
Dec 11 17:04:23 raspberrypi systemd[772]: rpi-connect.service: Failed with result 'exit-code'.
Dec 11 17:04:23 raspberrypi systemd[772]: Reload failed for rpi-connect.service - Raspberry Pi Connect.
Dec 11 17:04:23 raspberrypi systemctl[2956]: Job for rpi-connect.service failed because the control process exited with error code.
Dec 11 17:04:23 raspberrypi systemctl[2956]: See "systemctl --user status rpi-connect.service" and "journalctl --user -xeu rpi-connect.service" for details.
Dec 11 17:04:23 raspberrypi systemd[772]: rpi-connect-wayvnc-watcher.service: Main process exited, code=exited, status=1/FAILURE
Dec 11 17:04:23 raspberrypi systemd[772]: rpi-connect-wayvnc-watcher.service: Failed with result 'exit-code'.
Dec 11 17:04:23 raspberrypi systemd[772]: Failed to start rpi-connect-wayvnc-watcher.service - Reload rpi-connect when rpi-connect-wayvnc changes.
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[2958]: Info: Capturing output NOOP-1
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[2958]: Info: >> Headless output 2 1920x1080+0x0 Power:UNKNOWN
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[2958]: Info: Listening for connections on /run/user/1000/rpi-connect-wayvnc.sock
Dec 11 17:04:23 raspberrypi rpi-connect-wayvnc[2958]: DEBUG: ../src/ctl-server.c: 809: Initializing wayvncctl socket: /run/user/1000/rpi-connect-wayvnc-ctl.sock
Dec 11 17:04:23 raspberrypi systemd[772]: Starting rpi-connect-wayvnc-watcher.service - Reload rpi-connect when rpi-connect-wayvnc changes...
Dec 11 17:04:23 raspberrypi systemd[772]: rpi-connect.service: Unit cannot be reloaded because it is inactive.
Dec 11 17:04:23 raspberrypi systemctl[2986]: rpi-connect.service is not active, cannot reload.
Dec 11 17:04:23 raspberrypi systemd[772]: rpi-connect-wayvnc-watcher.service: Main process exited, code=exited, status=1/FAILURE
Dec 11 17:04:23 raspberrypi systemd[772]: rpi-connect-wayvnc-watcher.service: Failed with result 'exit-code'.
Dec 11 17:04:23 raspberrypi systemd[772]: Failed to start rpi-connect-wayvnc-watcher.service - Reload rpi-connect when rpi-connect-wayvnc changes.
Dec 11 17:04:25 raspberrypi systemd[772]: rpi-connect.service: Scheduled restart job, restart counter is at 2.

I've restored and reset to the previous bootloader version but that made no difference.
Uninstalling and reinstalling rpi-connect also made no difference.
Any ideas please?

[any1]

Maybe it would be a good idea to see what's consuming all that memory.

[george-ivanov]

Hmmm there's plenty of free system memory, if that's what you mean:

MiB Mem :   3797.9 total,   2743.3 free,    474.4 used,    673.7 buff/cache     
MiB Swap:    200.0 total,    200.0 free,      0.0 used.   3323.5 avail Mem 

[any1]

Does dmesg say anything interesting about the oom?

[george-ivanov]

Yes, lots of these:

 250.024017] __vm_enough_memory: pid: 2676, comm: rpi-connectd, not enough memory for the allocation
[  252.388414] __vm_enough_memory: pid: 2726, comm: rpi-connectd, not enough memory for the allocation
[  254.651530] __vm_enough_memory: pid: 2774, comm: rpi-connectd, not enough memory for the allocation
[  256.878963] __vm_enough_memory: pid: 2821, comm: rpi-connectd, not enough memory for the allocation
[  259.114922] __vm_enough_memory: pid: 2870, comm: rpi-connectd, not enough memory for the allocation
[  261.385321] __vm_enough_memory: pid: 2922, comm: rpi-connectd, not enough memory for the allocation
[  263.613517] __vm_enough_memory: pid: 2960, comm: rpi-connectd, not enough memory for the allocation
[  265.861285] __vm_enough_memory: pid: 3010, comm: rpi-connectd, not enough memory for the allocation
[  268.159837] __vm_enough_memory: pid: 3064, comm: rpi-connectd, not enough memory for the allocation
[  270.270944] systemd-journald[273]: Data hash table of /run/log/journal/2f56b173d23e4d72be01e2dcc655c419/system.journal has a fill level at 75.0 (6486 of 8647 items, 4980736 file size, 767 bytes per hash table item), suggesting rotation.
[  270.270963] systemd-journald[273]: /run/log/journal/2f56b173d23e4d72be01e2dcc655c419/system.journal: Journal header limits reached or header out-of-date, rotating.

[george-ivanov]

Just connected the device to an external display and it's not able to start wayland with labwc so something sinister has happened during or after that bootloader update. So shouldn't be an issue with rpi-connect/wayvnc.
Closing this. Thanks for your time!