ROS2 topic hz gives incorrect value with camera image

[EwingKang]

Bug report

Required Info:

• Operating System:
  • Ubuntu 22.04
• Installation type:
  • Binary/debian
• Version or commit hash:
  • Iron
• DDS implementation:
  • Default (fastDDS)
• Client library (if applicable):
  • rclpy (from ros2 topic hz)

Steps to reproduce issue

I'm using a Lenovo E16 laptop that comes with built-in camera

# Install camera node
sudo apt-get install ros-iron-usb-cam
# start camera
ros2 run usb_cam usb_cam_node_exe
# Start monitor
ros2 topic hz /image_raw

Expected behavior

Reporting 30 FPS

*The misspelling are purposely done to identify my modification to the installed package

Actual behavior

Reporting 15-22 FPS or even lower

*The misspelling are purposely done to identify my modification to the installed package

Additional information

Affecting source code: here
Following the hint from this answers.ros discussions, I try modifying the QoS setting directly in the installed location /opt/ros/iron/lib/python3.10/site-packages/ros2topic/verb/hz.py. By changing the qos profile line to rclpy.qos.qos_profile_parameters, it is possible for ros2 topic hz to report correct frequency.

QoS Profile	qos_profile_sensor_data	qos_profile_parameters
Reported rate	15-22	30.077-30.083
CPU usage	8-10%	11-14%

Expected value: 30 Hz as configured by the camera node

I believe the primary factor is the RELIABILITY_RELIABLE. This is because I can also replicate frame loss and thus image stutter in rviz2 image viewer if the reliability Policy is set to Best Effort.

This is related to : #843, and might be alleviated with #719

[EwingKang]

I think there's a debate to made here: I think there's little point for the tool if the ros2 topic hz cannot measure the correct frequency. However, sending large amount of data to a tool just for measuring the frequency, does create significant CPU impact, potentially lowers the actual FPS, to the system.
I'm using a comparatively powerful CPU here (Lenovo E15 with Intel i7-1355U). My system is far from being congested and can definitely handles 30 FPS. The RVIZ shows 30 FPS without problem even the ros2 topic hz is only giving me 15. (BTW: ros2 run image_view image_view and rqt both also stuttered a lot, I presume they also use best effort as their QoS setting.
This can cause a lot of confusion especially for new comers. Camera image is such a common topic for robots that I think this needs to be addressed properly.

[fujitatomoya]

@EwingKang as you mentioned already, this is the same with #719. that would be really appreciated if you consider the contribution.

[clalancette]

So, I think there are two different things here.

The first problem is that rclpy is significantly slower than rclcpp. That's the cause of the difference in FPS between RViz (which is using rclcpp), and ros2 topic hz (which is using rclpy). This is being discussed elsewhere, like in ros2/ros2#1499 .

The follow-up comment is about sending large amounts of data to a subscriber, just for that subscriber to throw it all away to count packets. I agree that it is fairly wasteful, though it isn't totally clear to me how we would avoid it. Possibly we could do so with some changes to the RMW layer/DDS implementations, but even then I'm not sure how we would do it.

For now, I'm going to close this as a duplicate of ros2/ros2#1499 . If you disagree with that assessment, feel free to reopen.

[fdila]

I have lost quite a bit of time due to this issue (thinking that my camera driver was not working correctly, when in fact it was just ros2 topic hz that was reporting the wrong frequency).
I believe that, even if this issue will not be solved, some kind of warning when running ros2 topic hz should be given, or at least mention this limitation when printing the help with ros2 topic hz --help

[fdila]

Just coming back to this issue to say that apparently setting up the DDS middleware to handle large amounts of data seems to fix the issue.
I switched to Cyclone DDS and followed the ROS2 guide on tuning (https://docs.ros.org/en/humble/How-To-Guides/DDS-tuning.html#cyclone-dds-tuning) and now it seems like ros2 hz reports the right frequency.
For reference I tried to measure the frequency of large pointclouds coming from an Ouster LiDAR at 10 hz and ithe ros2 hz correctly reports a frequency of 10 hz

[fujitatomoya]

@fdila i assumed that you were using rmw_fastrtps before rmw_cyclonedds? if that is so, could you try https://github.com/ros2/rmw_fastrtps?tab=readme-ov-file#large-data-transfer-over-lossy-network if that works for your case?

I believe that, even if this issue will not be solved, some kind of warning when running ros2 topic hz should be given, or at least mention this limitation when printing the help with ros2 topic hz --help

sounds reasonable. aligns with #719 (adding QoS settings for ros2 topic hz.), we can add more explanation.

[fdila]

@fujitatomoya Hi, yes before cyclonedds I was using the default fastrtps.
I already tried the solution linked but it had no effect, I think because it's about transport of messages between nodes that run in separate hosts, while I'm running everything on the same host. As far as I understood (I don't really know much about networking and processes communication) when two nodes run on the same host fastrtps uses something called shared memory transport.

As you have seen in this other issue (ros2/rmw_fastrtps#739) I've tried playing around with this transport mechanism but I had not much luck with configuring fastrtps, I'll try your suggestion once I get back on my work PC.

Anyway, I would assume that by configuring fastrtps "correctly" would fix the ros2 hz issue the same way that switching to cyclonedds.