use FourCC codes for image encodings

[christianrauch]

Feature request

The sensor_msgs/Image currently uses the encoding field to document how the raw data in the data field has to be interpreted. The fourcc (4 character code) is a 32bit integer formed by 4 ASCII characters, that uniquely describes the memory layout.

Feature description

The encoding field can contain an arbitrary text string, with some predefined strings in sensor_msgs::image_encodings. Those strings are not standardised and do not document the memory layout properly. The YUV formats even reference non-official documentation containing advertisements.

Instead of using these non-standard strings, I propose to rely on the fourcc widely used across the kernel and the userspace to uniquely and properly document the memory layout.

Implementation considerations

To support fourcc, either a uint32 or a char[4] field fourcc should be added to the sensor_msgs/Image message definition alongside the old encoding. To maintain backward compatibility, the logic could be that if encoding is empty, the fourcc would take the place as a definition of the memory layout.

Alternatively, the encoding string could also contain the fourcc alongside the "legacy" encodings. However, this would require an expensive string matching to test if encoding contains a fourcc or a "legacy" format encoding and it must be ensured that there are no collisions.

[clalancette]

We had a short discussion about this today internally.

The most important point that came out of that discussion is that we aren't totally sure whether fourcc is the way to go for the encoding. That is, we aren't sure what pieces of software use fourcc, and what pieces of software use something else. In particular, we could use references to the kernel code, V4L2 code, OpenCV, and ROS code showing that this standard is actually widely used. That would allow us to make an easier decision about this. Thanks.

[christianrauch]

• kernel: https://github.com/torvalds/linux/blob/v5.19/include/uapi/drm/drm_fourcc.h
• V4L2: https://github.com/torvalds/linux/blob/v5.19/include/uapi/linux/videodev2.h#L518-L784
• OpenCV: https://github.com/opencv/opencv/blob/4.7.0/modules/videoio/include/opencv2/videoio.hpp#L1131-L1138
• ROS: As far as I know, FourCC is not used in the ROS ecosystem. The point of this proposal is to introduce FourCC to the ROS ecosystem, starting with the message.

FourCC is used all over the image/video/media stack to specify the memory layout. We should adapt this and use this specification instead of inventing custom formats that are not properly defined.