Please add YCgCo-R lossless compression to avif

[gitoss]

As discussed all over the internet, 'real' rgb-lossless avif compression (i.e. not yuv444) isn't stellar, to put it mildly: #111

There seems to be a way to improve compression by using YCgCo-R, as indicated in this thread: https://www.reddit.com/r/AV1/comments/kupvvl/firefox_will_support_the_avif_image_format_based/

This ticket is here as a reminder about this possible way to fix the current lackluster situation - otherwise the older and widely supported webp1 will keep being better than avif at lossless.

[kodawah]

I think the problem is just that the signaling is part of an external document, and I believe the proposal to address that is stuck in iso/itu standardization, but hopefully it will be sorted out soon

edit: jvet submisson link https://jvet-experts.org/doc_end_user/current_document.php?id=10608

[gitoss]

This JVET-U0093 mentions everything BUT avif, it's about AVC, HEVC, EVC, VVC, JPEG 2000, JPEG-XR and JPEG-XL ... is this because the JVET of the ITU-T SG wouldn't touch av1 with a ten-foot pole?

[dwbuiten]

@gitoss That is more for political reasons, yes. It is easier to convince them with codecs they're already comfortable with. AV1 was the very first codec I gathered data for, but including it would have been counter productive IMO.

That's politics for you...

(Source: Am one of the authors.)

[Enokilis]

@kodabb Old post, but the external document being referenced mentions that the YCgCo model is contingent on the bit depth of the components. Extending the chroma to 9 bits would imply YCgCo-R. The real issue seems to be how to standardize such a pixel format.

[alexismt73]

@Enokilis New code points that handle this were already approved and will be included in the next release of the CICP specification. We already provided a first version for balloting to ISO.

[leo-barnes]

YCgCo-R should be in the CICP spec from 2023 (at least). There is nothing more needed from the AVIF spec side to start using them.

[dwbuiten]

@leo-barnes While it is accepted in CICP, the actual document is not available to the public yet. ITU BT H.273 is in pre-publish[0], and thus not accessible to non-members, and the ISO/IEC 23091-2 update is still in draft.

I say this as one of the contributors to the codepoints :).

[0] https://www.itu.int/rec/T-REC-H.273
[1] https://www.iso.org/standard/85291.html

[kodawah]

@leo-barnes are there any test samples available by any chance? if not, could we include them in the AVIF test suite for ease of adoption?

[leo-barnes]

@dwbuiten

While it is accepted in CICP, the actual document is not available to the public yet.

While the most recent version is not yet published, the 2023 version is published and freely available and contains the YCoCg-R codepoints. See here: https://www.itu.int/rec/T-REC-H.273-202309-T/en

@kodawah

are there any test samples available by any chance? if not, could we include them in the AVIF test suite for ease of adoption?

Not sure. libavif has had experimental support for YCgCo-R for a while and now that T-REC-H.273-202309-T is published I think @wantehchang and @y-guyon were considering moving it out from under experiments.

[wantehchang]

@leo-barnes Leo: If the original image is 8-bit RGB and it is converted to 10-bit YCgCo-Re and then compressed losslessly in AV1, should the AVIF image format convey the fact that this 10-bit AVIF file should be converted to 8-bit RGB for consumption? Or is the use of the YCgCo-Re matrix coefficients automatically imply the AVIF image should be consumed this way?

[wantehchang]

See AOMediaCodec/libavif#2077 for YCgCo-Re and YCgCo-Ro support in libavif. Right now it is an experimental feature. We are planning to promote it to the stable API.

[dwbuiten]

@leo-barnes

While the most recent version is not yet published, the 2023 version is published and freely available and contains the YCoCg-R codepoints. See here: https://www.itu.int/rec/T-REC-H.273-202309-T/en

Oh you're right, my apologies.

[leo-barnes]

@wantehchang

If the original image is 8-bit RGB and it is converted to 10-bit YCgCo-Re and then compressed losslessly in AV1, should the AVIF image format convey the fact that this 10-bit AVIF file should be converted to 8-bit RGB for consumption?

Tricky question. The original purpose of the pixi was mainly to allow you to quickly answer questions like the ones below without having to parse the codec config (or for derived items, going into the input items):

• Will I require 10-bit HEVC decode support to be able to decode this image item?
• When rendering this image, do I need something more than an 8-bit pipeline?
• When transcoding this image to PNG, should I choose 8 bits or 16 bits as the bit-depth?

I think the spec language says that the pixi should match what the codec config says, since that is what matters when it comes to decoder capabilities. Even though the original content is 8-bits, both the decoder and your YCgCo-R -> RGB conversion will need 10-bit support in order to correctly handle the image, so I think the pixi should be saying it's 10 bits.

I unfortunately don't think we have any good signaling for saying "even though the pixi says we have X bits, we only really have Y bits of precision".

[alexismt73]

I agree with @leo-barnes . The CICP information only describes the coded representation. Not the intended usage and potentially the final format for consumption.

if you need something like that then you would likely need to include some additional metadata that tells you a preferred final colour space and in general representation (including resolution aspects for example). This may even include possible recommended conversion steps (e.g. if your content is in a 4:2:0 format include how conversion to a 4:4:4 format could be performed).