AVIF lossless worse than WebP lossless, occasionally even PNG

[cagelight]

Why does AVIF even bother having a lossless mode if it's going to be almost universally worse than WebP's? Isn't AVIF supposed to be superseding WebP? Why can't AVIF simply at least use WebP's lossless algorithm, which is an entirely different codepath? I know the simple answer is "just use WebP for lossless", but this seems like quite a flaw in AVIF's design if it has a lossless mode that nobody should ever use in the first place since it can't even beat its predecessor.

[leo-barnes]

This issue was discussed during the last meeting.

Can you provide some examples of the types of images that you find to work worse with lossless AVIF than with lossless WebP and which encoder you used to compress them?

Lossless WebP uses a LZ77 style compressor which will most likely work better for the type of content typically compressed with PNG. But for lossless compression of photographic content, I would have expected AVIF to work better than lossless WebP/PNG. It was also mentioned during the meeting that the AV1 lossless compression mode has not been as finely tuned as the lossy modes, so it may be possible to get better compression given some more tuning.

[cagelight]

http://archive.dogi.us/lossless_encode_tests/

I'll be uploading more examples as I make them over time, but the first image I've tested is photographic in nature and already has better results with WebP than AVIF.

[cagelight]

One interesting thing to note is that for i1_4000, the lossless AVIF performed even worse than the optimized PNG.

[cconcolato]

I assume the source is the PNG, right? Can you clarify which encoder was used? and maybe an example of command line.

[cagelight]

Please see the #README.txt file.

[cagelight]

That said, the JPEG2000 file is the source file (i1_source.jp2), the source has very little artifacting on its own, and the test images were samapled down to lower resolutions to remove residual artifacting.

[cconcolato]

avifenc can be compiled with multiple encoder support (libaom, rav1e, svt-av1, ...) . The command line in the README does not specify which encoder to use so it depends how you compiled avifenc. Can you clarify?

[cagelight]

The default (aom 2.0.0) is used, but I can also encode the examples with rav1e 0.3.4 if you'd like.

[cagelight]

I've just uploaded a second image collection, this one shows an even more dramatic deficit in performance over WebP.

EDIT: There was a bit depth issue that has been corrected, the second image set is now more inline with the previous results.

[Neltulz]

I actually 100% disagree with the lossless implementation of AVIF.

WebP's method of lossless encoding, is to use a completely separate lossless coding method. The WebP lossy format is based on a video format, and the lossless is better for image formats. The only downside to WebP lossless is, it doesn't support 16bit or 32bit, so it can't be used for archival purposes, which is disappointing.

WebP was criticized for being a "kitchen sink" format. I.e. it does too much, but in this regard, it was actually very smart to have a separate lossless compression method, since even today, Lossless WebP 8bit is very high ranking amongst available lossless formats with widespread support.

AVIF is basically using a video format for both the lossy and lossless method. This means that, compared to WebP lossless, it can perform worse, sometimes even spectacularly worse.

Not only that, but AVIF lossless isn't even true lossless, since it requires the image be converted to yuv4:4:4 or yuv4:2:0, which means if your image is originally RGB, you're incurring a lossy conversion just by changing from RGB to YUV. I've heard of ways in which the image can be losslessly converted between RGB and YUV, but that is unorthodox and I do not recommend it.

So, using AVIF lossless as an archival format is almost entirely stupid since the vast majority of work is done in RGB.

Anyway, you want my opinion? Don't use Lossless AVIF except for small files where you can clearly benefit. For most other cases, just use WebP lossless. If you absolutely need 16bit Lossless, just use PNG.

I would go as far as to suggest AVIF shouldn't even have lossless support, since it's extremely stupid in its current form, and the only time you'll consider using it is in rare edge case situations, and the time spent converting an image to AVIF, is a gamble of time wasted. If using AVIF Lossless is a benefit, great. If not, then you wasted a bunch of time waiting to find out. I digress, you're better off sticking with WebP lossless for 8-bit content and PNG for 16-bit.

[cagelight]

I agree that AVIF's lossless format is fundamentally stupid in its current form. I would go as far as to say it should be removed if it cannot or will not be replaced, so that nobody accidentally uses it.

[leo-barnes]

I actually 100% disagree with the lossless implementation of AVIF.

WebP's method of lossless encoding, is to use a completely separate lossless coding method. The WebP lossy format is based on a video format, and the lossless is better for image formats. The only downside to WebP lossless is, it doesn't support 16bit or 32bit, so it can't be used for archival purposes, which is disappointing.

WebP was criticized for being a "kitchen sink" format. I.e. it does too much, but in this regard, it was actually very smart to have a separate lossless compression method, since even today, Lossless WebP 8bit is very high ranking amongst available lossless formats with widespread support.

AVIF is basically using a video format for both the lossy and lossless method. This means that, compared to WebP lossless, it can perform worse, sometimes even spectacularly worse.

This is true of all compression methods. I can create image content that behaves spectacularly bad in a LZ77 based coding scheme (PNG/WebP) but would compress reasonably well by a block based encoder like AV1 and vice versa. I would actually have expected lossless AV1 to work better for photographic content than WebP/PNG. The fact that it doesn't seems to indicate that it's not working as efficiently as it could.

For comparison, I can compress a panorama image with PNG and lossless 444 HEVC in HEIF. The PNG ends up being 61 MB while the HEIF ends up being 42 MB.

Not only that, but AVIF lossless isn't even true lossless, since it requires the image be converted to yuv4:4:4 or yuv4:2:0, which means if your image is originally RGB, you're incurring a lossy conversion just by changing from RGB to YUV. I've heard of ways in which the image can be losslessly converted between RGB and YUV, but that is unorthodox and I do not recommend it.

You may want to take a look at issue #89
With that fix in place, doing fully lossless coding with a modest compression ratio gain should be possible.

So, using AVIF lossless as an archival format is almost entirely stupid since the vast majority of work is done in RGB.

Anyway, you want my opinion? Don't use Lossless AVIF except for small files where you can clearly benefit. For most other cases, just use WebP lossless. If you absolutely need 16bit Lossless, just use PNG.

I would go as far as to suggest AVIF shouldn't even have lossless support, since it's extremely stupid in its current form, and the only time you'll consider using it is in rare edge case situations, and the time spent converting an image to AVIF, is a gamble of time wasted. If using AVIF Lossless is a benefit, great. If not, then you wasted a bunch of time waiting to find out. I digress, you're better off sticking with WebP lossless for 8-bit content and PNG for 16-bit.

I see your point and agree with it to a certain extent. Some types of images will always compress better with PNG/WebP. But this is assuming that your source content is RGB. If your imaging pipeline gets data in YCbCr (potentially subsampled) and you want to save it to a lossless image format, PNG/WebP are not really good choices. This is actually a pretty common scenario in camera pipelines.

One other thing that was brought up in the last meeting is that LZ77 coding tends to be very unfriendly to HW acceleration.

[Neltulz]

One other thing that was brought up in the last meeting is that LZ77 coding tends to be very unfriendly to HW acceleration.

I asked some people in the AV1 discord about HW acceleration of AVIF, and their response was overwhelmingly, "that's not going to work well, if at all" and gave some technical reasons for it that I can't remember. Something about, parallelism not being possible when decoding lots of images on screen? I don't know. I remain skeptical.

What's your response regarding that? Can AVIF or WebP Lossless be HW accelerated with parallelism?

Not only that but, imo, Lossless should be preferred in either of these 2 cases (strictly web browsing related):

1. you're displaying lots of small images on screen such as icons or UI elements
2. Rare, edge case scenarios. you're displaying very few images on screen and you want the images to be pixel perfect to the original for whatever reason you might have. Perhaps you're showing a comparison between 2 images and they must be losslessly compressed to avoid subsequent artifacting from skewing results. Pick any reason out of a hat, basically that falls into the category of "edge case".

Lossless is almost never going to be a wise option if you're displaying lots of images on screen, such as thumbnail images for a video service or an image gallery. You're better off using near lossless, or lossy in that case, since people are not going to be scrutinizing images. Therefore, HW acceleration / parallelism shouldn't be a huge problem that requires solving in the case of Lossless image display, imo. Correct me if I'm wrong.

If your imaging pipeline gets data in YCbCr (potentially subsampled) and you want to save it to a lossless image format, PNG/WebP are not really good choices.

That is very edge case, and assumes the user has not made any modifications to the image in a photo editing program. Those usually convert the image to RGB for editing purposes. So, while you're right about this, it doesn't offer a sufficient reason for its existence.

What AVIF needs, is a way for it to detect input colorspace, and store the image losslessly in that color space. Pick the right lossless compression format that best suits the job. That means, that AVIF is going to bloat in complexity as it tries to please every color space possibility. At that point, we need a dedicated lossless image format that isn't AVIF to avoid AVIF ballooning into an even bigger kitchen sink format than WebP. Adding support for all of this is just going to add complexity to decoding and make it difficult for developers to keep up with the advancements of AVIF.

It's my opinion that AVIF should have the lossless compression method removed completely (since it's stupid and encourages people to waste a bunch of time encoding in an inferior lossless format), and make room for an actual lossless image format that supports multiple color spaces and picks the best lossless compression method for the job, supports multiple bit depths, and HDR.

[leo-barnes]

One other thing that was brought up in the last meeting is that LZ77 coding tends to be very unfriendly to HW acceleration.

I asked some people in the AV1 discord about HW acceleration of AVIF, and their response was overwhelmingly, "that's not going to work well, if at all" and gave some technical reasons for it that I can't remember. Something about, parallelism not being possible when decoding lots of images on screen? I don't know. I remain skeptical.

It very much depends on the exact scenario and how tight your integration is with the HW. HW tends to take a while to set up, but once started tends to be much faster (and very importantly much more power efficient) than SW. It also tends to be bad at context switching. In some cases the overhead of setting up the HW may be large enough that SW can be faster, but the more complicated the codec becomes, the faster HW tends to be compared to the SW.

Does this mean HW acceleration can't be used for small images? No, it may actually in some cases be preferable to the SW decoder even if it's slower since it will use less battery on embedded devices. Steps can also be taken by the content creator to make HW acceleration easier and more efficient by minimizing the context switches.

I don't fully understand the comment on parallelism not being possible when decoding lots of images on screen. The same holds for decoding on the CPU if you have more images than CPU cores. Or is this with regards to progressive refinement where you want to jump from image to image before they have fully downloaded?

All in all, I know for a fact that iOS and macOS uses HW accelerated image decoding extensively, so it can definitely be done. It may require tight integration with the HW in question though.

What's your response regarding that? Can AVIF or WebP Lossless be HW accelerated with parallelism?

WebP Lossless can't be HW accelerated since LZ77 is very hard to implement in HW. I would guess that most AV1 HW will support lossless decoding. The with parallelism part needs clarification to fully understand what the ask is.

Not only that but, imo, Lossless should be preferred in either of these 2 cases (strictly web browsing related):

1. you're displaying lots of small images on screen such as icons or UI elements
2. Rare, edge case scenarios. you're displaying very few images on screen and you want the images to be pixel perfect to the original for whatever reason you might have. Perhaps you're showing a comparison between 2 images and they must be losslessly compressed to avoid subsequent artifacting from skewing results. Pick any reason out of a hat, basically that falls into the category of "edge case".

Lossless is almost never going to be a wise option if you're displaying lots of images on screen, such as thumbnail images for a video service or an image gallery. You're better off using near lossless, or lossy in that case, since people are not going to be scrutinizing images. Therefore, HW acceleration / parallelism shouldn't be a huge problem that requires solving in the case of Lossless image display, imo. Correct me if I'm wrong.

I agree that lossless compression is used in many cases on the web where lossy compression in a more modern format than JPEG would be better. The benefits of HW acceleration depends very heavily on the exact use case you're trying to solve. I know that the slowness of decoding of PNG is considered a pretty major issue in some cases and that having a format that would have been more amenable to acceleration would probably have been beneficial there.

If your imaging pipeline gets data in YCbCr (potentially subsampled) and you want to save it to a lossless image format, PNG/WebP are not really good choices.

That is very edge case,

It may be an edge case for you, but for me and many other people working with camera pipelines it is much more common to work in YCbCr than it is to work in RGB. I would consider AVIF dropping support for lossless encoding to be a severe shortcoming compared to HEVC in HEIF. If you drop support for lossless encoding you also drop the ability of having lossless auxiliary images which in some cases is very important.

and assumes the user has not made any modifications to the image in a photo editing program. Those usually convert the image to RGB for editing purposes. So, while you're right about this, it doesn't offer a sufficient reason for its existence.

Photo editing programs almost always work in some kind of increased or floating point precision intermediary formats. On export, that intermediary format is (lossily) converted into a representation that can be stored in the target image format.

If your source image is photographic, it is almost guaranteed to be stored in (potentially lossless) YCbCr. If you do an edit that only touches a part of the image and export it as RGB, you will have changed all pixels in the image. If you instead export it as lossless YCbCr, you will only have changed the pixels you have actually modified if the intermediary representation has enough precision (which it most likely has).

If you don't care about this loss, then what's the point of using a lossless format?

If I take uncompressed photographic 8-bit YCbCr image data and compress it as lossless AV1, the output is much smaller than the corresponding file when converted (lossily) to 8-bit RGB and compressed as PNG. So the codec itself can very efficiently encode lossless YCbCr. With regards to the experiments done by @cagelight, it's most likely getting encoded as RGB, which is not a very well used path in AV1. Given how much better the results are for YCbCr, it may be possible to improve it if time is spent analyzing what is happening.

[cagelight]

Assuming AV1 can't be optimized to compete with WebP's lossless encoding for RGB data, would it be acceptable to simply merge WebP's LZ77-based encoding into the AVIF format only for RGB-based images (leaving AV1's lossless YCbCr lossless) to fully phase out WebP? I know it's slightly bloating the format, but personally I'd rather one slightly bloated format than two largely overlapping formats that excel at their own edge cases.

[gitoss]

Can you provide some examples of the types of images that you find to work worse with lossless AVIF than with lossless WebP and which encoder you used to compress them?

There's a reddit thread on it including tables with a comparison of various lossless image codecs... https://www.reddit.com/r/jpegxl/comments/l9ta2u/how_does_lossless_jpegxl_compared_to_png

... and here the chart: https://cdn.discordapp.com/attachments/673202643916816384/791488914413584419/Public_Images_bytes.png

Unless avif at least manages to use YCgCo-R instead of the current YCbCr hack, this doesn't look like a "modern" codec should :-\

[gnat]

I will say, this was a pretty shocking deficiency to discover while considering avif in production myself. I very much want avif to succeed.

Assuming AV1 can't be optimized to compete with WebP's lossless encoding for RGB data, would it be acceptable to simply merge WebP's LZ77-based encoding into the AVIF format only for RGB-based images (leaving AV1's lossless YCbCr lossless) to fully phase out WebP? I know it's slightly bloating the format, but personally I'd rather one slightly bloated format than two largely overlapping formats that excel at their own edge cases.

This sounds like a novel idea.

2 strategies 1 format is the path forward for WEBP, WEBP2 and the incoming JXL format as well.

[gitoss]

And that's all from Google employees.

What did you expect? AV1 was never designed to be an effecive catch-all image codec, making AVIF an second-tier image format. The YCgCo-R hack seems to be blocked for political reasons, aka the MPEG people won't help getting AVIF fixed.

I don't know if adding webp1-ish lossless-compression was ever discussed. It breaks (possible) hardware decoding and adds complexity = target surface for exploits. With the rise of JPEG-XL it's too probably late anyway... back in the day's of webp1 there simply was no lossless competition.

[cagelight]

With the rise of JPEG-XL it's too probably late anyway

I've long since moved onto JPEG-XL since opening this issue personally. AVIF is just too entangled in BS, I consider it a format dead in the water.

[gitoss]

With the rise of JPEG-XL it's too probably late anyway

I've long since moved onto JPEG-XL since opening this issue personally. AVIF is just too entangled in BS, I consider it a format dead in the water.

After a sprawling development and discussion phase, libavif seems to have moved into low maintenance mode, too.

[cagelight]

rav1e 0.3.4

Does not support lossless... Only aomenc does.

Learn some reading comprehension. rav1e 0.3.4 was not used, I offered to use it not knowing much about it, but it was never used.

i1_4000, the lossless AVIF performed even worse than the optimized PNG.

That is because i1_4000.jp2 does not use lossless jpeg 2000, are you serious right now?

Are you serious right now? The encoding of the JPEG 2000 is completely irrelevant as for both the PNG and the AVIF, the decoded image data is used.

[cagelight]

The fact is you did not even know it does not support lossless?

Correct. And it could not possibly matter less because it was an off-hand comment that didn't lead into anything. Pick better hills to die on.

As for JPEG 2000 of course it does matter! 9/7 wavelet transform with ICT introduces crazy artifacts that work like that problem with film grain.

That literally changes nothing. It proves that in that example, PNG works better than AVIF when dealing with film grain which is a perfectly valid data point, what exactly is the problem?

[wangshengdong]

I hope add a visually lossless compression mode for archiving photos.