AVIF HDR and premultiplied alpha

[0xC0000054]

What is the recommended method for handling HDR images with premultiplied alpha?

I have been working on adding HDR support to my AVIF plug-in for Photoshop, and premultiplied alpha is one of the features that the plug-in supports.

I am assuming that the image data would be premultiplied before the BT. 2020 PQ curve is applied to the RGB values.

Is there any code (e.g. in Chromium) that shows how this is handled?
The Krita code has been helpful as a general guide on what is required for HDR decode/encode support, but it does not appear to include premultiplied alpha handling.

Sample images would also be useful for testing, but I can always generate my own after I start working on implementing the encoder side.
The only HDR test images I could find in this repository are the Netflix samples, and none of those have an alpha channel.

[leo-barnes]

Tricky question. Let me ask around a bit among the HDR experts.

I'm not sure if the spec explicitly states that premultiplication should be done in linear RGB space. Not doing it linear space works ok for SDR content, but I can imagine that the artifacts would be pretty bad if done in gamma space for a PQ image.

[0xC0000054]

I recently tested this.
It appears to work fine when the premutiplication is done before the PQ curve is applied or removed, no artifacts that I can see.

The premultiplication will result in the image data being clipped to the [0, 1] range, and the resulting image has a larger file size than the straight alpha image (at least with the Photoshop rendered image I was testing with).

The following zip file contains both the straight and premultiplied alpha AVIF images.
Both images were saved as 12-but YUV 4:2:2 with a PQ curve, but they are not technically HDR.

AVIF HDR premultiplied alpha test.zip

[leo-barnes]

We'll be having some more discussions about this internally in a while, but the gist of it so far is this:

1. Optimal blending of HDR images has not really been researched in-depth.
2. Optimal scaling of HDR images has been researched however. The findings can be found here. Contrary to my gut feeling about it, scaling of HDR content in linear space may actually cause more artifacts than when scaling in gamma space. The tl;dr is that filters like Lanczos have negative taps that may cause ringing, and that this is exacerbated in linear space for HDR content.
3. On the other hand, there is plenty of examples where scaling SDR content in gamma space also causes unwanted visual artifacts.
4. There is currently no signaling within HEIF/AVIF to say if image operations like blending or scaling should be done in some specific color space. This is something that maybe should be added, so that a content creator could add a hint saying that "this image should ideally be processed or handled in linear" or "this image should ideally be processed in ICtCp space" or similar.

Some relevant papers:
https://wagtail-prod-storage.s3.amazonaws.com/documents/Pytlarz_Paper.pdf
https://professional.dolby.com/siteassets/pdfs/ictcp_dolbywhitepaper_v071.pdf
https://www.w3.org/Graphics/Color/Workshop/slides/Kunkel.pdf