Not able to save from 32-bit documents?

[gregbenz]

When working with a 32-bit document, the "save as" and "save a copy" options do not list an AV1 option (though I see it in the open filter, so the plugin is properly installed). Is this a bug? Is support planned? Would be great to be able to export 32-bit TIF files as HDR images in the AVIF format.

Thank you so much for creating this plugin.

[0xC0000054]

You will need to convert the document to 16-bit to save as AVIF.

AVIF supports 10-bit or 12-bit HDR, so I only implemented support for Photoshop's 16-bit image mode.
It may be possible to add support for saving 32-bit documents, but I am not sure what that work would involve.

After thinking about it, the plugin is missing a few features in its HDR support.
The save dialog need to allow the user to set the color primaries, transfer characteristics and matrix coefficients, e.g. saving a 10-bit image as BT2020 would require the color primaries and transfer characteristics to be set to the appropriate values.

[gregbenz]

Unfortunately, Photoshop only supports HDR encoding at 32-bits (does seem rather odd that we've got 10-bit HDR display and need 32-bit source files to match - though the Lightroom HDR DNG encodes at 16-bits using floating point to cover the dynamic range).

If there's a straight-forward way to support saving from 32-bit mode, that would be amazing. I have yet to find any way to convert a (32-bit) HDR image edited in Photoshop over to a suitable format suitable for sharing on the web.

I'm not too clear on how the data pipeline for file plugins in PS works. You're saying those primaries and such aren't something the plugin would be able to get from the image? The encoding is a bit odd there. If you convert any profile in 32-bits, it gets altered substantially (seems like a bug in PS). It seems to pull the primaries from whatever profile you select at the time of creation or conversion for the 32-bit image and then applies a linear gamma. Not what how it might inherit or alter other aspects of the profile, but it is embedding the profile during saves.

[0xC0000054]

If you convert any profile in 32-bits, it gets altered substantially (seems like a bug in PS). It seems to pull the primaries from whatever profile you select at the time of creation or conversion for the 32-bit image and then applies a linear gamma

I ran into that the last time I investigated Photoshop's 32-bit mode.
Unfortunately, the details of the 32-bit image mode appear to be missing from the Photoshop SDK documentation (or at least I could not find it).
The only thing I found is the code for the Dissolve example filter, which confirms that it uses 32-bit floating point in the [0,1] range.

You're saying those primaries and such aren't something the plugin would be able to get from the image?

I do not think so. That information would have to come from the user.

I am not very familiar with the differences between the AVIF color information and ICC profiles.
AV1 was originally designed as a video compression format, so that probably helps explain why AVIF has multiple ways of storing color information.

I have yet to find any way to convert a (32-bit) HDR image edited in Photoshop over to a suitable format suitable for sharing on the web.

The only AVIF images I am aware of that use HDR are some of the Netflix samples in the av1-avif repository.
According to the read-me in that folder the HDR samples were generated from 16-bit PNGs using the avifenc utility from libavif.
I am not sure if there is any AVIF converter that could process a floating point image format.

[gregbenz]

That's unfortunate. I might be able to help track down some of that info if its something you want to pursue, but sounds like a significant project.

ImageMagick is able to extract an ICC profile from a 32-bit file. The profile shows all the key color information. If PS is writing it, I'm sure it must be available in C++ somehow, but sounds like it is not documented.
magick "sourceFile.tif" icc:"extractedProfile.icc"

Know of any tool that can convert 32-bit TIF to HDR PNG? I don't know if browsers generally support HDR from such a PNG, but would be interesting to test. The files would be larger, but it would at least create an option.

[0xC0000054]

Know of any tool that can convert 32-bit TIF to HDR PNG?

While checking the av1-avif repository I came across an issue that mentioned the AVIF HDR support in Krita.
Krita can load OpenEXR files and load and save AVIF files. From the same issue, it appears that Chrome has better support for AVIF HDR than Firefox.

According to AOMediaCodec/av1-avif#186 (comment) the 'HDR' PNGs contain floating point values that are encoded using a fixed point representation (The PNG format does not support floating point).
I doubt that the format conversion would be as simple as converting the floating point values to 10-bit or 12-bit fixed point, but I may be wrong.

I am considering opening an issue in the av1-avif repository to see if anyone on there can provide advice on what would be needed to try and implement this feature.

[gregbenz]

Thank you, I'll check that out.

Due to the loss of color data (when using something better than sRGB), OpenEXR is probably a bit limiting. Even Photoshop seems to discard wide gamut color when saving to this format and re-opening the file. The "HDR" format seems similarly limited. I've only found TIF, PSB, and PSB keep full color with Photoshop thus far, and I suspect that's probably going to be generally true of other tools for OpenEXR/HDR.

[0xC0000054]

I've only found TIF, PSB, and PSB keep full color with Photoshop thus far, and I suspect that's probably going to be generally true of other tools for OpenEXR/HDR.

Krita also has TIFF and PSD support.

[gregbenz]

Krita is interesting. It will export a proper HDR PNG (when you select the appropriate settings). The output shows as a true HDR image in PS, but sadly does not render properly in Chrome, FireFox or Safari. The PNG export is using the profile to achieve HDR output values, and I don't know if this suggests much about potential for HDR browser rendering of other formats such as AVIF or JXL.

Krita will also export AVIF and HEIC with options for HDR (HLG or PQ), but I'm not sure the output is valid. If you open the same exported images back in Krita, the HDR values have been clipped. If you open the HEIC in PS, the color is wrong. If you evaluate the output with ImageMagick, there appears to be no embedded profile. Hard to say given various limitations here, but I don't have a lot of confidence that these file types are exported properly as HDR images.

JXL exports do not seem to offer any HDR options, and the export fails with an error message anyway.

32-bit TIF support is broken on MacOS (current release of Krita will crash), but works on Windows.

[gregbenz]

FYI, the following image renders properly as HDR in Chrome, but opens as SDR (16-bit sRGB profile) when opened via avif-format in Photoshop.

https://raw.githubusercontent.com/AOMediaCodec/av1-avif/master/testFiles/Netflix/avif/hdr_cosmos01000_cicp9-16-0_lossless.avif

[0xC0000054]

FYI, the following image renders properly as HDR in Chrome, but opens as SDR (16-bit sRGB profile) when opened via avif-format in Photoshop.

That is the expected behavior, the plugin does not implement any HDR support.
The 10-bit and 12-bit support was intended to allow high bit-depth SDR images to be converted to AVIF, e.g. camera raw images.

Those HDR images do not include a color profile, so I am not sure what I could do about that issue.
Photoshop would most likely require a color profile in order to properly render the image.

[gregbenz]

Gotcha, thank you for confirming. I hadn't looked closely at how PS renders the PNG (https://github.com/AOMediaCodec/av1-avif/blob/master/testFiles/Netflix/avif/original_hdr_cosmos01000.png) vs how Chrome renders the AVIF derived from than PNG. I can't open both in a single application to ensure the images are the same, but the PS PNG and Chrome AVIF are not the same - PS is brighter and I can't say if the colors match but are certainly close - but so many factors could create the difference.

If that AVIF has no embedded profile, any idea what Chrome assuming to render? PQ + Rec2020 primaries would seem like an ideal guess for an HDR image, but could see HLG, sRGB, monitorRGB etc all being possible assumptions for untagged images.

[0xC0000054]

If that AVIF has no embedded profile, any idea what Chrome assuming to render?

From the Chromium source code, it looks like it may be creating an appropriate color profile based on the CICP data. Those AVIF images do not appear to contain an embedded ICC color profile.

[gregbenz]

I'm not too familiar with CICP. Does that mean a unique enum / reference to save space wasted with embedding common color spaces ?

[gregbenz]

FYI, I was able to generate an HDR AVIF by converting a 32-bit TIF to a 16-bit HDR PNG and then using AVIFENC to convert that PNG to AVIF. It will display as HDR in Chrome (Firefox renders it improperly/dark, and Safari won't open it at all yet but MacOS 13 is supposed to add some level of AVIF support).

The key with Chrome is to export the HDR AVIF without the ICC profile. An untagged image won't match the original exactly (I'm not sure what assumptions Chrome makes for untagged images), but it will display as a proper HDR.

If you embed the profile in the AVIF, Chrome will not render an HDR AVIF properly. It will display it as clipped, and at times shows horrible artifacts (over-range white shows as red, whole image may look mostly black with green dots, flickering, etc). So it appears there are problems with Chrome color management, but it will support some HDR AVIF files.

While the above works, it's a messy workflow. Conversion to 16-bit PNG in Photoshop is dependent on the 32-bit colorspace of your source image, which cannot be converted to other 32-bit spaces without harming the image (gets very light as if a TRC has been applied incorrectly, looks like a PS bug). And the PNG conversion itself is tricky as AVIFENC requires someone use a Windows command line tool (or compile the code themselves on MacOS as I don't believe there are public builds posted). But it at least shows proof of concept for rendering HDR in a browser.

Chrome seems to manage it well. If viewed on an SDR display, it scales the output in a reasonable way. If you connect an HDR display it will show as HDR (without needing to reload the page or restart Chrome). It handles multiple monitors well too. Independent monitors each render the best they can. If mirrored, the image is displayed as HDR on all mirrored screens (which means proper display on the HDR monitor and clipped on any SDR monitor).

[0xC0000054]

Does that mean a unique enum / reference to save space wasted with embedding common color spaces ?

Yes, it is a simpler alternative to ICC profiles used by many video codecs including HEVC (Apple's HEIC format), and AV1 (AVIF).
The libavif wiki has a page describing the CICP format and its relation to AVIF: https://github.com/AOMediaCodec/libavif/wiki/CICP

Krita appears to generate an ICC profile from the CICP values when loading HEIC/AVIF images that do not have an ICC profile.

If you embed the profile in the AVIF, Chrome will not render an HDR AVIF properly.

The command line used to encode the Netflix HDR samples always includes the --ignore-icc option when calling avifenc.
This makes sense as the color primaries and transfer characteristics in the ICC profile will take precedence over the CICP values.

[gregbenz]

Thank you for the explanation.

I don't believe the CICP value is causing the issue I'm seeing with Chrome. The embedded profile in my PNG is very unusual. Unless AVIFENC is mapping it to some unexpected value, ignores it when reverse converting the AVIF to PNG (my round-trip file is exactly the same if I keep the profile), and I'm missing CICP in the metadata when I keep the ICC profile - I believe there is probably some other reason why Chrome is failing.

Have you been able to convert an image with an embedded profile into an HDR AVIF and have it display properly in Chrome or FireFox?

[0xC0000054]

Have you been able to convert an image with an embedded profile into an HDR AVIF and have it display properly in Chrome or FireFox?

I have not tried it.
As for Firefox, its AVIF support is a work in progress, HDR is one of the features they do not yet support.

PNG does not support the CICP metadata, it only supports ICC profiles.

[gregbenz]

As for Firefox, its AVIF support is a work in progress, HDR is one of the features they do not yet support.

Thank you for confirming. I was unclear if they were using some other assumed tonal response curve against the images I've tested (pixels very close 1.0 go to HDR values, but the vast majority of pixels get pushed to near black).

[gregbenz]

FYI, I've created a page full of tests for HDR support, as well as HDR AVIF images (created with AVIFENC converted from JXL images created by ACR v15).
https://gregbenzphotography.com/hdr/

More on JXL export from ACR v15 late in this video: https://gregbenzphotography.com/photography-tips/acr-15-adds-high-dynamic-range-output/

[0xC0000054]

I recently added support for generating an ICC color profile from some of the NCLX profiles.
I am now trying to get the AVIF HDR images to load in 32-bit mode.

[gregbenz]

It is my understanding that ICC support for HDR in browsers is lacking and may never come, which makes sense given size savings and simplicity. CICP and similar alternatives sound like they will be the standard for HDR on the web. Might still need more metadata for best rendering long term, this will probably evolve a bit.

[0xC0000054]

The ICC generation support is for Photoshop, so that it gets an ICC profile when loading images that only use one of the supported NCLX profiles.
I managed to get the Netflix BT. 2020 PQ sample images loading correctly.

Krita also supports the BT. 2100 HLG and SMPTE ST 428-1 HDR color spaces, but I have not implemented that because I do not have any AVIF test images in those formats. I also could not find any non-GPL sample code for the transfer curve manipulation.

I am planning to open an issue in the av1-avif repository to ask about the correct way to handle premultiplied alpha with HDR images.
I think that the premultiplication would be done before the HDR transfer curve is removed or applied, but I am not sure about that.

[gregbenz]

Makes sense. Are you using the new ICC v4.4 CICP tag as well? Sounds like a mechanism that will be increasingly used.

If you need any HDR AVIF images for test, I have several on my site: [https://gregbenzphotography.com/hdr]
These were generated with AVIFENC using PQ CICP and I believe used Rec2020 primaries (from 16-bit HDR PNGs created by DJXL from JXL images exported from ACR v15).

Interesting, why would the alpha change HDR? Is the same TRC applied to the alpha channel as well? I'd assumed it would be encoded more likely with a linear gamma to be more even and efficient for storing opacity values, but never really dug into the details of it for any major file format.

[0xC0000054]

Interesting, why would the alpha change HDR?

The color values would change if you are using premultiplied alpha.
I opened an issue in the av1-avif repository to ask about the correct way to handle that: AOMediaCodec/av1-avif#197

Are you using the new ICC v4.4 CICP tag as well?

Yes, the generated ICC profiles include that tag.

[gregbenz]

Thanks, very interesting!

[0xC0000054]

I added support for saving 32-bit float images as either 10-bit or 12-bit.
It appears to work fairly well, but it could use more testing. I am attaching an x64 test build to this post.

Replaced by the build in #8 (comment)

Release notes

Beta version 1.0.6.2

Fixed a PQ curve scaling issue that was causing the saved image to appear too bright.

Beta version 1.0.6.1

The AVIF format used when saving 32-bit images is the Rec. 2020 color space with the Rec. 2100 PQ transfer function, CICP 9/16/9/full range.
ICC profiles are not supported for HDR images because the ICC profile overrides the HDR color space and transfer functions.

Premultiplied alpha should be supported, but it will result in the image data being clipped to the [0, 1] range. I am considering disabling premultiplied alpha for HDR images, it does not make much sense to have a setting that clips the output range.
As of this post I have not gotten a response to the GitHub issue I filed, so I am not sure if premultiplied alpha is even valid for a HDR image.

Re-saving an AVIF HDR image will cause the saved image to be over exposed, this is due to the PQ gamut adjustment that is made when loading the image.
Without the gamut adjustment the loaded image will look too dark when displayed in Photoshop.

The image is was using for testing is the Netflix HDR sample hdr_cosmos12920_cicp9-16-0_lossless.avif.
When I changed the code to remove the PQ gamut adjustment the saved image matched the original when viewed in Chrome, so that at least indicates that I am on the right track. 😄

[gregbenz]

Very exciting to see this!

I've got it running and seeing no errors when using a variety of options. However, in all cases, the exported image is much brighter than the source. My source images are encoded as 32-bit TIF files with either Adobe RGB linear or ProPhoto RGB linear colorspace.

I'm viewing the output in Chrome, which has consistently shown HDR AVIF images very well in my testing.

[0xC0000054]

However, in all cases, the exported image is much brighter than the source.

I noticed that as well. That should be fixed in beta version 1.0.6.2, I updated the download in my previous post.

I changed the PQ scaling code to preform the reverse of the transformation it uses when loading an image.
Both transformations adjust for the difference in the maximum luminance level between sRGB (80 nits) and PQ (10000 nits).
I am not sure if that behavior is correct for all images, but it appears to work fairly well.

I have also been using Chrome for HDR testing, it is one of the few programs that can correctly render AVIF HDR.
Paint.NET can render AVIF images as thumbnails in Windows Explorer, but it does not have HDR support.

[gregbenz]

Great! Is there a build for that?

Is there an easy way to query the version of the plugin other than Help / System info?

[0xC0000054]

Great! Is there a build for that?

The download in the following post: #8 (comment)

Is there an easy way to query the version of the plugin other than Help / System info?

The plugin about box can be shown by going to: Help > About Plugins > Av1 Image...
The version number is shown in that dialog.

[gregbenz]

I'm now seeing 1.0.6.2 generate expected tones.

However, the color is not correct if the source image is in a color space such as ProPhoto. Photoshop itself also struggles to convert color within 32-bit, so I'm sure this is a tricky thing to do (though possible as ACR v15 will take such a file and correctly export a JXL).

Here are examples showing how the output is fine with a Rec2020 source, but is incorrect with a ProPhoto RGB source: [https://drive.google.com/drive/folders/1dXlce3Az1f03WtRRL7Gm9E85rp__3On4?usp=share_link]

Do you have a Mac build? I could get some more people testing the plugin. And from what I've seen, it already works great for images encoded with Rec2020 primaries.

Thank you so much for all your work on this, very exciting to see!

[0xC0000054]

However, the color is not correct if the source image is in a color space such as ProPhoto. Photoshop itself also struggles to convert color within 32-bit, so I'm sure this is a tricky thing to do (though possible as ACR v15 will take such a file and correctly export a JXL).

The plugin assumes that the user converted the image to a Rec. 2020 profile before saving the image.
Krita appears to perform that conversion when saving the image, so I guess the plugin could do the conversion but the result may not match the same conversion performed within Photoshop.
I agree that 32-bit color management is tricky, when I added color management support to my G'MIC filter I noticed some significant color shifts between the output of Little CMS and Photoshop's color management engine.

Do you have a Mac build?

Unfortunately not.
As I mentioned in another issue, the plugin would have to be ported to work on macOS and I do not have the hardware or macOS development experience to do that.

[gregbenz]

If the user has a Rec2020 profile installed, you can work around this and get a decent conversion of the 32-bit image to Rec 2020 primaries. Sadly, not something that comes with PS by default.

Bummer on MacOS! Is porting (including dependencies if not covered) something you would expect to be a significant project, or not too involved for someone who has the background/hardware?

[0xC0000054]

If the user has a Rec2020 profile installed, you can work around this and get a decent conversion of the 32-bit image to Rec 2020 primaries.

I can use Little CMS to generate a Rec. 2020 linear profile in memory and use that for the conversion inside the plugin.
Unfortunately Photoshop does not provide plugins with the ability to convert the image data to an arbitrary ICC profile, the API only handles conversion to the monitor RGB profile.

Bummer on MacOS! Is porting (including dependencies if not covered) something you would expect to be a significant project, or not too involved for someone who has the background/hardware?

I would expect it to be a significant project, the save UI may have to be rewritten to work on Mac.
But after looking through the the Photoshop SDK it appears that it has a cross-platform UI toolkit, so I will have to see if I can convert the Windows dialogs to use it.
Plus someone would have to maintain the MacOS builds, I doubt that GitHub's automated build system could handle Apple's code signing requirements.

[gregbenz]

A headless mode would be very useful on Mac, I'm not sure the UI is necessary for some great uses.

Please drop me an email, want to run something by you: https://gregbenzphotography.com/contact

[0xC0000054]

A headless mode would be very useful on Mac, I'm not sure the UI is necessary for some great uses.

I do not think that a Photoshop plugin can have a headless mode without a UI, Photoshop's scripting system gets its data from the plugin UI. Unless I am misunderstanding what you are referring to.

I experimented with performing the color conversion when saving, the color shift was greater than I expected.

Edit: I was basing that assessment off the Windows Explorer AVIF thumbnails, which are generated by Paint.NET and not color managed.
The converted colors match Photoshop with a Rec. 2020 color profile,

I still am concerned that users would be surprised by a plugin changing the image color profile when saving.
But that is probably better than saving an image with the incorrect colors.

Please drop me an email, want to run something by you

I sent you an emeil, my name is on my GitHub profile.

[gregbenz]

Agreed, user should do that if it leads to non-trivial color shifts. Might be good to warn the user for unsupported use cases, though may be tricky to validate exactly what would constitute a valid profile given various names and slight variability of the primaries for profiles which would be considered "Rec2020" or "Rec 2100".

[0xC0000054]

Agreed, user should do that if it leads to non-trivial color shifts.

The color shifts I was observing were due to the images being rendered by programs that does not support color management with the CICP data, the image loads perfectly in Krita and Photoshop.

I think I will leave the automatic conversion code in the plugin, as I previously stated Krita always performs the Rec. 2020 conversion when saving PQ images.
I am currently working on extending it to convert SDR images without an embedded color profile to sRGB, which is what the default CICP values use.

Might be good to warn the user for unsupported use cases, though may be tricky to validate exactly what would constitute a valid profile given various names and slight variability of the primaries for profiles which would be considered "Rec2020" or "Rec 2100".

Detecting profiles is a difficult problem, I currently check for the CICP tag, or description and/or primaries if there is no CICP tag.
The description code uses hard coded list of Rec 2020 profile descriptions.
The primary detection code has never worked due to variations in floating point math and how the primaries are represented.

The detection is only intended as a best faith effort to avoid performing the conversion if it is unnecessary.

[gregbenz]

With the end use for AVIF being the web mostly, I'd want to test the image in Chrome to see that it looks as expected. It will often be darker in Chrome due to tonemapping vs absolute rendering of the luminance in PS, I'm just referring to the color shifts.

Sounds reasonable for missing profiles. Common issue for images from the web or scanners.

Sounds like a reasonable approach. The ICC v4.4 spec includes a CICP tag and it seems like variability should be reduced in time when the standards are more solidified. Kind of messy for HDR right now.

[0xC0000054]

Kind of messy for HDR right now.

I agree.
I recently received another reply in my HDR and Premultiplied Alpha issue regarding blending and HDR: AOMediaCodec/av1-avif#197 (comment)

I am thinking about disabling premultiplied alpha for HDR images, at least until I can get clear guidance on how it should be done.
While the method that I use in the plugin appears to work fine, there is too much risk of incompatibilities with other software.

Sounds reasonable for missing profiles.

If the user wants a SDR image to be saved with a specific embedded color profile, they have to select that option.
Otherwise, the plugin will convert the image to sRGB and tag it as such in the CICP data.
This is more consistent than the previous behavior, where the plugin would assume that any SDR image saved without an embedded color profile was sRGB.

[gregbenz]

Alpha seems a bit tricky with HDR as it is sensitive to the TRC. Moving between gamma 1.8-2.4 and linear for HDR gets messy in Photoshop. I'd think the demand for HDR cutouts is pretty limited, I don't imagine I'll export anything HDR with transparency anytime soon.

[0xC0000054]

I think I will leave the straight alpha export option as-is, I cannot think of a user-friendly way to disable it when saving HDR images. But I added a warning to the read-me about the potential compatibility issues.

Here is a new test build (v1.0.6.3):

https://github.com/0xC0000054/avif-format/releases/tag/v1.0.6.3-beta

Release notes

• Added support for saving HDR images as SMPTE 428-1
• Added a None, clip option to the HDR transfer characteristics drop down
  • This option allows a 32-bit image to be exported as 10-bit or 12-bit SDR
• Removed the bit-depth restrictions on lossless compression
  • 10-bit and 12-bit images can now be saved using lossless compression
• Generate an ICC profile from supported NCLX profiles when loading images without an embedded ICC profile
• Automatically convert the saved image to Rec2020 or sRGB if necessary
  • Rec2020 is required for HDR images saved as Rec. 2100 PQ or SMPTE 428-1
  • SDR images without an embedded color profile will be converted to sRGB and tagged as such in the CICP metadata
• Fixed a few issues with the plug-in scripting support
• Disabled premultiplied alpha for 32-bit HDR images
  • It is currently unclear how this feature would work with HDR images.

[0xC0000054]

Resolved with the release of beta version 1.0.6.3.