Please add some AVIF testfiles with matrix_coefficients == 0 #195
Comments
|
There is actually already test content using identity matrix in the repo. Here is one example: The reason why the file you attached renders differently in various places is because it's actually kind of ambiguous. It contains an ICC profile but no The latest wording in MIAF attempts to clear this up, but I don't think that version has been published yet. The gist of it is:
There is also a note saying:
I still think the wording is slightly unclear on what should happen if you only have an ICC profile. @cconcolato, can you remember what the decision was? Since the ICC profile does not override the default matrix coefficients, should the default be used rather than what might be in the elementary stream? |
The clarification would be useful. |
Well, it does actually explicitly say I still think the spec could spell out exactly what is meant to happen when you only have an ICC profile just to make it abundantly clear what the expectation is. |
|
It means that files with ICC only will be handled like matrix coefficients 5,6 But it will break lot of older files which use coefficients 1 in the bitstream. MC == 0 is not so frequent but MC == 1 is used. Those old files will be decoded with slightly different colors, people will not be happy that. |
|
Per @cconcolato 's comment here, if there is no I believe that's what was agreed to in #84 and is what I implemented in Firefox. I think the files you're referring to with ICC embedded in a |
That's not what the closing statement from @cconcolato says though:
Hmm... I'm not completely sure that's what we agreed upon in the meeting, but I'll try to dig up the meeting notes to check. HEIF and especially MIAF has from the start been trying to make sure that you should never need to reach into the video bitstream itself to figure out color stuff since that is not codec agnostic. For HEVC, it was considered ok (but not mandated) to check in the codec config box, since that is not part of the bitstream and the codec config is defined in ISOBMFF. But with AV1, this information is usually not present in the codec config (most likely since it can be described with a codec agnostic So the exact behaviour for AVIF files with only an ICC profile with the old wording in MIAF is actually undefined. Which @novomesk actually noted here: #84 (comment) So the example file here is relying on undefined behaviour. Like in C, anything might happen when you do that. :) |
|
Writing two boxes (ICC+NCLX) helps in iOS 16.0 to get expected rendering. It doesn't help for the Windows extension yet. I also observed that AVIF with 12bit depth is not displayed on iOS 16.0. I think web developers need to know what AVIF features they can use (or what to avoid) so that their files work across the browsers. Android 12 used to support less AVIF files than Google Chrome on the same phone. |
Why is that? I agree it's more convenient and consistent to get it from the Why should we try so hard to avoid getting this data from the bitstream? To me, it seems most likely to be accurate to how the image was encoded, and by the time a renderer is applying the MC, the bitstream has already been decoded, right? I'm all for codec agnosticism, but something at this point has to be aware of the codec type just to call the right decode function. I could get behind the idea that this case is too ambiguous and that it's a "shall not", so readers are encouraged to reject the input, but I find it highly unlikely that we're going to see this handled consistently in practice if the standard says to ignore the bitstream's explicit signal in favor of the implied default from the container. As for putting it in the codec config, do you mean in the |
Given the incredibly wide variety of features possible with HEIF, it may be interesting to devise some sort of syntax to communicate this information via additional parameters on the MIME type, right? That should probably be a whole separate issue/discussion, though. |
YCbCr <-> RGB conversion is typically not handled by the codec but rather at the container level, in the same way that color primaries and transfer function typically is not really applied or used by the codec. I would guess the reason it's possible to signal it at the codec level is twofold:
Given that this is happening at the container level rather than at the codec level, I would argue that it's actually more likely that the container level has the correct values and that whatever is in the bitstream is incorrect. But all of this is actually kind of beside the point. :) It would be fine to check the bitstream for CICP if the spec mandated that this is the required behaviour. But the spec doesn't (or rather didn't) say anything about it at all, which means the behaviour is undefined. If we want to make the behaviour defined, we have two choices:
NOTE: Option 1 may sound perfectly reasonable for a simple single-item file. But as soon as you introduce a simple grid item you have problems. MIAF added restrictions on grid items to allow easier and more performant parsing and rendering. The idea was to make sure that decoding could be done into a single canvas buffer of the native format of the bitstream. Why is this important?
So how does MIAF add restrictions to make sure that all tiles can be decoded into a shared canvas?
For all other codecs than AV1, this is enough. But the AV1 codec config does not specify matrix coefficients or full/video-range. So I can create a file like the following:
Given that items 1 and 2 share How do we know that they have the same matrix coefficients and range? We don't. We can't decode into a shared canvas unless we first check the bitstreams for all the tiles. This means we can't do streamed decoding (typically pretty important on the web) since we need to wait until we have the final tile. Things only get more complicated as you start to create more complex files. Hence why I filed this issue: #194 How does this compare with mandating that the container level explicitly or implicitly overrides whatever is in the bitstream?
And you probably won't either. :) |
|
We can of course play devil's advocate and say that the number of files out there that contain a I'm fine with that, but it then needs to actually be specified in the spec that this is the required behaviour. Which needs the following to happen:
|
|
@leo-barnes When you have time, do you know why this animation doesn't work in Safari on my iPhone8 with iOS 16.1? I saw some news that animated AVIF should be supported. |
@novomesk
|
|
Thanks! So we will wait till the next update. |
This comment was marked as off-topic.
This comment was marked as off-topic.
1 similar comment
This comment was marked as off-topic.
This comment was marked as off-topic.
Overview of Changes from GIMP 2.99.12 to GIMP 2.99.14
=====================================================
Core:
- The download button in About dialog when a new version is available
will now show the development download page when running unstable
branch code.
- The update check on macOS now uses native HTTPS-able API, so that we
don't have to wait for GIO to have HTTPS modules for macOS.
- The main process is now run as a GimpApp which is a new class derived from
GtkApplication. The main process of `gimp-console` on the other hand is a
`GimpConsoleApp` which is derived from GApplication. Both new classes share
a same GimpCoreApp interface. This is a main step for the GTK+3 port.
- Various improvements on awareness of multi-item selection across core
features. A notable fix is the preview when transforming multiple layers at
once (with various transform tools). Various actions are now multi-drawable
aware as well.
- New "Vectors Structure" in the XCF format: XCF files (format bumped to
version 18) can now store and load all the usual and common properties of
other items. In other words, it makes XCF now able to store locks, color
tags and several selected paths.
- XCF saving with RLE and zlib encoding are now multi-threaded and therefore
much faster in most cases.
- Pasting an image now creates a new layer by default (not a Floating Layer
anymore). The only 3 cases where we still have floating items are:
* when pasting into a layer mask;
* when doing quick copy/cut paste on-canvas with the Alt modifiers;
* when floating layers explicitly with the "Float" action.
- Copy-paste code was deeply reviewed and re-specified in the light of
multi-item selection; it's still a WIP:
* When pasting several drawables, we currently paste them over the top
selected layer (visually in Layers dockable).
* Pasted data position was rewritten, based on existing logic, but taking
into account the multiple selected items.
* Pasting a selected area from multiple layers still creates multiple
layers, not merged pixel contents as a single layer.
* New layers created when copying from a selection are consistently the
offset and dimensions of the bounding box of the dimension.
* When a layer and one of its group layer parent are selected, it is
equivalent to have only the child layer selected.
- 2 new actions were added: "Paste as Single Layer" and "Paste as Single Layer
in Place" under the "Paste as" submenu of Edit menu. These paste the copied
layers as a single merged layer, instead of as several layers (as "Paste"
and "Paste in Place" do).
Graphical User Interface:
- New "Gray" theme based on a 18.42% luminance middle-gray background, which
should be a good neutral environment for color work.
- The foreground/background editor in the toolbox will now take into account
the toolbox icon size and resize itself accordingly (live, as you change
theme). This allows to have really narrow toolbox when you use small icons.
- Theme-override icon size selection in Preferences > Themes: this allows to
override theme-set icon sizes, with a global concept of small, medium, large
and huge. The following widgets are so far modified: toolbox icons, fg/bg
editor in toolbox, fg/bg editor in Colors dockable, dockables tab icons,
bottom buttons (in the button box) of dockables, header eye and lock icons
above item trees, and eye and lock icon switches in item tree cells.
- Symmetry dockable contents is now shown, yet deactivated, when no images are
opened, improving discoverability.
- Reworked the "Convert to * Working Space?" dialog into a "Keep the Embedded
Working Space?" one. Keeping an image working space is now the recommended
and default action. "Convert" became an explicit action requiring to click
(neither mapped to Enter nor Escape keys).
- "Floating Selection" renamed to "Floating Layer" or "Floating Mask"
depending on the type of item it applies to.
- "Floating Masks" are now drawn above the layer mask in the Layers dockable,
making the fact that they would anchor to the below layer mask (not the
layer) much more obvious.
- "Paste into Selection" and "Paste into Selection in Place" were moved under
the "Paste as" submenu of Edit menu.
Tools:
- Text tool: new "Outlined" and "Outlined and filled" options, with various
sub-options to choose the outline style, color, pattern, width, cap and join
styles, miter limit, anti-aliasing and dash pattern.
- Align tool:
* now multi-item aware, it is much more usable than it used to be when we
had to click on canvas to select items.
* On-canvas clicks are now only needed to select guides (Alt or Alt-Shift
click and selected guide colors change) or for the reference object
(normal click).
* Also the reference object gets on-canvas handles and the name is written
in the dockable, making it obvious if you selected the right reference or
not.
* Moreover the selected reference will now loop when layers are stacked on
each other, which allow to select a bottom layer, even if there are layers
above it everywhere.
* New option "Use extents of layer contents" to Align tool: this is similar
to first run "Crop to Content" on every layer to align or distribute
(without actually cropping the layers).
* Fine-grained align/distribute button sensitivity to make it more obvious
when an action would not make any change anyway.
* New anchor point setting (pivot widget) to choose which part of the target
items will be aligned or distributed.
* Get rid of various broken distribution actions.
* Distribution actions don't move the 2 extreme (top/bottom or left/right
depending on distribution direction) targets, but distribute all other
targets within their range. It is more consistent with how it works in
other software.
* Adding 2 "Distribute with evenly (horizontal|vertical) gaps" actions,
which distribute by keeping a common gap between objects instead of
between anchor points.
* Offset settings have been removed.
- Transform tools are now auto-activated on selection (and when switching
images or item selection).
Plug-ins:
- PDF:
* Export code was ported to GimpProcedureDialog.
* New "root-layers-only" argument to "file-pdf-save", which comes
with a checkbox in the export dialog to allow exporting as pages
the root layers only. The main usage is to organize your pages'
contents in layer groups.
- AVIF:
* RGB AVIF compatibility with Safari on iOS 16.0: Some AVIF images are
rendered differently in Apple's implementation compared to implementations
of Google and Mozilla. See: AOMediaCodec/av1-avif#195
This changes requires libheif 1.10.0 though the plug-in can still build
with older libheif.
- PSD:
* export of CMYK(A) files added, with 8 or 16-bit precision per channel,
using a CMYK soft-proof profile for conversion.
* Paths are now exported with PSD files.
- JPEG-XL:
* Metadata import/export now supported (requires libjxl 0.7.0).
- Python-Console:
* sys.stdout.flush() implemented as a no-op inside the console, to be able
to easily copy-paste code, or using libraries which flush the output.
- ICNS:
* Initial support for loading and exporting.
- TIFF:
* New toggle to optionally load reduced pages. We keep a heuristic to try
and guess whether these are thumbnails (single reduced image in the second
position), but it's only used to decide whether the option is checked by
default or not. It is now up to anyone to decide or not whether they want
to load these reduced images.
API:
- Changes in libgimp:
* Abstract method get_window() of GimpProgressVtable had its signature
changed. The window ID is now a guint64.
* New functions:
+ gimp_text_layer_set_markup()
+ gimp_image_get_selected_channels()
+ gimp_image_get_selected_vectors()
+ gimp_image_list_selected_channels()
+ gimp_image_list_selected_vectors()
+ gimp_image_set_selected_channels()
+ gimp_image_set_selected_vectors()
+ gimp_image_take_selected_channels()
+ gimp_image_take_selected_vectors()
+ gimp_image_list_selected_drawables()
* Updated functions:
+ gimp_vectors_stroke_translate() now uses offsets in double type.
* New classes:
+ GimpTextLayer: child class of GimpLayer.
- Changes in libgimpwidgets:
* Updated widgets:
+ GimpPickButton now has a specific implementation for Windows. In
particular it improves color picking with multi-monitor and scales
different than 100%.
Build:
- meson requirement bump to meson 0.56.0.
- Many fixes to the meson build scripts, making it closer to be our
official build for GIMP 3.0.
- The CI now generates a tarball containing the GIMP references,
generated by gi-docgen and g-ir-doc.
- Improved Clang 15.0.0 support.
- "win*-nightly" jobs were added back and are now more efficient with the
--output-dll-list option.
- babl requirement bumped to babl 0.1.98.
- GEGL requirement bumped to GEGL 0.4.40.
- GIMP macOS builds (gimp-macos-build repository) was moved to using MacPorts
in order to take advantage of a bigger community to maintain our
dependencies.
- GIMP now has an Apple Silicon build.
…is a consistent image failure https://bugs.webkit.org/show_bug.cgi?id=248544 rdar://102823196 Reviewed by Simon Fraser. libavif does not render fast/images/resources/green-313x313.avif and fast/images/resources/green-400x400.avif correctly. See AOMediaCodec/av1-avif#195. We need to fix the expectation file fast/images/avif-as-image-expected.html with the correct rendering; i.e. the image should be lime not green. This will require marking this test [ ImageOnlyFailure ] for non Apple ports and macOS down-level ports. * LayoutTests/TestExpectations: * LayoutTests/fast/images/avif-as-image-expected.html: * LayoutTests/platform/ios/TestExpectations: * LayoutTests/platform/mac-ventura/TestExpectations: * LayoutTests/platform/mac/TestExpectations: Canonical link: https://commits.webkit.org/257738@main
|
This issue asks for test content that uses identity matrix. Such a file already exists: I think this issue can be closed. Please reopen if you think this issue covers something that I missed! |
I think it would be useful to have testfiles with zero MatrixCoefficients, so that various AVIF implementations can test before release.
For example following file is rendered differently in various applications, because they perform unnecessary YUV->RGB conversion:
avif-rgb-8bit.zip
Safari on iPhone (iOS 16.0):
AV1 Extension on Windows:
BTW, there is some weird stripe on the right side.
It looks OK in the following apps (background color may be different but that's expected):
Opera browser:
qimgv viewer:
GIMP:
The text was updated successfully, but these errors were encountered: