-
Notifications
You must be signed in to change notification settings - Fork 724
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement support for JPEG XL image format #3865
Comments
Very maybe. I like some things about the format, but I'd want to figure out if there's a patent problems, and if browsers actually turn Jpeg XL on by default. |
Thanks for looking in to it! Something like AVIF (#2977) might be similar or better in some cases then JPEG XL in lossy formats but AVIF seems to be really bad ( |
I'm somewhat agree that maybe in loseless JXL is better than AVIF but it has one huge problem - JXL have very slow encoding speed and since it don't have tools like avifenc/avifdec (that can fully load CPU) compiled with very fast rav1e+dav1d codecs that take a few seconds (against about a minute for JXL) to encode/decode AVIF files JXL wouldn't get popular like AVIF. |
I don't think these are very relevent reasons for/not to include support for these formats in renpy. For example FLAC is very tedious to manipulate and to edit, yet we support it because it's relatively easy for us to do so. Same for the WEBP format, which Windows doesn't even natively recognize (without bugs).
Slowness of encoding can be an obstacle for point 4) to happen, but I don't think it will be an obstacle for supporting it in renpy. |
Actually I've never said that JXL should not be in Ren'Py... Btw I've tested it with most recent build (JPEG XL encoder v0.8.0 bb96fb0) and speed was good (both loseless and lossy) and in loseless mode JXL beat AVIF by speed (because in loseless mode AVIF uses slow aom encoder) and by size (actually very big difference). |
Yeah JXL is not slow at all but a lot faster and uses multiple threads when encoding. libjxl is a thing (https://github.com/libjxl/libjxl) but I don't think it's relevant since SDL_image probably handles the decoding of images. Some benchmarks: https://twitter.com/jonsneyers/status/1346389917816008704 |
Decoder is open source and licensed under BSD-3 (https://github.com/libjxl/libjxl) and also as mentioned above in the original post jpeg xl is "free, open source and royalty-free". The patent on libjxl is listed here https://github.com/libjxl/libjxl/blob/main/PATENTS but it seems very permissive. As for support as I mentioned above, Chrome, Edge and Firefox do support the format but it needs to be enabled in settings and as for Firefox you also need a nightly build of Firefox. Even if jpeg xl isn't enabled right now on browsers and maybe doesn't exist yet on android I think it's still nice to have on PCs and Macs. PC and Mac ports are seperate from android / web so you could always use jpeg xl for PC / Mac and webps for Android / Web but I suspect support for web will be enabled soon. |
Some of libjxl's dependencies are However, if WOFF2 support is to be enabled also in freetype, it may be better for the build process to use own |
+1 to WOTF Support being enabled in freetype aswell |
I'm somewhat leaning towards AVIF over JXL, due to AVIF being in all the browsers, and JXL being pulled from chrome. |
Unsure why that should matter here? |
There's also the size of the Ren'Py download to think about - which matters a lot for the web version and smaller games. I like JXL quite a bit, but I don't really want to have that download size unless the format gets tool support. |
I think that is a bad idea. For multiple reasons but the most relevant being:
I believe So I ask you, @renpytom. Please reconsider. |
If there are concerns about download size, I believe JXL+JXL decoder will have much smaller size than AVIF+AVIF decoder. But since when downloading, the executable is compressed, so it may not be a big of a difference. |
What makes you think this, given the experience with, say, JPEG2000 - a format that wasn't supported by browsers, that never really went anywhere? Compare https://caniuse.com/avif vs https://caniuse.com/jpegxl I like Jpeg XL, quite a bit, but I have trouble seeing it taking off. (I'll re-evaluate this if it does.) |
That's what I know related to |
https://photo.stackexchange.com/a/11714/62857 Regardless anyway, I see now how using the built-in capabilities would be pretty darn neat for renpyweb. |
Sure, but that's a feature argument. As far as the popularity argument goes, it's only a promise, that JXL has the potential of booming widely. |
I just hope not too many think like that... Otherwise, it will be a large sum waiting for themselves... However, I do understand that stance very well. If I were to make renpy... I think I'd be thinking about Pillow supporting it. |
A couple of ideas, again from a very detached prospective:
There are actually two somewhat exclusive features that only each core has though:
Perhaps there are other nitty-gritty quirks that I have missed (perhaps some peculiar difference in HDR, CMYK or whatnot) but AFAICT these seem the elephants in the room. |
I'm back leaning towards including JXL - there's apparently a patent issue with the AVIF container that might be problematic. |
Even if I'd like to see |
There might be a patent problem because AVIF is a format that puts an AV1 bitstream into a HEIF container and HEIF seems to be patented by Nokia. I am not a lawyer but I assume this could probably cause issues. As for JXL even if it does not get web support, it can still be used on desktop and I assume on android if you deliver a native decoder. In my game I deliver lossless images for desktop and those aren't an option on AVIF since lossless images on AVIF seem to take more space then even pngs. renpytom also mentioned size as a concern, AVIF's libraries are much larger then JPEGXL's are |
Size wouldn't be a problem for AVIF, because desktops don't care about it and browsers already support it out of the box regardless. And anyway, as long as we assume you'd want to support AV1, I think 95% of the weight would be already there. On the other hand I'm really appalled to learn that HEIF is potentially patent encumbered (it doesn't seem like Nokia has ever tried to assert them, and it would be really fucking nuts if netflix had cornered themselves in this amateurish way, but yet there's no explicit green light). |
One benefit for jxl is for ongoing project already using jpg (and not keeping lossless backups, which is foolish but a lot of users are not technical), it offers lossless transcoding, so they could gain possible space savings essentially "for free" without degrading quality. Also, I do think that supporting AVIF through ffmpeg rather than SDL image, might help keep the binary size down, when AV1 support is added. |
What does it matter? Its an option, people that would select it can sort out where they want to open it. It literally halves the data needed for lossless imaging and every major photo editor can open it. Not to mention the lossy advantages. It should be added. You have bmp and tiff as options for gods sake, is this 1998 |
I ran some tests this morning and damn the benefits of JpegXL over Webp are strong, even for desktops alone. For years I used using Webp lossless, no less than that because Webp even when set at 99% lossy (the next maximum in-line quality setting after lossless), you can see a loss a deterioration in the vibrant color (especially for the red) and a very tiny bit of smearing in certain areas. It made me stay away from 99% webp lossy compression except for android because the screen is tiny and you couldn't tell the difference. Here is the test result in size, using xnconvert: Now the biggest game changer and what the size alone can't tell, is that jpegXL lossy 99% is subjectively an identical image to EDIT: Didn't run an intensive test but Avif seems to be just as good in terms of quality/size at lossy 99%, any of these two seems to be a big improvement over Webp in this scenario. |
Just for the very records (and hoping this wasn't also the reason for everybody else in here to nominate lossless compression)... And it's only WebP (being derived from a lousy video codec) that requires a specialized hackjob of a profile to toggle it. |
I wasn't aware, however color is only one part of the equation, what about the smearing on some of the edges, not sure if that is easily fixable, I could send you an example if you have an email and show you how much an image can deteriorate even when set at 99% lossy using webp versus lossless. Which doesn't happen with Avif or Jpegxl. |
With some formats, you don't need to get a licence if the tool you are using already has it. Probably a good choice to check first, though. From some short research, it looks like Nokia owns a Java Decoder for decoding HEIF containers, not sure if they actually own the format itself. |
Yes. I know, I understand and I believe it. Then, yes, transparency is important and only jxl or avif (which supports lossy 444 with the high profile) could solve every problem. |
So, spent most of the weekend trying hard to add Jpeg-XL support to Ren'Py. I'm now fairly convinced that this isn't going to happen until there's a JXL decoder that is written using standard C. The current libjxl implementation is written in modern C++, and trying to get modern C++ cross-compiled to the 14 or so combinations of operating system and CPU that Ren'Py supports... it just was not happening. This is made a bit worse by the way we target older versions of many of these platforms, where the sysroots that are present don't support features used by modern C++, and by the way that Ren'Py statically links many of its dependencies to make this possible. My current thinking is that I'd like to include Jpeg-XL decoding in Ren'Py. I plan to do it when there is a C library that can easily be compiled to the platforms Ren'Py supports at the time. https://github.com/lifthrasiir/j40 Looks like one candidate for such a library, though it currently seems to be missing enough features at the moment that we couldn't deploy it. If anyone is aware of others, please let me know. |
One thing that could make it easier to target multiple platforms is to compile e.g. libjxl to WASM using wasi-sdk, then use wasm2c to convert the WASM to C. From there, it should be easier to use the exports provided to make use of the library. Using this same method, it could also make it easier to integrate libraries that are written in different languages or have difficult build systems, such as Inochi2D. |
C++11 doesn't exactly seem all this fancy in the year of the lord 2023 (like, I think even the PSP and PS2 toolchains have some degree of support for C++20 by now). |
I'm suspecting I will add avif, as all the browsers seem to have support, which means that there's unlikely to be a major patent problem. (And big pockets to defend against it.) Plus, I experimentally added avif last night, and it built without problems, which is a good sign. The issue isn't really C++11 vs C++20, it's more going from no C++ to some. My current plan with JXL is to wait for non-reference implementations to appear. (That's probably a good idea in any case, as it's a sign that the format is being adopted.) |
A bit saddening to see what seems to be a slightly better format in Jxl being robbed away by what looks like to be a sort of lobbying from google... But overall Avif is still a pretty big improvement over Webp and from a non-political pov, it makes sense for Ren'py to support Avif since that means you can actually convert your images in the same format and have them working across all platforms I assume. |
Anybody already tried renpy/renpy-build@728013d to build/use avif with current renpy-build? |
I wouldn't consider this to be a malicious intent, when there are other valid reasons to explain this. Jpeg XL compression yields the smallest file sizes, however, encoding and decoding are the slowest of all formats on every architecture. https://storage.googleapis.com/avif-comparison/index.html https://storage.googleapis.com/avif-comparison/decode-timing.html The use of the AVIF format in Ren'py is a better choice, especially if it's faster and more widely supported. I think it would be confusing if certain Ren'py features only worked with certain file formats, and only on specific browsers. If the format becomes more widely adopted, things may of course change. |
Google's tests are done wrongly. Not done in equivalent comparison position and using color gamuts that JXL wasn't properly optimized for yet (they focused on human perception vs machine perception). They used the worst jxl can deal with and then compared what AVIF deals acceptably with. |
Yeah, tests were wrong and using a version of Jxl that had problems, and for having tested both encoding speed, AVIF is significantly slower for roughly the same quality at the present time (but hopefully will improve quite a bit over time). |
Sorry but this is false. The linked test results are from December 12 (updated December 14), using the newest version of Jpeg XL (Version: 0.7-base-43) as outlined in the verification process. |
You can check the details of why this is a controversial result, by reading the comments (at the end for the most recent ones): In any case for ren'py use case, encoding speed atm is faster on jpegXL. Though can't say for sure about decoding speed, but should be fast enough for a visual novel in any case I think. |
Guys, please. Focus. |
Again, the problem with JXL was the difficulty of getting the reference implementation running on the many platforms Ren'Py supports, without massively rearranging the build to have dependencies on C++ runtimes that aren't needed anywhere else. |
as an update, there are currently, as far as I know two active jxl decoders, a native rust decoder jxl-oxide is a rust based one that has today hit v0.1.0 and was published, and a java one jxlatte of which I am unsure of it's current status. jxl-oxide does pass a good amount of the conformance test suite (it doesnt seem to be tested against all of it), and performance at least on my desktop isn't bad. I have no idea how well these may or may not integrate but if either of these looks like it might work for you, maybe worth keeping an eye on. |
could you use wasm2c/RLBox to get the reference implementation of JXL running? (disclaimer: wasm2c contributor here) |
JPEG XL is a new image format made by the "JPEG" committee intended as a replacement for jpeg. It support both lossless and lossy compression while having a smaller size. This would be awesome for people like me who like to use lossless images in their VN for the best quality possible and with JPEG XL we will have lower sizes too.
About JPEG XL:
About it's compression:
And as for it's support all major browsers support it but it's not yet enabled by default. Many image viewers and picture editors already do support viewing JPEG XLs.
JPEG XL has also been added in to SDL_image: libsdl-org/SDL_image@742cd13
Some links:
The text was updated successfully, but these errors were encountered: