Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement support for JPEG XL image format #3865

Open
thelordofcorruption opened this issue Aug 12, 2022 · 46 comments
Open

Implement support for JPEG XL image format #3865

thelordofcorruption opened this issue Aug 12, 2022 · 46 comments
Labels
duration - day Resolving this is estimated to a day of development enhancement A proposed or requested enhancement or improvement to renpy's features. long term

Comments

@thelordofcorruption
Copy link

thelordofcorruption commented Aug 12, 2022

JPEG XL is a new image format made by the "JPEG" committee intended as a replacement for jpeg. It support both lossless and lossy compression while having a smaller size. This would be awesome for people like me who like to use lossless images in their VN for the best quality possible and with JPEG XL we will have lower sizes too.

About JPEG XL:

  • free, open source and royalty-free
  • bitstream was frozen a year ago, December 2020
  • standard formally approved 17.December.2021 twitter
  • beats every other standardized format in compression and features (HDR, progressive decoding, precision, animation, channels, speed), only AVIF wins in compression for very low quality, low fidelity images
  • old JPEGs can be encoded to 20% smaller JPEG XLs lossless and 100% restored to old JPEGs again
  • extension is .jxl

About it's compression:

  • Lossy: about 60% smaller than JPEG for same visual quality
  • Lossless JPEG transcoding: about 20% smaller
  • Lossless: about 35% smaller than PNG (50% smaller for HDR)

And as for it's support all major browsers support it but it's not yet enabled by default. Many image viewers and picture editors already do support viewing JPEG XLs.

JPEG XL has also been added in to SDL_image: libsdl-org/SDL_image@742cd13

Some links:

@renpytom
Copy link
Member

Very maybe. I like some things about the format, but I'd want to figure out if there's a patent problems, and if browsers actually turn Jpeg XL on by default.

@thelordofcorruption
Copy link
Author

thelordofcorruption commented Aug 13, 2022

Thanks for looking in to it! Something like AVIF (#2977) might be similar or better in some cases then JPEG XL in lossy formats but AVIF seems to be really bad (
AOMediaCodec/av1-avif#111) when it comes to any lossless compression producing images that are sometimes larger then PNGs and not being true lossless since it needs to convert RGB to YUV so I think JPEG XL would be a good format for both lossless and lossy and worst case scenario it will be better for lossless.

@renpytom renpytom reopened this Aug 13, 2022
@suphamster
Copy link

Thanks for looking in to it! Something like AVIF (#2977) might be similar or better in some cases then JPEG XL in lossy formats but AVIF seems to be really bad ( AOMediaCodec/av1-avif#111) when it comes to any lossless compression producing images that are sometimes larger then PNGs and not being true lossless since it needs to convert RGB to YUV so I think JPEG XL would be a good format for both lossless and lossy and worst case scenario it will be better for lossless.

I'm somewhat agree that maybe in loseless JXL is better than AVIF but it has one huge problem - JXL have very slow encoding speed and since it don't have tools like avifenc/avifdec (that can fully load CPU) compiled with very fast rav1e+dav1d codecs that take a few seconds (against about a minute for JXL) to encode/decode AVIF files JXL wouldn't get popular like AVIF.

@Gouvernathor
Copy link
Member

Gouvernathor commented Aug 13, 2022

I don't think these are very relevent reasons for/not to include support for these formats in renpy. For example FLAC is very tedious to manipulate and to edit, yet we support it because it's relatively easy for us to do so. Same for the WEBP format, which Windows doesn't even natively recognize (without bugs).
I think the major criterions would be

  1. technical support on the various platforms renpy needs to run on (incl. android and web browsers)
  2. patent and license issues with the decoder codec
  3. speed and performance when decoding the image (not encoding it)
    and finally
  4. creator use of the new formats

Slowness of encoding can be an obstacle for point 4) to happen, but I don't think it will be an obstacle for supporting it in renpy.

@suphamster
Copy link

I don't think these are very relevent reasons for/not to include support for these formats in renpy. For example FLAC is very tedious to manipulate and to edit, yet we support it because it's relatively easy for us to do so. Same for the WEBP format, which Windows doesn't even natively recognize (without bugs). I think the major criterions would be

  1. technical support on the various platforms renpy needs to run on (incl. android and web browsers)
  2. patent and license issues with the decoder codec
  3. speed and performance when decoding the image (not encoding it)
    and finally
  4. creator use of the new formats

Slowness of encoding can be an obstacle for point 4) to happen, but I don't think it will be an obstacle for supporting it in renpy.

Actually I've never said that JXL should not be in Ren'Py... Btw I've tested it with most recent build (JPEG XL encoder v0.8.0 bb96fb0) and speed was good (both loseless and lossy) and in loseless mode JXL beat AVIF by speed (because in loseless mode AVIF uses slow aom encoder) and by size (actually very big difference).

@thelordofcorruption
Copy link
Author

thelordofcorruption commented Aug 13, 2022

I don't think these are very relevent reasons for/not to include support for these formats in renpy. For example FLAC is very tedious to manipulate and to edit, yet we support it because it's relatively easy for us to do so. Same for the WEBP format, which Windows doesn't even natively recognize (without bugs). I think the major criterions would be

  1. technical support on the various platforms renpy needs to run on (incl. android and web browsers)
  2. patent and license issues with the decoder codec
  3. speed and performance when decoding the image (not encoding it)
    and finally
  4. creator use of the new formats

Slowness of encoding can be an obstacle for point 4) to happen, but I don't think it will be an obstacle for supporting it in renpy.

Actually I've never said that JXL should not be in Ren'Py... Btw I've tested it with most recent build (JPEG XL encoder v0.8.0 bb96fb0) and speed was good (both loseless and lossy) and in loseless mode JXL beat AVIF by speed (because in loseless mode AVIF uses slow aom encoder) and by size (actually very big difference).

Yeah JXL is not slow at all but a lot faster and uses multiple threads when encoding. libjxl is a thing (https://github.com/libjxl/libjxl) but I don't think it's relevant since SDL_image probably handles the decoding of images.

Some benchmarks: https://twitter.com/jonsneyers/status/1346389917816008704

@thelordofcorruption
Copy link
Author

thelordofcorruption commented Aug 13, 2022

I don't think these are very relevent reasons for/not to include support for these formats in renpy. For example FLAC is very tedious to manipulate and to edit, yet we support it because it's relatively easy for us to do so. Same for the WEBP format, which Windows doesn't even natively recognize (without bugs). I think the major criterions would be

  1. technical support on the various platforms renpy needs to run on (incl. android and web browsers)
  2. patent and license issues with the decoder codec
  3. speed and performance when decoding the image (not encoding it)
    and finally
  4. creator use of the new formats

Slowness of encoding can be an obstacle for point 4) to happen, but I don't think it will be an obstacle for supporting it in renpy.

Actually I've never said that JXL should not be in Ren'Py... Btw I've tested it with most recent build (JPEG XL encoder v0.8.0 bb96fb0) and speed was good (both loseless and lossy) and in loseless mode JXL beat AVIF by speed (because in loseless mode AVIF uses slow aom encoder) and by size (actually very big difference).

Decoder is open source and licensed under BSD-3 (https://github.com/libjxl/libjxl) and also as mentioned above in the original post jpeg xl is "free, open source and royalty-free". The patent on libjxl is listed here https://github.com/libjxl/libjxl/blob/main/PATENTS but it seems very permissive.

As for support as I mentioned above, Chrome, Edge and Firefox do support the format but it needs to be enabled in settings and as for Firefox you also need a nightly build of Firefox.
As for Android, I've not been able to find much info about jpeg xl support.

Even if jpeg xl isn't enabled right now on browsers and maybe doesn't exist yet on android I think it's still nice to have on PCs and Macs. PC and Mac ports are seperate from android / web so you could always use jpeg xl for PC / Mac and webps for Android / Web but I suspect support for web will be enabled soon.

@uyjulian
Copy link
Contributor

uyjulian commented Aug 13, 2022

Some of libjxl's dependencies are brotli, and highway. These are vendored in libjxl, so if libjxl is to be added only, then using the vendored dependencies should be fine.

However, if WOFF2 support is to be enabled also in freetype, it may be better for the build process to use own brotli.

@renpytom renpytom changed the title [Feature Request] Implement support for JPEG XL image format Implement support for JPEG XL image format Sep 2, 2022
@Daasin
Copy link

Daasin commented Dec 7, 2022

+1 to WOTF Support being enabled in freetype aswell

@renpytom
Copy link
Member

I'm somewhat leaning towards AVIF over JXL, due to AVIF being in all the browsers, and JXL being pulled from chrome.

@mirh
Copy link

mirh commented Dec 25, 2022

Unsure why that should matter here?
By all means, if one format is more or less strictly better than the other, there's no point in having both.. but otherwise it doesn't seem so automatic.

@renpytom
Copy link
Member

There's also the size of the Ren'Py download to think about - which matters a lot for the web version and smaller games. I like JXL quite a bit, but I don't really want to have that download size unless the format gets tool support.

@brunoais
Copy link

brunoais commented Dec 25, 2022

I'm somewhat leaning towards AVIF over JXL, due to AVIF being in all the browsers, and JXL being pulled from chrome.

I think that is a bad idea. For multiple reasons but the most relevant being:

  1. You are giving legitimacy to the chromium people that there's no interest in the industry for JXL.
  2. The size improvements rivals AVIF hardly (I tested them myself) (in comparative visual quality)
    1. In size, AVIF excels when getting an image still from a video encoded as AVIF making it excellent for video previews and thumbnails
    2. JXL excels at everything else (including lossless)
  3. JXL has rivals decoding performance, when tuned for better decoding.
    1. Note that AVIF implementations are started 6 years old and they have ~5 years of optimization
    2. JXL is 2 years old with only ~1 year of optimizations.
    3. (You probably know this but I'll share anyway): Optimizations are usually much more effective in the first 3-5 years than later, when (with exceptions) only small improvements are left to explore.
    4. JXL is still badly optimized. One of the lead devs even has good ideas on how to improve its performance tremendously. I'll find his post sometime. I forgot where he wrote it.
  4. People with significant roles do support JXL is faster to encode and decode. Also, a WASM decoder for JXL can be fit into 290KB gzip
  5. Lots of notable image editors are adopting it! Krita, GIMP Paint.net!
  6. People are voting in favor of having this answering chromium. Heavily! 819 votes now. One of the fastest growing on the bug tracker
  7. Waterfox supports JXL
  8. Pale moon supports JXL
  9. There are many features JXL supports that AVIF doesn't (or not nearly as well)
    1. Progressive decoding (just load less data to have lower quality image)
    2. "Focused" progressive decoding (E.g. first decode the faces depicted in the image)
    3. Data loss looks terrible with AVIF while being acceptable with JXL (due to how data is organized)
      1. Note AVIF is pretty much just a keyframe of a video.
    4. ...
  10. AVIF creates very bad heavily compressed images. It requires more data than JXL to do low quality images in a human pleasant way
  11. JXL supports many more colors than AVIF

I believe JXL is the next format for the forseeable future. Should last as long as png lasted so far!

So I ask you, @renpytom. Please reconsider.

@uyjulian
Copy link
Contributor

If there are concerns about download size, I believe JXL+JXL decoder will have much smaller size than AVIF+AVIF decoder.

But since when downloading, the executable is compressed, so it may not be a big of a difference.
So including both AVIF and JXL decoders is not that much of a problem.

@renpytom
Copy link
Member

I believe JXL is the next format for the forseeable future. Should last as long as png lasted so far!

What makes you think this, given the experience with, say, JPEG2000 - a format that wasn't supported by browsers, that never really went anywhere?

Compare https://caniuse.com/avif vs https://caniuse.com/jpegxl

I like Jpeg XL, quite a bit, but I have trouble seeing it taking off. (I'll re-evaluate this if it does.)

@brunoais
Copy link

brunoais commented Dec 25, 2022

What makes you think this, given the experience with, say, JPEG2000 - a format that wasn't supported by browsers, that never really went anywhere?

  1. J2k is in patent hell. Is it patented? Is it not? No one wants to risk. JXL has none of that.
  2. JXL recompresses even better than J2K for JPEG and JXL surpasses PNG for lossless compression
    1. based on cloudinary tests
    2. Based on my tests, there are manual tricks that can be used with PNG which allows it to compress well for <=16 colors, however, for a brain-dead process, JXL usually wins.
  3. J2K requires (by comparison and context) a huge amount of computation power to encode and decode a J2K image. J2k pretty much needs a GPU for a good performance
    1. Nowadays JXL performance already surpasses J2K's while there's still a LOT of performance improvements already being looked into and others not tagged as such.

That's what I know related to J2K for now.

@mirh
Copy link

mirh commented Dec 25, 2022

https://photo.stackexchange.com/a/11714/62857
JPEG2000 never went anywhere because it was far too complex for the few improvements that it added, and then its legal status had always been murky until at least a few years ago.
The fact that JXL is already shipping in every desktop browser should be already a big sign that things are can be different this time.

Regardless anyway, I see now how using the built-in capabilities would be pretty darn neat for renpyweb.
But if the alternative decoder is just at most 300KB extra to ship, then I just cannot see how that would be a meaningful difference (especially with one single scene being easily just as big).
With this said, again, I don't really have an opinion on the technical merit of any standard over the other (AFAIK it should even be that AVIF has both a lossless and RGB mode, even though they may not be as "first citizens" as the rest).. but that's what I feel like this thread should revolve around. Features and inherent shortcomings, not (for lack of a better term, even though I'm not giving it a negative connotation) politics.

@Gouvernathor
Copy link
Member

Gouvernathor commented Dec 25, 2022

Sure, but that's a feature argument. As far as the popularity argument goes, it's only a promise, that JXL has the potential of booming widely.
If it does, we certainly will support it, but until then...
(I was answering to Brunoais)

@brunoais
Copy link

brunoais commented Dec 25, 2022

If it does, we certainly will support it, but until then...

I just hope not too many think like that... Otherwise, it will be a large sum waiting for themselves... However, I do understand that stance very well.

If I were to make renpy... I think I'd be thinking about Pillow supporting it.

@mirh
Copy link

mirh commented Dec 26, 2022

A couple of ideas, again from a very detached prospective:

  • even if one of the JXL creators argue the AVIF default settings only gets you halfway between JXL and the original JPEG, I think this is probably the last of the concerns that you'd have to consider at the end of the day (moreover, I don't think JXL superiority in lossless compression or AVIF shining with stupid low quality settings matter too much)
  • putting aside anyway, that the two codecs seem to target different metrics to optimize against
  • I don't know if we care about processing speed, but AVIF is immensely worse there (unless you want to play the hardware decoding card, which would be kinda preaching to the self-fulfilling prophecy, and it would still exclude billions of devices)

There are actually two somewhat exclusive features that only each core has though:

  • AVIF has some sort of gif-like format, obviously very optimized too since the thing is derived from a video codec in the first place (to be honest in the year of the lord 2022, you'd really wonder why not just using a bloody video instead, but then you could just as well argue that Support AV1 codec movies on RenPy 7.x/8.x #3400 would give you this format for free).
  • JXL supports progressive decoding, with some pretty rad tricks if I may add. I'm not a creator or an artist, but I cannot really think to a bigger deal when you are talking about the experience inside a browser

Perhaps there are other nitty-gritty quirks that I have missed (perhaps some peculiar difference in HDR, CMYK or whatnot) but AFAICT these seem the elephants in the room.

@renpytom
Copy link
Member

I'm back leaning towards including JXL - there's apparently a patent issue with the AVIF container that might be problematic.

@brunoais
Copy link

Even if I'd like to see JXL in renpy. My research shows that AVIF has no patents attached, just like JXL. I don't think incorrect information should be the decider factor.

@thelordofcorruption
Copy link
Author

thelordofcorruption commented Dec 26, 2022

There might be a patent problem because AVIF is a format that puts an AV1 bitstream into a HEIF container and HEIF seems to be patented by Nokia. I am not a lawyer but I assume this could probably cause issues.

As for JXL even if it does not get web support, it can still be used on desktop and I assume on android if you deliver a native decoder. In my game I deliver lossless images for desktop and those aren't an option on AVIF since lossless images on AVIF seem to take more space then even pngs.

renpytom also mentioned size as a concern, AVIF's libraries are much larger then JPEGXL's are

@mirh
Copy link

mirh commented Dec 27, 2022

Size wouldn't be a problem for AVIF, because desktops don't care about it and browsers already support it out of the box regardless. And anyway, as long as we assume you'd want to support AV1, I think 95% of the weight would be already there.
Also, I guess that for as much the lossless features parity on paper JXL may have some "quantitative" advantage there.. and of course any use case should be covered. But is there really a practical reason that "visually lossless" isn't enough?

On the other hand I'm really appalled to learn that HEIF is potentially patent encumbered (it doesn't seem like Nokia has ever tried to assert them, and it would be really fucking nuts if netflix had cornered themselves in this amateurish way, but yet there's no explicit green light).

@morbil
Copy link

morbil commented Dec 30, 2022

One benefit for jxl is for ongoing project already using jpg (and not keeping lossless backups, which is foolish but a lot of users are not technical), it offers lossless transcoding, so they could gain possible space savings essentially "for free" without degrading quality.

Also, I do think that supporting AVIF through ffmpeg rather than SDL image, might help keep the binary size down, when AV1 support is added.

@Quest79
Copy link

Quest79 commented Jan 7, 2023

Very maybe. I like some things about the format, but I'd want to figure out if there's a patent problems, and if browsers actually turn Jpeg XL on by default.

What does it matter? Its an option, people that would select it can sort out where they want to open it. It literally halves the data needed for lossless imaging and every major photo editor can open it. Not to mention the lossy advantages. It should be added. You have bmp and tiff as options for gods sake, is this 1998

@bober-gith
Copy link

bober-gith commented Jan 8, 2023

I ran some tests this morning and damn the benefits of JpegXL over Webp are strong, even for desktops alone.
The benefits don't come so much from the gain in size compared to webp but more from the actual image quality achieved while keeping the size low.

For years I used using Webp lossless, no less than that because Webp even when set at 99% lossy (the next maximum in-line quality setting after lossless), you can see a loss a deterioration in the vibrant color (especially for the red) and a very tiny bit of smearing in certain areas. It made me stay away from 99% webp lossy compression except for android because the screen is tiny and you couldn't tell the difference.

Here is the test result in size, using xnconvert:
original file: 3.1M
webp lossless: 1.6M
jpegXL lossless: 1.5M
webp lossy 99%: 619KB
jpegXL lossy 99%: 744KB

Now the biggest game changer and what the size alone can't tell, is that jpegXL lossy 99% is subjectively an identical image to
lossless conversion, when 99% webp lossy is a far inferior image with smearing and loss in the color spectrum.
Basically, for the people that care for image quality, for the first time, it makes lossy something that you can actually use.
And this alone makes a huge difference, the day jpegxl is supported is the day many ren'py games in current development, including mine, will have their size halved because lossy is actually 99.99% identical to the original image in terms of quality so it becomes a no brainer to use it.

EDIT: Didn't run an intensive test but Avif seems to be just as good in terms of quality/size at lossy 99%, any of these two seems to be a big improvement over Webp in this scenario.

@mirh
Copy link

mirh commented Jan 8, 2023

Just for the very records (and hoping this wasn't also the reason for everybody else in here to nominate lossless compression)...
The great majority of times, noticeable losses in "vibrant colors (especially red)" is just chroma subsampling being enabled.

And it's only WebP (being derived from a lousy video codec) that requires a specialized hackjob of a profile to toggle it.
You could probably use 90% quality JPG (yes, the original base one, absolutely nothing new or fancy) without downsampling in 4:4:4 mode, and you'd probably still be talking about "subjectively identical".

@bober-gith
Copy link

I wasn't aware, however color is only one part of the equation, what about the smearing on some of the edges, not sure if that is easily fixable, I could send you an example if you have an email and show you how much an image can deteriorate even when set at 99% lossy using webp versus lossless. Which doesn't happen with Avif or Jpegxl.
For jpeg, it's always been a great format in terms of size and quality but afaik it doesn't support transparency which makes it a no-go except if you are making a VN with everything merged into one layer, which I think is pretty rare nowadays.

@go-delicious
Copy link

With some formats, you don't need to get a licence if the tool you are using already has it. Probably a good choice to check first, though. From some short research, it looks like Nokia owns a Java Decoder for decoding HEIF containers, not sure if they actually own the format itself.

@mirh
Copy link

mirh commented Jan 9, 2023

I could send you an example if you have an email and show you how much an image can deteriorate even when set at 99% lossy using webp versus lossless.

Yes. I know, I understand and I believe it.
It is because they aren't just compressing the image.. uh, "spatially" but they are also straightaway removing 75% of the chrominance information. And webp is a pretty shoehorned standard that can either do "full video-like" mode with every shortcut of the situation enabled or "full fidelity" mode with none at all.
I just mentioned JPEG because you wouldn't expect the far older and famously far less efficient codec to do better, and giving that a try would immediately confirm or disprove my suspicions.

Then, yes, transparency is important and only jxl or avif (which supports lossy 444 with the high profile) could solve every problem.
But if I'm right, I'm afraid this isn't really concerning (even if this or this then could further disrupt "mathematical losslessness").
But who knows, maybe if you try to encode something fancy like "10-bit HDR full-chroma RGB with alpha" one of them will break.

@renpytom
Copy link
Member

renpytom commented Jan 9, 2023

So, spent most of the weekend trying hard to add Jpeg-XL support to Ren'Py.

I'm now fairly convinced that this isn't going to happen until there's a JXL decoder that is written using standard C. The current libjxl implementation is written in modern C++, and trying to get modern C++ cross-compiled to the 14 or so combinations of operating system and CPU that Ren'Py supports... it just was not happening.

This is made a bit worse by the way we target older versions of many of these platforms, where the sysroots that are present don't support features used by modern C++, and by the way that Ren'Py statically links many of its dependencies to make this possible.

My current thinking is that I'd like to include Jpeg-XL decoding in Ren'Py. I plan to do it when there is a C library that can easily be compiled to the platforms Ren'Py supports at the time.

https://github.com/lifthrasiir/j40

Looks like one candidate for such a library, though it currently seems to be missing enough features at the moment that we couldn't deploy it. If anyone is aware of others, please let me know.

@renpytom renpytom removed this from the 8.1 / 7.6 Prerelease milestone Jan 9, 2023
@uyjulian
Copy link
Contributor

uyjulian commented Jan 9, 2023

One thing that could make it easier to target multiple platforms is to compile e.g. libjxl to WASM using wasi-sdk, then use wasm2c to convert the WASM to C. From there, it should be easier to use the exports provided to make use of the library.

Using this same method, it could also make it easier to integrate libraries that are written in different languages or have difficult build systems, such as Inochi2D.

@mirh
Copy link

mirh commented Jan 9, 2023

C++11 doesn't exactly seem all this fancy in the year of the lord 2023 (like, I think even the PSP and PS2 toolchains have some degree of support for C++20 by now).
Anyhow, I guess #3906 goes nowhere then?

@renpytom
Copy link
Member

renpytom commented Jan 9, 2023

I'm suspecting I will add avif, as all the browsers seem to have support, which means that there's unlikely to be a major patent problem. (And big pockets to defend against it.) Plus, I experimentally added avif last night, and it built without problems, which is a good sign.

The issue isn't really C++11 vs C++20, it's more going from no C++ to some.

My current plan with JXL is to wait for non-reference implementations to appear. (That's probably a good idea in any case, as it's a sign that the format is being adopted.)

@bober-gith
Copy link

A bit saddening to see what seems to be a slightly better format in Jxl being robbed away by what looks like to be a sort of lobbying from google... But overall Avif is still a pretty big improvement over Webp and from a non-political pov, it makes sense for Ren'py to support Avif since that means you can actually convert your images in the same format and have them working across all platforms I assume.

@suphamster
Copy link

suphamster commented Jan 10, 2023

Anybody already tried renpy/renpy-build@728013d to build/use avif with current renpy-build?
Btw animated avif not planned/supported? It' very efficient (regarding file size) - almost AV1 video alternative until it added too.

@loafylemon
Copy link
Contributor

A bit saddening to see what seems to be a slightly better format in Jxl being robbed away by what looks like to be a sort of lobbying from google... But overall Avif is still a pretty big improvement over Webp and from a non-political pov, it makes sense for Ren'py to support Avif since that means you can actually convert your images in the same format and have them working across all platforms I assume.

I wouldn't consider this to be a malicious intent, when there are other valid reasons to explain this. Jpeg XL compression yields the smallest file sizes, however, encoding and decoding are the slowest of all formats on every architecture.

https://storage.googleapis.com/avif-comparison/index.html

https://storage.googleapis.com/avif-comparison/decode-timing.html

The use of the AVIF format in Ren'py is a better choice, especially if it's faster and more widely supported. I think it would be confusing if certain Ren'py features only worked with certain file formats, and only on specific browsers.

If the format becomes more widely adopted, things may of course change.

@brunoais
Copy link

brunoais commented Jan 16, 2023

https://storage.googleapis.com/avif-comparison/index.html
https://storage.googleapis.com/avif-comparison/decode-timing.html

Google's tests are done wrongly. Not done in equivalent comparison position and using color gamuts that JXL wasn't properly optimized for yet (they focused on human perception vs machine perception). They used the worst jxl can deal with and then compared what AVIF deals acceptably with.
Jon Sneyers explains that well.

@bober-gith
Copy link

bober-gith commented Jan 16, 2023

Yeah, tests were wrong and using a version of Jxl that had problems, and for having tested both encoding speed, AVIF is significantly slower for roughly the same quality at the present time (but hopefully will improve quite a bit over time).

@loafylemon
Copy link
Contributor

https://storage.googleapis.com/avif-comparison/index.html
https://storage.googleapis.com/avif-comparison/decode-timing.html

Google's tests are done wrongly. Not done in equivalent comparison position and using color gamuts that JXL wasn't properly optimized for yet (they focused on human perception vs machine perception). They used the worst jxl can deal with and then compared what AVIF deals acceptably with. Jon Sneyers explains that well.

Sorry but this is false.

The linked test results are from December 12 (updated December 14), using the newest version of Jpeg XL (Version: 0.7-base-43) as outlined in the verification process.

@bober-gith
Copy link

bober-gith commented Jan 16, 2023

You can check the details of why this is a controversial result, by reading the comments (at the end for the most recent ones):
https://bugs.chromium.org/p/chromium/issues/detail?id=1178058

In any case for ren'py use case, encoding speed atm is faster on jpegXL. Though can't say for sure about decoding speed, but should be fast enough for a visual novel in any case I think.

@mirh
Copy link

mirh commented Jan 16, 2023

Guys, please. Focus.
The real deal here is that nobody gives a damn about performance for a visual novel.
And neither native compatibility with native platforms is really critical, since we can ship our libraries anyway.
(and if people care so much about "neatness inside the browser" then progressive coding seems the best thing since sliced bread)

@renpytom
Copy link
Member

Again, the problem with JXL was the difficulty of getting the reference implementation running on the many platforms Ren'Py supports, without massively rearranging the build to have dependencies on C++ runtimes that aren't needed anywhere else.

@Quackdoc
Copy link

as an update, there are currently, as far as I know two active jxl decoders, a native rust decoder jxl-oxide is a rust based one that has today hit v0.1.0 and was published, and a java one jxlatte of which I am unsure of it's current status.

jxl-oxide does pass a good amount of the conformance test suite (it doesnt seem to be tested against all of it), and performance at least on my desktop isn't bad.

I have no idea how well these may or may not integrate but if either of these looks like it might work for you, maybe worth keeping an eye on.

@Gouvernathor Gouvernathor added enhancement A proposed or requested enhancement or improvement to renpy's features. duration - day Resolving this is estimated to a day of development and removed enhancement - day labels May 29, 2023
@SoniEx2
Copy link

SoniEx2 commented Dec 11, 2023

could you use wasm2c/RLBox to get the reference implementation of JXL running?

(disclaimer: wasm2c contributor here)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duration - day Resolving this is estimated to a day of development enhancement A proposed or requested enhancement or improvement to renpy's features. long term
Projects
None yet
Development

Successfully merging a pull request may close this issue.