Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add RTX video enhancement HDR to MPC video renderer. #24

Open
finasfinas opened this issue Jan 24, 2024 · 47 comments
Open

add RTX video enhancement HDR to MPC video renderer. #24

finasfinas opened this issue Jan 24, 2024 · 47 comments

Comments

@finasfinas
Copy link

RTX video enhancement HDR was added to driver version 551.23. It's an AI powered feature just like super resolution.
It can be enabled on the nvidia control panel and it needs requesting the feature from the renderer.

https://blogs.nvidia.com/blog/rtx-video-hdr-remix-studio-driver/
https://www.tomshardware.com/pc-components/gpu-drivers/nvidia-adds-ai-powered-sdr-to-hdr-video-enhancement-to-its-latest-drivers-rtx-video-hdr-now-available-for-all-rtx-owners

@hsy5571615
Copy link

I'm looking forward to integrating this feature as well.

@emoose
Copy link
Owner

emoose commented Jan 26, 2024

The build at Aleksoid1978#116 (comment) appears to support it, not sure if it's working as well as in chrome/edge though.

@PedroM4
Copy link

PedroM4 commented Jan 26, 2024

I knew you would come to the rescue! many many thanks for putting in the time and work. Will try it!

Cheers

@Naboum
Copy link

Naboum commented Jan 26, 2024

The build at Aleksoid1978#116 (comment) appears to support it, not sure if it's working as well as in chrome/edge though.

I tried this test build and works just as well as in chrome.

However it washes out the colors if the video is already in HDR format.
Screenshot when playing an HDR video using the test build : https://i.imgur.com/7ihMFgk.jpeg
Screenshot when playing the same video using the latest released build 0.7.1.2172 : https://i.imgur.com/0zJHW1U.jpeg

@emoose
Copy link
Owner

emoose commented Jan 26, 2024

@Naboum could you see if this build works any better? Added a check so RTX HDR is only enabled if source video isn't HDR:

MpcVideoRenderer64-rtxhdr1.1.zip

@geahad
Copy link

geahad commented Jan 26, 2024

@Naboum could you see if this build works any better? Added a check so RTX HDR is only enabled if source video isn't HDR:

MpcVideoRenderer64-rtxhdr1.1.zip

I've tested this version and it correctly distinugishes between native-hdr video files and sdr-ones. Well done!

Btw. thank you SO much for making this happen! <3

@Naboum
Copy link

Naboum commented Jan 26, 2024

@Naboum could you see if this build works any better? Added a check so RTX HDR is only enabled if source video isn't HDR:

MpcVideoRenderer64-rtxhdr1.1.zip

It now works correctly, thank you.

@emoose
Copy link
Owner

emoose commented Jan 26, 2024

Glad to hear, pushed 1.1 release at https://github.com/emoose/VideoRenderer/releases/tag/rtx-hdr-1.1 which includes the change along with a fix to let VSR work up to 1440p

@geahad
Copy link

geahad commented Jan 26, 2024

There seems to be one little caveat though (but that one's on nvidia):
If the file is native HDR and is lower res than display native res (ex. 1080 on a 2160p screen), VSR is used, and then the hdr image stays washed-out, because for some reason VSR can be done before upgrading to an HDR tonemap, but it can't be done if VSR tries to manipulate a native HDR image. If I remember correctly, this was the case with VSR since the beginning.

Also, if I may ask for a little something more: the latest versions of the renderer have options to select above which resolution VSR should kick-in. The highest available from the drop down is "eqal to or lower than 1080p". I'm on a 2160p display and actually liked using VSR to upscale from 1440p also (is offciially supported, worked in previous versions). Could that ("equal to or lower than 1440p) option be added to the drop down?

@geahad
Copy link

geahad commented Jan 26, 2024

Ahaha, sorry... Haven't seen you've already integrated tha 1440p thing already...

You work very quickly :-D

@Naboum
Copy link

Naboum commented Jan 26, 2024

Glad to hear, pushed 1.1 release at https://github.com/emoose/VideoRenderer/releases/tag/rtx-hdr-1.1 which includes the change along with a fix to let VSR work up to 1440p

I played a 4K native-hdr video in fullscreen and the colors are again washed out. However in windowed mode it works great. It only happens when Super Resolution is enabled in the settings.

EDIT : same thing happens when playing a 720p native-hdr video while in fullscreen and Super Resolution setting set to <= HD or higher, and when playing a 1080p native-hdr video while in fullscreen and Super Resolution setting set to <= Full HD or lower.

@Naboum
Copy link

Naboum commented Jan 26, 2024

There seems to be one little caveat though (but that one's on nvidia): If the file is native HDR and is lower res than display native res (ex. 1080 on a 2160p screen), VSR is used, and then the hdr image stays washed-out, because for some reason VSR can be done before upgrading to an HDR tonemap, but it can't be done if VSR tries to manipulate a native HDR image. If I remember correctly, this was the case with VSR since the beginning.

If you try playing a native hdr 1080p video on youtube with a 4K monitor, with both VSR and HDR enabled in Nvidia control panel, the colors aren't washed out. Does that mean VSR is automatically disabled by chrome during this scenario ?

EDIT :
It does seem that Chrome doesn't enable VSR when playing a native-hdr video even if the setting is checked in NVCP.
Maybe MPCVR should have the same behavior ?
Otherwise we have to manually disable VSR in MPCVR settings when we want to play a native hdr video and enable it again after.

@iG8R
Copy link

iG8R commented Jan 26, 2024

Also observe issues with colors, but in my case they are bleak in the "HDR Off" mode.



@Raxcoms
Copy link

Raxcoms commented Jan 26, 2024

The RTX HDR seems to be breaking when watching SDR 10-bit content

10-bit video
20240126_152734

After converting the file to 8-bit the HDR works as expected
20240126_152711

EDIT: tried loading the file in a browser video player extension and the HDR doesn't seem to kick in when loading 10 bit content (it works for the 8-bit version in the extension as well, so I guess 10 bit is not supported)

@emoose
Copy link
Owner

emoose commented Jan 26, 2024

Hmm, guess maybe we should only try enabling superres for non-native-HDR content seeing as NV mention that VSR doesn't support HDR, and should also only enable RTX HDR when it's a HDR display.

tried loading the file in a browser video player extension and the HDR doesn't seem to kick in when loading 10 bit content (it works for the 8-bit version in the extension as well, so I guess 10 bit is not supported)

Ah nice was just going to ask if you could try that, yeah would guess it probably doesn't like 10-bit, I'll see if I can add a check to only enable for 8-bit formats (if you change your internal filters > video decoder settings in MPC then it should be able to convert 10-bit to 8-bit automatically)

E: here's a build with those changes:
rtx-hdr-1.2-test.zip

@Raxcoms
Copy link

Raxcoms commented Jan 26, 2024

Hmm, guess maybe we should only try enabling superres for non-native-HDR content seeing as NV mention that VSR doesn't support HDR, and should also only enable RTX HDR when it's a HDR display.

tried loading the file in a browser video player extension and the HDR doesn't seem to kick in when loading 10 bit content (it works for the 8-bit version in the extension as well, so I guess 10 bit is not supported)

Ah nice was just going to ask if you could try that, yeah would guess it probably doesn't like 10-bit, I'll see if I can add a check to only enable for 8-bit formats (if you change your internal filters > video decoder settings in MPC then it should be able to convert 10-bit to 8-bit automatically)

E: here's a build with those changes: rtx-hdr-1.2-test.zip

Weird. NCP reports that HDR Video Enhancement is off when playing 10-bit video in MPC, but it still looks like the first image in my last post

Rolled back to version 1.1 and the same happens there, NCP reports HDR Video Enhancement off, but the player still outputs a broken HDR image

Also tried opening the file with the stock Video Renderer from Aleksoid1978 to check if there was something broken with the player always showing 10bit video as HDR, but the video plays fine there

@emoose
Copy link
Owner

emoose commented Jan 26, 2024

Oh might have an idea what it is, for RTX HDR to work this removed a check for it being HDR source so it'd use right color space, with SDR it works fine since RTX HDR will convert it, but since RTX doesn't handle 10-bit I guess that makes it use the wrong colors.

Maybe this build can fix it, should only use that color space if it's HDR or 8-bit:
rtx-hdr-1.2-test2.zip

@Raxcoms
Copy link

Raxcoms commented Jan 26, 2024

Oh might have an idea what it is, for RTX HDR to work this removed a check for it being HDR source so it'd use right color space, with SDR it works fine since RTX HDR will convert it, but since RTX doesn't handle 10-bit I guess that makes it use the wrong colors.

Maybe this build can fix it, should only use that color space if it's HDR or 8-bit: rtx-hdr-1.2-test2.zip

Seems to be working fine now!

@XcellMiguel
Copy link

XcellMiguel commented Jan 26, 2024

Hi, I have a weir problem in MPC-BE x64 1.6.11, HDR and Super Resolution work when windowed, but when going fullscreen it works when the OSD is visible, once it disappear both get disabled.
It also work when the UI is visible when going to the bottom of the player in fullscreen, it disabled when only the video is visible.

I tried disabling the OSD, it's worse, going fullscreen when the video is playing just gives a black screen and I have to reload the video to get it to work again.
If I pause the video before going fullscreen, there's no black screen but no HDR nor VSR.

I tried the 1.1 and 1.2-test2 and got the same issue.

EDIT : looks like an NVidia issue, it does the same in the browser now, on Youtube it works until the UI disappears, something broke I don't know how.

@Naboum
Copy link

Naboum commented Jan 26, 2024

@Raxcoms
I tried playing an sdr 10bit video in chrome and the rtx hdr does its job correctly, it's weird that it doesn't work for you.
MediaInfo of my sample test : https://i.imgur.com/7BUgez2.png

@Raxcoms
Copy link

Raxcoms commented Jan 27, 2024

@Naboum weird. I have two 10-bit files and neither work for me
Both can't be played with chrome's default video player
And when played with the media player video extension HDR doesn't kick in in neither as well (but does with other files)

Media info 1: link
Media info 2: link

Also both give me the broken hdr output

@SESNut
Copy link

SESNut commented Jan 27, 2024

still getting overblown out colors playing 10 bit hevc videos with test 2 myself

@Naboum
Copy link

Naboum commented Jan 27, 2024

@Raxcoms What do you mean by "they cant be played" ? You can't drag and drop the file to a new tab ? Are they mkv ?
Unfortunately without playing your file on my monitor, I don't know how to replicate your issue. Your mediainfo data seems fine.

@Raxcoms
Copy link

Raxcoms commented Jan 27, 2024

Yes, they are .mkv files
Chrome just prompts me to download the file rather than playing it

@emoose
Copy link
Owner

emoose commented Jan 27, 2024

still getting overblown out colors playing 10 bit hevc videos with test 2 myself

Seems one of the color depth checks I added wasn't working right since the depth value wasn't setup yet, hopefully this build fixes that:
rtx-hdr-1.2-test3.zip

Still won't enable RTXHDR for 10-bit SDR though, in Vivaldi it looks like RTXHDR does support it, but haven't been able to get it working in MPC without colors being washed out :/

E: oh might have figured out RTXHDR for 10-bit SDR now, just had to loosen some of the checks I'd added.
Here's a build if anyone wants to test, let me know if you see any wrong colors:
rtx-hdr-1.2-test4.zip

E2: removed some redundant code I'd added:
rtx-hdr-1.2-test4a.zip

E3: one more test build, this should be able to disable all the autoHDR stuff if it failed to be enabled (eg. old driver ver, or not NV card), should now reinit with correct colors instead of still using HDR colorspace:

rtx-hdr-1.2-test5.zip

Posted release at https://github.com/emoose/VideoRenderer/releases/tag/rtx-hdr-1.2

@SESNut
Copy link

SESNut commented Jan 27, 2024

works now 👍

@SESNut
Copy link

SESNut commented Jan 27, 2024

if i select 8 bit texture format on an 8 bit video the colors are still screwed up and it reports rtx hdr as inactive. It activates fine if i choose 10 or 16 though

@Naboum
Copy link

Naboum commented Jan 27, 2024

Yes, they are .mkv files Chrome just prompts me to download the file rather than playing it

Chrome doesn't like mkv, you have to convert it to mp4

@Raxcoms
Copy link

Raxcoms commented Jan 27, 2024

Posted release at https://github.com/emoose/VideoRenderer/releases/tag/rtx-hdr-1.2

My .mkv 10-bit files are playing with auto HDR properly now!

@Kacarek
Copy link

Kacarek commented Jan 27, 2024

Would be kinda cool if it worked for image inputs as well.

@GautamtmD
Copy link

GautamtmD commented Jan 29, 2024

I tested all the output formats in the LAV video decoder settings and only these work perfectly
image

My MPC Renderer settings are
image

Most of the other formats have messed up colors like
image

Tested on the HEVC YUV444 10 bit format trailer of Argylle downloaded from https://thedigitaltheater.com/argylle-2024/

I tested this by enabling only one of the output format options at a time and playing the trailer, the results for forcing conversion each of these output formats is posted below
nv12 - works
yv12 - messed up color
p010 - works
p016 - messed up color
yuy2 - works
uyvy - works, can't display overlay
p210 - messed up color
y410 - messed up color
v210 - crashes - failed to find a connectable filter
p216 - messed up color
yv24 - messed up color
ayuv - works
y410 - messed up color
v410 - crashes - failed to find a connectable filter
y416 - messed up color
rgb32 - messed up color
rgb24 - messed up color
rgb48 - messed up color

Hope it helps in the development or configuration for someone.
So it is possible to play 10 bit video files with rtx hdr and super resolution just set the enabled output formats as in my settings.

@clsid2
Copy link

clsid2 commented Jan 29, 2024

AYUV gives bad performance and should always be UNchecked like it is by default.

Thanks for testing. Emoose should add a check to only allow RTXHDR for NV12/P010/YUY2. Those colorspaces are used in 99% of all cases anyway.

@emoose
Copy link
Owner

emoose commented Jan 31, 2024

@GautamtmD thanks for looking into them, any chance you could try some of the broken ones with the build at #26 (comment)? Maybe possible that MPCVR was trying to convert them and breaking since that was missing.

E: ah I missed the download link in your post, seems to give the same results as you had, too bad. Will add the checks clsid2 suggested in next release.

@geahad
Copy link

geahad commented Feb 4, 2024

Hello!

Would it be possible/feasable to add some image tweaking sliders (for contrast, gamma, saturation, peak nits and paper white) for RTX HDR in this renderer, like you've done for this project: emoose/DLSSTweaks#120 ?

I've tried putting the x64 version of the provided asi loader, tried renaming it into the suggested possible names and all, and then, as per instructions, put it into the folder where mpc-hc64.exe is, but alas, it crashes, and thus (as expected) is not compatible with non-games...

That would be, like, beyond awesome.

@emoose
Copy link
Owner

emoose commented Feb 5, 2024

I've tried putting the x64 version of the provided asi loader, tried renaming it into the suggested possible names and all, and then, as per instructions, put it into the folder where mpc-hc64.exe is, but alas, it crashes, and thus (as expected) is not compatible with non-games...

For me it seems to load into mpc-hc64.exe fine without crashing (E: ah it does crash here sometimes too, got an idea for it though)

Seems there's no effect with it atm though, guess the way they configure Video HDR is probably different to how TrueHDR works.

Does it make any truehdrtweaks.log file for you? If it doesn't maybe try running it as admin instead.

E: yep, seems video HDR settings get inited in a different part, looks like it does take same kind of parameters as TrueHDR does though, hopefully can find a way to adjust them soon.

Also looks like VideoHDR does actually update the nits/contrast/saturation/paperwhite TrueHDR settings based on your display, not really sure where it pulls all the values for them yet though.

@emoose
Copy link
Owner

emoose commented Feb 5, 2024

Just posted tweaks 0.5 at https://www.nexusmods.com/site/mods/781 which seems to work with VideoHDR now, tried running it a couple times and couldn't make it crash anymore neither, if you still have any issues let me know.

@geahad
Copy link

geahad commented Feb 5, 2024

Just posted tweaks 0.5 at https://www.nexusmods.com/site/mods/781 which seems to work with VideoHDR now, tried running it a couple times and couldn't make it crash anymore neither, if you still have any issues let me know.

I have tested this version, and it indeed loads and almost all function-identified ini parameters works as advertised. The cvar that makes the picture go bananas is the one for peak nits. It seems this one needs to be left alone at "-1",
The bigger problem is that, whenever the asi loader is injecting, there is a bright green vertical line on the right side of the screen about 4 pixels in width. Because of that it's practically unusable (am using an oled, so a no-no because of burn-in). I suspect this probably can't be solved because it most likely is an internal nvidia issue. I very much appreciate the effort though, as always!

@emoose
Copy link
Owner

emoose commented Feb 5, 2024

Huh didn't notice a green line on my end, did you rename the winmm.dll at all? It could be the saturation/contrast values that come pre-overridden in the INI too, maybe setting those to -1 would help.

I posted 0.5.1 to help fix a DX9 issue, probably won't make much difference but worth a try if you haven't already.

@geahad
Copy link

geahad commented Feb 6, 2024

Huh didn't notice a green line on my end, did you rename the winmm.dll at all? It could be the saturation/contrast values that come pre-overridden in the INI too, maybe setting those to -1 would help.

I posted 0.5.1 to help fix a DX9 issue, probably won't make much difference but worth a try if you haven't already.

Thank you for the suggestions.

The setting "Contrast = 1.0" was the culprit for the green line. Interestingly enough, as soon as I set it to anything above 1.0 (i.e. even "1.01") the green line goes away. Very peculiar indeed.

Thank you, now it's perfectly usable. Cheers!

@emoose
Copy link
Owner

emoose commented Feb 7, 2024

The setting "Contrast = 1.0" was the culprit for the green line. Interestingly enough, as soon as I set it to anything above 1.0 (i.e. even "1.01") the green line goes away. Very peculiar indeed.

Glad to hear, I did just run into the green line issue earlier after posting too, as you said changing contrast seems to fix it, kinda odd that it would cause something like that, haven't seen it in any games yet at least.

BTW it seems a lot of people measured VideoRTX at 650nits, and grabbing the settings VideoRTX sets up it does seem to set PeakBrightness to 650 for me on a 1000nit display, seems the 650 number is taken from somewhere else though, not sure if it's just 65% of display nits, or maybe VideoRTX has its own settings data which this copies from - if it's the latter maybe there's some way to change it from the player, hm...

@clsid2
Copy link

clsid2 commented Feb 9, 2024

@emoose
RTX Video HDR has been implemented in main project. Can you test it and check if any fixes are needed?

@geahad
Copy link

geahad commented Feb 28, 2024

Hello!

The only tangible difference between the last version emoose posted (https://github.com/emoose/VideoRenderer/releases/tag/rtx-hdr-1.2) and the last two ones offered for download with MPC-HC v2.1.5 (MPCVideoRenderer_0.7.2.2195) and v2.1.6 (MPCVideoRenderer_0.7.2.2203) is that they don't engage superresolution for resolutions larger than 1080p.
It supports up to 1440p per nvidia spec. I myself have a 2160p screen and the video files I have that are in 1440p greatly benefit when the renderer engages VSR.

No biggie in the end, but for me personally, it's the only reason why I'm still sticking with v1.2.
Cheers!

@Penetralia69
Copy link

Hi! How do i disable rtx hdr for native 10 bit hdr content? It seems to be in use according to high gpu usage.

@djsolidsnake86
Copy link

I tested all the output formats in the LAV video decoder settings and only these work perfectly image

My MPC Renderer settings are image

Most of the other formats have messed up colors like image

Tested on the HEVC YUV444 10 bit format trailer of Argylle downloaded from https://thedigitaltheater.com/argylle-2024/

I tested this by enabling only one of the output format options at a time and playing the trailer, the results for forcing conversion each of these output formats is posted below nv12 - works yv12 - messed up color p010 - works p016 - messed up color yuy2 - works uyvy - works, can't display overlay p210 - messed up color y410 - messed up color v210 - crashes - failed to find a connectable filter p216 - messed up color yv24 - messed up color ayuv - works y410 - messed up color v410 - crashes - failed to find a connectable filter y416 - messed up color rgb32 - messed up color rgb24 - messed up color rgb48 - messed up color

Hope it helps in the development or configuration for someone. So it is possible to play 10 bit video files with rtx hdr and super resolution just set the enabled output formats as in my settings.

Can this solution lead to problems with other types of videos or greater energy consumption?

@clsid2
Copy link

clsid2 commented Jun 21, 2024

Yes, it will lead to problems with HDR and it should also not be needed anymore.

These are outdated comments for outdated driver for outdated fork of the renderer.

@Madblaster6
Copy link

Glad to hear, pushed 1.1 release at https://github.com/emoose/VideoRenderer/releases/tag/rtx-hdr-1.1 which includes the change along with a fix to let VSR work up to 1440p

I played a 4K native-hdr video in fullscreen and the colors are again washed out. However in windowed mode it works great. It only happens when Super Resolution is enabled in the settings.

EDIT : same thing happens when playing a 720p native-hdr video while in fullscreen and Super Resolution setting set to <= HD or higher, and when playing a 1080p native-hdr video while in fullscreen and Super Resolution setting set to <= Full HD or lower.

Turn off "Auto HDR" in windows. This fixed it for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests