Luminance / wasm integration

[robert-chiniquy]

Hi!

I'm trying to get Egui drawing in Luminance in the browser. I thought this would be pretty easy since I can draw with Luminance, and Egui's API is perfect, just giving me the tesselation.

I have all the pieces together for rendering to happen (I think). I am getting no errors in the browser console. I followed all your tips here: https://github.com/emilk/egui#debugging-your-integration

What I'm observing is that the vertices are making it to the shaders, because if I change the blending function (or tweak the frag shader), I can make the area that my egui::SidePanel::left("❤", 200.) occupies change color, but by default it seems that it is white (the same as the pipeline clear color). So nothing is getting drawn, or, if it is, it is white, or it is being blended away.

✔️ I've got the vertices, texture coords, and indices loaded and they look good, along with the color attribute
✔️ I've got my blending function set correctly (I think, I've tried a few different combinations there)
✔️ I'm using (with small modifications) the shaders from egui_web

If anything stands out from the above, I would really appreciate any advice or guidance you can offer.

Thanks!

---

Edit to add: I pushed a sample repo if anyone would like to help: https://github.com/robert-chiniquy/egui-luminance

[zicklag]

Hey @robert-chiniquy I'm messing with your example and I figured I'd share what I've found so far:

Blending

So I modified the fragment shader to always return solid red to debug the positions of the vertices and I discovered that with the current blending, the alpha of the UI is somehow wrong and ending up transparent even with solid, opaque colors. I copied the blending I used for Bevy Retro and it seemed to fix it, but I know nothing about premultiplied alpha or how that changes things so it may need tweaking to be conformant:

let render_st = &RenderState::default()
            .set_blending_separate(
                Blending {
                    equation: Equation::Additive,
                    src: Factor::SrcAlpha,
                    dst: Factor::SrcAlphaComplement,
                },
                Blending {
                    equation: Equation::Additive,
                    src: Factor::SrcAlpha,
                    dst: Factor::SrcAlphaComplement,
                },
            )
            // This is also needed ( explained below )
            .set_depth_test(None);

Depth and Vertext Colors

As another experiment I tried setting the fragment color to the vertex color, completely ignoring the texture, to see what result that produced. I found out that it is necessary to set the depth test to None so that all subsequently drawn triangles will appear on top of the ones before unconditionally ( I'm guessing, anyway ).

After doing that, and still seeing nothing, I took out the linear-to-srgb step from the vertex color:

  // Old
  // v_rgba = linear_from_srgba(a_srgba);
  // New
  v_rgba = a_srgba;

Apparently there is some issue with that conversion because that yields:

So at that point it seems basically working other than the texture.

Other

From other messing around trying to get the texture to apply it seems like the texture might be 0% opaque, but I don't know for sure yet.

---

Also, I'm not sure exactly what your use-case for egui on top of Luminance is, but you might be interested in the luminance-glow backend that I made that works on WebGL 1 instead of WebGL 2, which I made primarily so that I could target Safari on iOS, which doesn't support WebGL 2. luminance-glow also supports desktop without requiring a separate luminance backend for it. The disadvantage of targeting WebGL 1 is that you have to write your shaders in GLSL ES 1.0, but that's not a big deal in a small app with only two shader files in it, I don't think.

---

I'm not sure if I'll get to messing around more than I have already, but if you want to use luminance-glow for any reason, I could probably show you how to get setup with it.

[emilk]

Your blend function is wrong, you can follow this: https://github.com/emilk/egui/blob/master/egui_glium/src/painter.rs#L166-L177

let render_st = &RenderState::default()
            .set_blending_separate(
                Blending {
                    equation: Equation::Additive,
                    src: Factor::One,
                    dst: Factor::SrcAlphaComplement,
                },
                Blending {
                    equation: Equation::Additive,
                    src: Factor::DstAlphaComplement,
                    dst: Factor::One,
                },
            )
            // This is also needed ( explained below )
            .set_depth_test(None);

[robert-chiniquy]

Thank you for all the help!

I have gotten this far applying the tips above (including the temporary ones for troubleshooting):

Still must be something wrong with the texture, either how it is being written to the GPU or how it is being read.

I'm thinking I shouldn't need to do any conversion to the texture pixels here: https://github.com/robert-chiniquy/egui-luminance/blob/main/src/egui_luminance.rs#L109 similar to https://github.com/emilk/egui/blob/master/egui_glium/src/painter.rs#L83 as the WebGL2 painter does not do that: https://github.com/emilk/egui/blob/master/egui_web/src/webgl2.rs#L366

[robert-chiniquy]

Comparing the state in Spector.js between my repo and the online demo of egui_web, I am seeing this difference in the texture:
From my repo using Luminance:

Texture
magFilter: LINEAR
minFilter: LINEAR
wrapS: CLAMP_TO_EDGE
wrapT: CLAMP_TO_EDGE
anisotropy: 1
baseLevel: 0
immutable: true
immutableLevels: 1
maxLevel: 0
maxLod: 1000
minLod: -1000
compareFunc: LEQUAL
compareMode: NONE
wrapR: CLAMP_TO_EDGE

From egui_web:

Texture
magFilter: LINEAR
minFilter: LINEAR
wrapS: CLAMP_TO_EDGE
wrapT: CLAMP_TO_EDGE
anisotropy: 1
baseLevel: 0
immutable: false
immutableLevels: 0
maxLevel: 1000
maxLod: 1000
minLod: -1000
compareFunc: LEQUAL
compareMode: NONE
wrapR: REPEAT
textureType: UNSIGNED_BYTE
format: RGBA
internalFormat: SRGB8_ALPHA8_EXT
width: 2048
height: 256

Several of those properties seem important to be set on the texture, but I think I am setting them correctly in Luminance. I think I need to dig this up on the Luminance side to understand where those properties are getting dropped before I ask for more help here.

Thanks again!

[zicklag]

You may want to debug print the bytes for the texture before you upload it to the GPU or something. Looking at those values you pasted above, it might be that your EGUI texture doesn't have a width and a height. Maybe there aren't any pixels getting uploaded. That would seem in line with what I experienced when trying to multiply the vertex colors by the texture: they would all end up transparent no matter what I did. That could make sense if the texture was empty.

[robert-chiniquy]

Thanks, yeah, I've done that and the pixels looked good to me. I was thinking of using something like get_raw_texels to look at the pixels after they're uploaded, to see if something was happening to them there.

I hadn't noticed the missing height/width, that is really weird. I should double-check that and figure that out too.

[robert-chiniquy]

Just thought I would update with progress. Feels like I am getting close. Details in the README here: https://github.com/robert-chiniquy/display-pure-egui-texture

Here's what I've got rendering with Luminance on the web now:

[robert-chiniquy]

Ok, another question: I am normalizing the vertex colors here: https://github.com/robert-chiniquy/egui-luminance/blob/main/src/egui_luminance.rs#L44 so they can be assigned to a vec4 rather than a uvec4 in the shader: https://github.com/robert-chiniquy/egui-luminance/blob/main/src/shaders/vertex_300es.glsl#L11 (Luminance won't build a program where the types of the attributes don't correctly match the types in the shaders)

Looking at egui_web, the vertex color attribute is not normalized: https://github.com/emilk/egui/blob/master/egui_web/src/webgl2.rs#L300 but it is still being assigned to a vec4 in the shader: https://github.com/emilk/egui/blob/master/egui_web/src/shader/vertex_300es.glsl#L5

I'm not confident that normalization is the cause of my issue, but that seems like a big discrepancy. That issue is why Zicklag above took out the linear-to-srgb step for the vertex color attribute: with Luminance normalizing the colors, they did not need to be converted (however, Luminance is not applying gamma correction in the normalization).

Any insight into that would be appreciated.