Converting an OpenGL bind texture to an Skia Image

[EldoDebug]

This is required to render 3D in OpenG and then use Skia to apply blur effects
reference
Luna0x01/lwjgl-skiko@59c6ea2
thank you

[THEREALWWEFAN231]

I one up this, it's related to/sloves #67
Which I later found was possible creating another native surface with the texture id then drawing that canvas onto the main canvas, but that is so inconvenient, adoptFromTexture is such a better implementation.

[EldoDebug]

For reference I have implemented adoptTextureFrom
https://github.com/Soar-Client/SoarSkija

[Lyzev]

For reference I have implemented adoptTextureFrom https://github.com/Soar-Client/SoarSkija

I tried using your fork, but I still can't seem to get it working. Can you provide information if your fork is still in development or ready to use?

I am currently trying to integrate Skija into Minecraft and have encountered a few challenges. While I can successfully draw rectangles and other shapes, I’m struggling to make the surface transparent. When I reduce the alpha value of the surface color, the surface simply becomes darker rather than transparent. If anyone knows why this happens, I’d appreciate an explanation.

To work around this, I considered using the texture of Minecraft's main framebuffer as the background. This led me to discover your fork, which I attempted to use, but it didn’t work as expected for me.

I would be very grateful for any assistance, whether it’s guidance on achieving a transparent surface or advice on using Minecraft as a background in Skija.

Note: In my code (a Fabric Example Mod) I currently use your fork (SoarSkija)

This is what it currently looks like (in Minecraft's main menu):

This is my current code (it's probably worth noting that this is just a test environment I'm trying to implement it in):

package dev.lyzev.skija.client

import com.mojang.blaze3d.systems.RenderSystem
import io.github.humbleui.skija.*
import io.github.humbleui.skija.impl.Library
import io.github.humbleui.skija.impl.Stats
import net.fabricmc.api.ClientModInitializer
import net.minecraft.client.MinecraftClient
import org.lwjgl.opengl.GL11

object SkijaClient : ClientModInitializer {

    val mc = MinecraftClient.getInstance()

    override fun onInitializeClient() {
        // This code runs as soon as Minecraft is in a mod-load-ready state.
    }

    var context: DirectContext? = null
    var renderTarget: BackendRenderTarget? = null
    var surface: Surface? = null
    var canvas: Canvas? = null
    var dpi = 1f

    fun initSkia() {
        if (context == null) {
            if ("false".equals(System.getProperty("skija.staticLoad")))
                Library.load();
            context = DirectContext.makeGL()
        }

        Stats.enabled = true

        surface?.close()
        renderTarget?.close()

        renderTarget = BackendRenderTarget.makeGL(
            (mc.window.framebufferWidth * dpi).toInt(),
            (mc.window.framebufferHeight * dpi).toInt(),
            0,
            8,
            0,
            FramebufferFormat.GR_GL_RGBA8
        )


        surface = Surface.wrapBackendRenderTarget(
            context!!, renderTarget!!, SurfaceOrigin.BOTTOM_LEFT, SurfaceColorFormat.RGBA_8888, ColorSpace.getSRGB()
        )

        canvas = surface!!.canvas
    }

    fun draw() {
        RenderSystem.enableBlend() // Tried to use blend mode to make transparency work, but it doesn't
        RenderSystem.defaultBlendFunc()
        canvas!!.clear(0x80FFFFFF.toInt()) // Transparency somehow doesn't work

        canvas!!.drawImage(
            Image.adoptTextureFrom(
                context,
                mc.framebuffer.colorAttachment,
                GL11.GL_TEXTURE_2D,
                mc.framebuffer.textureWidth,
                mc.framebuffer.textureHeight,
                GL11.GL_RGBA8,
                SurfaceOrigin.BOTTOM_LEFT,
                ColorType.RGBA_8888
            ), 0f, 0f
        )

//        val paint = Paint().setColor(0x80FFFFFF.toInt())
//        canvas!!.drawRect(Rect.makeLTRB(10f, 10f, mc.window.framebufferWidth - 100f, mc.window.framebufferHeight - 100f), paint);

        context!!.flush()
        RenderSystem.disableBlend()
    }
}

For additional information here are my mixins:

package dev.lyzev.skija.mixin;

import dev.lyzev.skija.client.SkijaClient;
import net.minecraft.client.util.Window;
import org.spongepowered.asm.mixin.Mixin;
import org.spongepowered.asm.mixin.injection.At;
import org.spongepowered.asm.mixin.injection.Inject;
import org.spongepowered.asm.mixin.injection.callback.CallbackInfo;

@Mixin(Window.class)
public class WindowMixin {

    @Inject(method = "onWindowSizeChanged", at = @At("TAIL"))
    private void onWindowSizeChanged(long window, int width, int height, CallbackInfo ci) {
        SkijaClient.INSTANCE.initSkia();
    }
}

package dev.lyzev.skija.mixin;

import dev.lyzev.skija.client.SkijaClient;
import net.minecraft.client.MinecraftClient;
import net.minecraft.client.RunArgs;
import org.spongepowered.asm.mixin.Mixin;
import org.spongepowered.asm.mixin.injection.At;
import org.spongepowered.asm.mixin.injection.Inject;
import org.spongepowered.asm.mixin.injection.callback.CallbackInfo;

@Mixin(MinecraftClient.class)
public class MinecraftClientMixin {

    @Inject(method = "<init>", at = @At("TAIL"))
    private void init(RunArgs args, CallbackInfo ci) {
        SkijaClient.INSTANCE.initSkia();
    }

    @Inject(method = "render", at = @At(value = "INVOKE", target = "Lnet/minecraft/client/gl/Framebuffer;draw(II)V", shift = At.Shift.AFTER))
    private void render(CallbackInfo ci) {
        SkijaClient.INSTANCE.draw();
    }
}

[EldoDebug]

I don't know about the latest version of minecraft, but is mc.framebuffer.colorAttachment an OpenGL textureId?
It is a little strange, but it can be rendered (Minecraft 1.8.9)

[EldoDebug]

And using Skija in Minecraft requires some ingenuity
This would help with the alpha issue
https://github.com/Quantamyt/Skija-Client-Base/tree/main/src/main/java/dev/quantam/skija/core/state

[THEREALWWEFAN231]

I'm wondering what exactly you are trying to do? You are hooking skia to the 0 frame buffer and drawing the Minecraft framebuffer texture with @EldoDebug adopt texture implementation, Does this make sense, why?

Also I'm almost 100% certain your transparency issue is the frame buffers clear color, it took me about 3 days to figure this out, because skia does pre multiplied blending the clear color for the frame buffer needs to be 0 0 0 0, even white with 0 alpha doesn't work. The way you have things setup right now, I think you need to set the 0 frame buffers clear color to 0000. Personally though, I would hook skia to Minecraft's frame buffer, and create a new frame buffer(set clear color to 0000) draw what you want to there and use adopt texture to draw that FBO with skia.

[Lyzev]

And using Skija in Minecraft requires some ingenuity This would help with the alpha issue https://github.com/Quantamyt/Skija-Client-Base/tree/main/src/main/java/dev/quantam/skija/core/state

I really appreciate your quick responses, it really helps me. I'm still new to Skija, so I'm still learning.

[Lyzev]

I'm wondering what exactly you are trying to do? You are hooking skia to the 0 frame buffer and drawing the Minecraft framebuffer texture with @EldoDebug adopt texture implementation, Does this make sense, why?

Also I'm almost 100% certain your transparency issue is the frame buffers clear color, it took me about 3 days to figure this out, because skia does pre multiplied blending the clear color for the frame buffer needs to be 0 0 0 0, even white with 0 alpha doesn't work. The way you have things setup right now, I think you need to set the 0 frame buffers clear color to 0000. Personally though, I would hook skia to Minecraft's frame buffer, and create a new frame buffer(set clear color to 0000) draw what you want to there and use adopt texture to draw that FBO with skia.

Thanks for the reply, I am still new to Skija. I didn't know about this transparency thing with clear color.