[Bug] Compressed Textures fail in 9.1

[ibgreen]

Module and/or Feature

@luma.gl/webgl

Description

Compressed textures fail to load
visgl/deck.gl#9341

Expected Behavior

No response

Steps to Reproduce

visgl/deck.gl#9341

Environment

• Framework version: 9.1
• Browser:
• Node:
• OS:

Logs

No response

[donmccurdy]

A few findings:

• In the loaders.gl textured example, calls to device.createTexture include format only in the mipmaps/images array, but not a top-level props.format value. The example has access to the GL format (integer) but may need a way to convert that to a WebGPU format (string), I think there were utilities for that in luma 9.0 but I'm not sure about 9.1.
• In luma.gl's texture.ts we may need some adjustment to upload mipmaps:

  • luma.gl/modules/core/src/adapter/resources/texture.ts

    Lines 230 to 263 in 9ca6f80

    	/** Initialize texture with supplied props */
    	// eslint-disable-next-line max-statements
    	_initializeData(data: TextureProps['data']): void {
    	// Store opts for accessors
    	if (this.device.isExternalImage(data)) {
    	this.copyExternalImage({
    	image: data,
    	width: this.width,
    	height: this.height,
    	depth: this.depth,
    	mipLevel: 0,
    	x: 0,
    	y: 0,
    	z: 0,
    	aspect: 'all',
    	colorSpace: 'srgb',
    	premultipliedAlpha: false,
    	flipY: false
    	});
    	} else if (data) {
    	this.copyImageData({
    	data,
    	// width: this.width,
    	// height: this.height,
    	// depth: this.depth,
    	mipLevel: 0,
    	x: 0,
    	y: 0,
    	z: 0,
    	aspect: 'all'
    	});
    	}
    	}

• In luma.gl's webgl-texture-table.ts it looks like getTextureFormatWebGL is returning the format corresponding to uncompressed data, whereas we need values like ext.COMPRESSED_RGBA_ASTC_4x4_KHR for calls to compressedTexSubImage2D(...).

  • luma.gl/modules/webgl/src/adapter/converters/webgl-texture-table.ts

    Lines 327 to 347 in 9ca6f80

    	export function getTextureFormatWebGL(format: TextureFormat): {
    	internalFormat: GL;
    	format: GLTexelDataFormat;
    	type: GLPixelType;
    	compressed: boolean;
    	} {
    	const formatData = WEBGL_TEXTURE_FORMATS[format];
    	const webglFormat = convertTextureFormatToGL(format);
    	const decoded = getTextureFormatInfo(format);
    	return {
    	internalFormat: webglFormat,
    	format:
    	formatData?.dataFormat ||
    	getWebGLPixelDataFormat(decoded.channels, decoded.integer, decoded.normalized, webglFormat),
    	// depth formats don't have a type
    	type: decoded.dataType
    	? getGLFromVertexType(decoded.dataType)
    	: formatData?.types?.[0] || GL.UNSIGNED_BYTE,
    	compressed: decoded.compressed || false
    	};
    	}

These might be helpful as samples/tests:

2d_ktx2_samples.zip

They're various transcodings of a red Basis Universal texture, re-encoded as:

• astc
• etc1
• etc2
• bc1
• bc3
• bc5
• rgba8

None of them (except the source ETC1S file) require Basis transcoding, ZSTD decompression, or WASM or Worker dependencies. So they should be mostly trivial to load in a unit test, parse with ktx-parse, and then pass the mipmap data directly into device.createTexture(...). I'm hoping to try a PR for that, but I may still be missing one or two missing pieces to get compressed textures rendering.

[ibgreen]

Agreed.

• Test cases will be important to keep this from breaking.
• Also There is an example in loaders.gl that shows various format textures. It might be good to also have that example in luma.gl: https://loaders.gl/examples/textures

The example has access to the GL format (integer) but may need a way to convert that to a WebGPU format (string)

• In the spirit of luma.gl we should standardize on WebGPU strings
• (a small problem being that not all the compressed format have been assigned such strings yet).
• loaders.gl should ideally be updated to return webgpu strings rather than GL constants.
• Ideally the mapping should be in the webgl module and only included when using the webgl backend, but perhaps we need this mapping in core to be able to accept textures from sources that use GL constants.