Multichannel image convolution

[pborsutzki]

Hi,

I am trying to use vkFFT to do a 2d image ("picture") convolution on an image with multiple channels using the Vulkan backend. I need each channel to be convolved on its own with the same kernel (e.g. a blur kernel). So far I managed to get a convolution on a one-channel image using r2c and the integrated kernel convolution settings to work. The question is now, do I have to decompose my multichannel image into several single channel images or is there a way to specify an element stride in vkFFT?

The current memory layout is this:

rgba rgba rgba
rgba rgba rgba
rgba rgba rgba

I would like to avoid separating the channels like this:

r r r
r r r
r r r
g g g
g g g
g g g
b b b
...

But I don't see any other way given the options vkFFT offers. The bufferStride options seem to affect a whole FFT dimension, but don't seem to allow specifying a distance between elements of that FFT dimension. The coordinateFeatures option would probably work on the separated version for a single launch or the bufferOffset options on separate launchs. So did I miss something or is separation of the channels the only way to go?

Also it seems like enabling zero padding disables kernel convolutions. Is this expected? For me it probably would only be useful in combination with an integrated fftshift option (which isn't available, I think?), as my kernels seldomly span the whole image.

Thanks for any help and the great library!

[DTolm]

Hello,

For R2C it is not currently possible to do a strided first axis as I have only implemented a non-strided version of R2C optimization so far (it modifies the layout from N real values to N/2+1 complex). I will try and add the strided R2C in one of the future updates.

Zero-padding should work with convolutions (see example 51), but in some cases, it may depend on what you are trying to do (for example, convolutions don't work with the Bluestein algorithm on the same axis so far). As for fftshift, no, VkFFT only has the functionality of Fast Fourier Transforms so far.

Best regards,
Dmitrii

[pborsutzki]

Hi,

thank you for the answer. I'll then decompose the image for now.

One more thing: I tried to do multiple launches with offsets to avoid duplicating my kernel per channel (it is the same kernel for each channel), but enabling specifyOffsetsAtLaunch alone already leads to a division by zero in somewhere in dispatchEnhanced. axis->specializationConstants.kernelNumberByteSize is zero for me. Is this a bug or am I missing something?

Thanks!

[DTolm]

Hello,

The issue is identified and I will fix it in the next release.

Best regards,
Dmitrii

[pborsutzki]

Many thanks!