Squeeze and excitation (scSE) modules

[daniel-j-h]

We should look into squeeze and excitation (SE) modules for adaptive feature map recalibration.

The initially introduces cSE blocks are incredibly powerful for classification problems; the recently introduces sSE blocks are similarly powerful for segmentation.

We should try a combination of these in an scSE module and see what happens if we attach them to our encoder and decoder modules. From a computational standpoint they are very cheap.

References below

https://arxiv.org/abs/1709.01507

Squeeze-and-Excitation Networks

https://arxiv.org/abs/1803.02579

Concurrent Spatial and Channel 'Squeeze & Excitation' in Fully Convolutional Networks

from https://arxiv.org/abs/1803.02579