Can i apply the same method to 3D conv?

[seominseok0429]

Can i apply the same method to 3D conv?

[seominseok0429]

ex ) class Conv3d(nn.Conv3d):

def __init__(self, in_channels, out_channels, kernel_size, stride=1,
             padding=0, dilation=1, groups=1, bias=True):
    super(Conv2d, self).__init__(in_channels, out_channels, kernel_size, stride,
             padding, dilation, groups, bias)

def forward(self, x):
    weight = self.weight
    weight_mean = weight.mean(dim=1, keepdim=True).mean(dim=2,
                              keepdim=True).mean(dim=3, keepdim=True).mean(dim=4, keepdim=True)
    weight = weight - weight_mean
    std = weight.view(weight.size(0), -1).std(dim=1).view(-1, 1, 1, 1, 1) + 1e-5
    weight = weight / std.expand_as(weight)
    return F.conv3d(x, weight, self.bias, self.stride, self.padding, self.dilation, self.groups)

[joe-siyuan-qiao]

Sure. WS should be also applicable to 3D convolution. Thanks.

[seominseok0429]

Thank you very much for your kind reply.

[shakjm]

ex ) class Conv3d(nn.Conv3d):

def __init__(self, in_channels, out_channels, kernel_size, stride=1,
             padding=0, dilation=1, groups=1, bias=True):
    super(Conv2d, self).__init__(in_channels, out_channels, kernel_size, stride,
             padding, dilation, groups, bias)

def forward(self, x):
    weight = self.weight
    weight_mean = weight.mean(dim=1, keepdim=True).mean(dim=2,
                              keepdim=True).mean(dim=3, keepdim=True).mean(dim=4, keepdim=True)
    weight = weight - weight_mean
    std = weight.view(weight.size(0), -1).std(dim=1).view(-1, 1, 1, 1, 1) + 1e-5
    weight = weight / std.expand_as(weight)
    return F.conv3d(x, weight, self.bias, self.stride, self.padding, self.dilation, self.groups)

Hi did you manage to run this in 3D convolution? I edited his codes to have the same exact one as yours, but I had errors running it in pytorch. I figured it might because I coded it wrongly.

Thank you.

[Erotemic]

Note, you can more cleanly implement the averaging step via: weight.mean(dim=(1, 2, 3, 4), keepdim=True).