-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
turning UNet into Bayesian UNet #25
Comments
Hi, self.out = nn.Conv3d( |
Hi, thank you for the suggestion but it does not work. I got the same effect of blowing tensors. |
Sorry for the delayed response, I had completely missed the initial issue over Neurips. Does the error occur right away or after a few training iterations? And have you by any chance checked if you are getting NaNs or negative values for the scales of the Normal distribution? Either would fail the constraint check iirc. |
Hi. The error occurs right after the start of the training. I got something like this: ValueError: Expected parameter scale (Tensor of shape (4, 2, 32, 32, 16)) of distribution Normal(loc: torch.Size([4, 2, 32, 32, 16]), scale: torch.Size([4, 2, 32, 32, 16])) to satisfy the constraint GreaterThan(lower_bound=0.0), but found invalid values: I expect that something may be wrong with my model. I simply copied your regression example and replaced your model with UNet which is a little bit more complicated that regression. The complete code is in the enclosed archive - the code generates its own data so it can be just run. |
Ok interesting. My guess would be that this is an issue with initialisation since you are getting fairly extreme values for the standard deviations. If you replace the |
Hi,
I am trying to use your library to turn UNet into a Bayesian Unet. I paste the code below: in the implementation UNet works as a pixel-to-pixel translator for 3D data. The code follows your regression example (as I am also doing regression but for higher dimensional data).
When I run the code I got a run-time error:
ValueError: Expected parameter scale (Tensor of shape (4, 1, 32, 32, 16)) of distribution Normal(loc: torch.Size([4, 1, 32, 32, 16]), scale: torch.Size([4, 1, 32, 32, 16])) to satisfy the constraint GreaterThan(lower_bound=0.0), but found invalid values...
I expect that the problem is with a wrong selection of the prior and/or guide. I would appreciate any suggestion which will make the model to learn.
Regards,
Zbisław
The code:
from functools import partial
import torch
import torch.nn as nn
import torch.utils.data as data
import pyro
import pyro.distributions as dist
import tyxe
def double_convolution(in_channels, out_channels):
"""
In the original paper implementation, the convolution operations were
not padded but we are padding them here. This is because, we need the
output result size to be same as input size.
"""
conv_op = nn.Sequential(
nn.Conv3d(in_channels, out_channels, kernel_size=3, padding=1),
nn.ReLU(inplace=True),
nn.Conv3d(out_channels, out_channels, kernel_size=3, padding=1),
nn.ReLU(inplace=True)
)
return conv_op
class UNet(nn.Module):
def init(self, num_classes):
super(UNet, self).init()
################################################################################
if name == 'main':
The text was updated successfully, but these errors were encountered: