求助：在执行 loss.backwad() 时发生 RuntimeError， #194

Maninnlack · 2022-03-31T03:26:34Z

Maninnlack
Mar 31, 2022

我使用spikingjelly框架搭建了一个网络，但是在训练到第二个batch的时候，执行 loss.backwad() 时会发生报错。
为了检查是否是我网络搭建的问题，我搭建了一个很简单的测试模型，但是还是会报错。
我自己搭建的模型以及测试代码如下：

import torch
import torch.nn as nn

from utils import neuron, surrogate

class Net(nn.Module):
    def __init__(self) -> None:
        super(Net, self).__init__()

        self.conv1 = nn.Sequential(
            nn.Conv2d(3, 8, kernel_size=3,padding=1),
            nn.BatchNorm2d(8),
            neuron.IFNode(surrogate_function=surrogate.ATan()),
            nn.Conv2d(8, 8, kernel_size=3,padding=1),
            nn.BatchNorm2d(8),
            neuron.IFNode(surrogate_function=surrogate.ATan())
        )
        self.fc1 = nn.Sequential(
            nn.Flatten(),
            nn.Linear(8 * 16 * 16, 1, bias=False),
            neuron.LIFNode(tau=2.0, surrogate_function=surrogate.ATan())
        )
    
    def forward(self, x):
        c1 = self.conv1(x)
        output = self.fc1(c1)
        return output


def main():
    
    net = Net()
    optimizer = torch.optim.Adam(net.parameters(), lr=1e-3)
    criterion = nn.MSELoss()

    for i in range(5):
        x = torch.rand([3, 3, 16, 16])
        y = torch.rand([3, 1])

        output = net(x)
        loss = criterion(output, y)
        print(loss.item())
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()


if __name__ == '__main__':
    main()

网络在输出两个loss之后会报错：

0.618949830532074
0.12842749059200287
Traceback (most recent call last):
  File "./UNet_INN/surrogate_test.py", line 49, in <module>
    main()
  File "./UNet_INN/surrogate_test.py", line 44, in main
    loss.backward()
  File "/mnt/data/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/mnt/data/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/autograd/__init__.py", line 130, in backward
    Variable._execution_engine.run_backward(
  File "/mnt/data/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/autograd/function.py", line 89, in apply
    return self._forward_cls.backward(self, *args)  # type: ignore
  File "./UNet_INN/utils/surrogate.py", line 676, in backward
    if ctx.saved_tensors.__len__() == 1:
RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.

我实在是不知道该怎么办了，所以想在这求助一下。
谢谢！