Some question of KLD

[CXX1113]

KLD = -0.5 / n_nodes * torch.mean(torch.sum(1 + 2 * logvar - mu.pow(2) - logvar.exp().pow(2), 1))
/ n_nodes should be removed or
torch.mean → torch.sum

[AllenWu18]

then is it should be
KLD = -0.5 * torch.mean(torch.sum(1 + 2 * logvar - mu.pow(2) - logvar.exp().pow(2), 1))
or
KLD = -0.5 / n_nodes * torch.sum(torch.sum(1 +2 * logvar - mu.pow(2)-logvar.exp().pow(2) , 1))?

[YH-UtMSB]

@alanlisten @AllenWu18 you both are right about the KLD. The author has clarified that 1 / n_nodes serves as a rescaling parameter (like \beta in beta-vae) to weaken the regularization from KLD. Check this issue: Loss function in optimizer.py #20.

[Dzhilin]

Please tell me what is the meaning of "norm" in the loss function? Looking forward to your reply！