pdb at line 51 in the file metric_calculators.py

[nsaadati]

hi can you please expline why you put this condition at line 51 in the file metric_calculators.py, I'm getting this error a lot and im not sure what is the resaon for that
def finalize(self, numel, eps=1e-4):
self.outer /= numel
self.mean /= numel
self.std /= numel
cov = self.outer - torch.outer(self.mean, self.mean)
if torch.isnan(cov).any():
breakpoint()
if (torch.diagonal(cov) < 0).sum():
pdb.set_trace()
return cov

[ansharora7]

Hi, I am facing the same problem. Did you find the solution?

[gstoica27]

Hi all,

Are you still facing this issue? If so, is it with models from our experiment settings? Or from new ones that you have implemented?

[adampan0527]

Hi all,

Are you still facing this issue? If so, is it with models from our experiment settings? Or from new ones that you have implemented?

Hi, I met same issue, the model is from your experiment settings(resnet20), nothing changed, but found cov has 1 element <0, can you tell me how can covariance<0 happen? And what's the problem may be.
I trained another pair of models, and this time everything goes well, is this problem happens randomly?

[gstoica27]

What's the value of the element that is < 0? Is it near zero? I'm wondering if it's because of numerical precision.

[adampan0527]

What's the value of the element that is < 0? Is it near zero? I'm wondering if it's because of numerical precision.

Really close, about 1e-6 or -7, I agree with numerical precision problem, thx a lot