Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it right to use torch.mean(s_tgt) when C < B? #13

Open
cantabile-kwok opened this issue Nov 18, 2021 · 1 comment
Open

Is it right to use torch.mean(s_tgt) when C < B? #13

cantabile-kwok opened this issue Nov 18, 2021 · 1 comment

Comments

@cantabile-kwok
Copy link

Hi and I am studying your approach with your implementation. My question is that in your paper you use (Equation 12) to compute the BNM loss, and the divisor is the batch size. But in BNM/DA/BNM/train_image.py L#164 I found that this is done with torch.mean(). Then if the class number is smaller than batch size, the SVD operation will generate a s_tgt with length C instead of B. Wouldn't that be incorrect according to the original equation? Why don't explicitly divide with the batch size?

@cuishuhao
Copy link
Owner

I admit I am a little careless about the weight.
In the equation, batch B is divided.
In the code, I achieve it with min(B, C). L_bnm can be combined with a hyperparameter \lambda, and in this case the actual value of \lambda is changed.
In other practices, I find the value might be better to set as \sqrt(B * C), and the performance can be better with different values of \lambda.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants