Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

A bug in the 'extra_grad_norm_sq' func #14

Open
JiayiFeng opened this issue Jan 30, 2023 · 0 comments · May be fixed by #16
Open

A bug in the 'extra_grad_norm_sq' func #14

JiayiFeng opened this issue Jan 30, 2023 · 0 comments · May be fixed by #16

Comments

@JiayiFeng
Copy link

In the extra_grad_norm_sq func, the self.shared_weights is enumerated to calculate the extra_norm_sq:

varuna/varuna/varuna.py

Line 657 in a2145cc

for i,w in enumerate(self.shared_weights):

However, if self.shared_weights is None, which is not enumeratable, it will throw an exception and make training process crash.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@JiayiFeng