Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum - Heeler-Deer #60

Open
heeler-deer opened this issue Aug 29, 2023 · 0 comments
Open

Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum - Heeler-Deer #60

heeler-deer opened this issue Aug 29, 2023 · 0 comments
Labels
4cc09e1d673c7eb5c87944b04d3c2dc6 Gitalk

Comments

@heeler-deer
Copy link
Owner

https://heeler-deer.top/posts/3391/

Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum 本博客暂略公式推导 Summary 一个好的优化器，应该能够容易的逃离saddle(鞍点)，在遇到flat minima时留下；反之，一个坏的优化器，难以逃离saddle,却不乐于呆在flat minima（注意不是逃离flat mi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
4cc09e1d673c7eb5c87944b04d3c2dc6 Gitalk
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@heeler-deer