Skip to content

Latest commit

 

History

History
22 lines (14 loc) · 1.07 KB

README.md

File metadata and controls

22 lines (14 loc) · 1.07 KB
Raw

THE ADEMAMIX OPTIMIZER: BETTER, FASTER, OLDER

Algo:

image

Experiments from paper:

image

image

image

Reference

Pagliardini, M., Ablin, P., & Grangier, D. (2024). The AdEMAMix Optimizer: Better, Faster, Older. arXiv preprint arXiv:2409.03137. https://arxiv.org/abs/2409.03137

Support

If you find this project valuable, please consider starring it on GitHub. Your support is greatly appreciated, and we welcome any feedback or suggestions through GitHub issues.

For those interested in contributing further, we have a GitHub Sponsors page. Your sponsorship will significantly aid us in maintaining and enhancing this project. Thank you for your support!