Skip to content

Files

Latest commit

e0a0886 · Sep 14, 2024

History

History
3 lines (3 loc) · 264 Bytes

README.md

File metadata and controls

3 lines (3 loc) · 264 Bytes

Recreating gpt2

About

Recreated OpenAI's GPT-2 through looking over the GPT-2 and GPT-3 papers and following Andrej Karpathy's Make More series. Trained the model on 8 H100s rented through Lambda Labs achieving a lower final loss than OpenAI's original GPT-2.