Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Difference between 0724 and 0424 7B models #746

Open
jiahai-feng opened this issue Nov 13, 2024 · 0 comments
Open

Difference between 0724 and 0424 7B models #746

jiahai-feng opened this issue Nov 13, 2024 · 0 comments
Labels
type/documentation An issue or pull request related to documentation

Comments

@jiahai-feng
Copy link

📚 The doc issue

What is the difference between the 0724 and 0424 model? I can't find documentation any where. It seems like the official config files are identical. Looking at the intermediate checkpoints, it looks like 0724 is a continuation of 0424, resuming from the preannealing checkpoint. If so, what is the LR schedule for the continuation, and what is the additional dataset?

Suggest a potential alternative/fix

No response
@jiahai-feng jiahai-feng added the type/documentation An issue or pull request related to documentation label Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/documentation An issue or pull request related to documentation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jiahai-feng