Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Shopee-SlimMoA-v1 to AlpacaEval #398

Merged
merged 9 commits into from
Aug 26, 2024
Merged

Conversation

LLM-Alignment-sh
Copy link
Contributor

We have built and examined a new faster and more accurate MoA method, could you help us get included in the leaderboard? Thanks!

                                    length_controlled_winrate  win_rate  standard_error  n_total  avg_length
Shopee-SlimMoA-v1                                       77.45     75.61            1.27      805        1994
gpt-4o-2024-05-13                                       57.46     51.33            1.47      805        1873
gpt-4-turbo-2024-04-09                                  55.02     46.12            1.47      805        1802
gpt-4o-mini-2024-07-18                                  50.73     44.65            1.46      805        1861
gpt4_1106_preview                                       50.00     50.00            0.00      805        2049
claude-3-opus-20240229                                  40.51     29.11            1.39      805        1388
Meta-Llama-3.1-405B-Instruct-Turbo                      39.26     39.11            1.43      805        1988
Meta-Llama-3.1-70B-Instruct-Turbo                       38.06     39.13            1.43      805        2044
claude-3-sonnet-20240229                                34.87     25.56            1.34      805        1420
Meta-Llama-3-70B-Instruct                               34.42     33.18            1.39      805        1919
gemini-pro                                              24.38     18.18            1.16      805        1456
Mixtral-8x7B-Instruct-v0.1                              23.69     18.26            1.19      805        1465
Meta-Llama-3-8B-Instruct                                22.92     22.57            1.26      805        1899
Meta-Llama-3.1-8B-Instruct-Turbo                        20.85     21.84            1.25      802        2181
Mistral-7B-Instruct-v0.2                                17.11     14.72            1.08      805        1676
alpaca-7b                                                5.88      2.59            0.49      805         396

@YannDubs
Copy link
Collaborator

Great job @LLM-Alignment-sh 💯

@YannDubs YannDubs merged commit 4da8a95 into tatsu-lab:main Aug 26, 2024
2 checks passed
LLM-Alignment-sh added a commit to LLM-Alignment-sh/alpaca_eval that referenced this pull request Aug 28, 2024
* update

* update

* update

* update

* update

* update

* update

* update

---------

Co-authored-by: qingtao.yu <[email protected]>
Co-authored-by: Yann Dubois <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants