Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature - New Metrics #335

Merged
merged 21 commits into from
Feb 6, 2025
Merged

Feature - New Metrics #335

merged 21 commits into from
Feb 6, 2025

Conversation

shincap8
Copy link
Contributor

@shincap8 shincap8 commented Feb 6, 2025

Context

  • Task owner ask to change name of metric.

Changes Made

  1. change pretty name of the metric.
  • This would allow the challenge owner to see the change he wanted with minimum change from our side.

shincap8 and others added 21 commits January 16, 2025 19:25
@shincap8
Update model methods to run background task when creating the endpoin…
04cd514 
…t and creating score endpoint for runpod to use
@shincap8
Handle error from Runpod, URL inference from yaml file
d86ae16
@shincap8 @remg1997
Update backend/app/api/endpoints/base/score.py
8794693 
Co-authored-by: Rafael Mosquera <[email protected]>
@shincap8
Rename method
fc08634
@shincap8
Merge with cloud branch
aa0f022
@shincap8
rename endpoint
971b986
@shincap8
Print background tasks and correct typo
476d4e4
@shincap8
resolve merge conflict
9ed5f20
@shincap8
Merge branch 'main' of https://github.com/mlcommons/dynabench into Fe…
326acbb 
…ature/RunPod
@shincap8
Correct typos and correct schemas
68f126e
@shincap8
model uid
c2638b9
@shincap8
merge with main
605d8fb
@shincap8
remove c
0d74498
@shincap8
Merge branch 'main' of https://github.com/mlcommons/dynabench into Fe…
9c6f088 
…ature/RunPod
@shincap8
Merge branch 'main' of https://github.com/mlcommons/dynabench into Fe…
ac0b90b 
…ature/RunPod
@shincap8
New metrics for MLSuperb challenge
bbb2695
@shincap8
Correct pretty name
6b0f1c6
@shincap8
Merge branch 'main' of https://github.com/mlcommons/dynabench into Fe…
de489f5 
…ature/new_metrics
@shincap8
replace standard accuracy for standard LID Accuracy, and dialect accu…
2cce5d4 
…racy for dialect LID accuracy
@shincap8
Merge branch 'main' of https://github.com/mlcommons/dynabench into Fe…
142c265 
…ature/new_metrics
@shincap8
change metric name to show in front end
1014401
@shincap8 shincap8 requested a review from a team as a code owner February 6, 2025 17:16
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 6, 2025

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@shincap8 shincap8 requested a review from remg1997 February 6, 2025 19:13
@shincap8 shincap8 merged commit 5f33b10 into main Feb 6, 2025
13 checks passed
@shincap8 shincap8 deleted the Feature/new_metrics branch February 6, 2025 22:05
@github-actions github-actions bot locked and limited conversation to collaborators Feb 6, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@remg1997 remg1997 remg1997 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@shincap8 @remg1997