Feature - New Metrics (#335)

* Update model methods to run background task when creating the endpoint and creating score endpoint for runpod to use * Handle error from Runpod, URL inference from yaml file * Update backend/app/api/endpoints/base/score.py Co-authored-by: Rafael Mosquera <[email protected]> * Rename method * rename endpoint * Print background tasks and correct typo * Correct typos and correct schemas * model uid * remove c * New metrics for MLSuperb challenge * Correct pretty name * replace standard accuracy for standard LID Accuracy, and dialect accuracy for dialect LID accuracy * change metric name to show in front end --------- Co-authored-by: Rafael Mosquera <[email protected]>
mlcommons · Feb 6, 2025 · 5f33b10 · 5f33b10
1 parent 13fe3c1
commit 5f33b10
Show file tree

Hide file tree

Showing 2 changed files with 2 additions and 2 deletions.
diff --git a/api/evaluation/metrics/metrics.py b/api/evaluation/metrics/metrics.py
@@ -175,7 +175,7 @@ def get_STD_CER_meta(task=None):
 def get_CER_15_WORSE_meta(task=None):
     return {
         "unit": "%",
-        "pretty_name": "CER 15 WORSE",
+        "pretty_name": "CER 15 WORST",
         "utility_direction": -1,
         "offset": 0,
     }

diff --git a/backend/app/domain/services/builder_and_evaluation/eval_utils/metrics.py b/backend/app/domain/services/builder_and_evaluation/eval_utils/metrics.py
@@ -205,7 +205,7 @@ def get_STD_CER_meta(task=None):
 def get_CER_15_WORSE_meta(task=None):
     return {
         "unit": "%",
-        "pretty_name": "CER_15_WORSE",
+        "pretty_name": "CER_15_WORST",
         "utility_direction": -1,
         "offset": 0,
     }