Skip to content

Commit

Permalink
add todo
Browse files Browse the repository at this point in the history
  • Loading branch information
semio committed Nov 11, 2023
1 parent 95e6447 commit d9c18a5
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions yival_experiments/output/generate_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
# rs[1].asdict()

# We will combine all pickle files in output dir and calculate final scores.
# TODO: follow the format in `Latest Results` sheet of AI eval spreadsheet
# 1. Store all responses into excel file.
output_list = []

Expand Down Expand Up @@ -45,6 +46,7 @@


# 2. calculate a final score per model configuration
# TODO: I think it's possible to convert these into a Yival Evaluator.
def is_correct_p(round_results):
c = Counter(round_results)
top2 = c.most_common(2)
Expand Down

0 comments on commit d9c18a5

Please sign in to comment.