Skip to content

Commit

Permalink
Minor README update
Browse files Browse the repository at this point in the history
  • Loading branch information
dvs23 committed Jul 5, 2024
1 parent 6ee1d3d commit de036f2
Showing 1 changed file with 8 additions and 8 deletions.
16 changes: 8 additions & 8 deletions results/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,16 @@ This folder contains tables with results, generated queries etc. as well as the
- `all-generated-queries-test.csv.zst` - Shows all queries generated during the 3 hours of benchmarking
- `chosen-queries-per-strategy-test.csv` - Shows evaluation statistics per evaluation strategy and question
- `strategy-total-eval-test.csv` - Shows total evaluation stats per strategy
- `29376s-all-train` - Benchmark results for train dataset with 29376 seconds timeout and parallel evaluation of all single-model strategies
- `29376s-all-train/` - Benchmark results for train dataset with 29376 seconds timeout and parallel evaluation of all single-model strategies
- Files analogous to `3h-all-test/`
- `3h-bestonly` - Benchmark results for test dataset with 3 hours timeout only using BestScore evaluation strategy, reducing overhead
- `3h-bestonly/` - Benchmark results for test dataset with 3 hours timeout only using BestScore evaluation strategy, reducing overhead
- Files analogous to `3h-all-test/`
- `top-n-eval` - Evaluation results for taking together the results of the top n models based on performance shown in `29376s-all-train/strategy-total-eval-train.csv` and using the queries of `3h-all-test/all-generated-queries-test.csv.zst` to create results comparable to the other results of `3h-all-test/`
- `top-n-eval/` - Evaluation results for taking together the results of the top n models based on performance shown in `29376s-all-train/strategy-total-eval-train.csv` and using the queries of `3h-all-test/all-generated-queries-test.csv.zst` to create results comparable to the other results of `3h-all-test/`
- `top2-models-eval-on-same-queries.txt` - Logs of evaluating the top 2 models with queries of `3h-all-test/all-generated-queries-test.csv.zst`
- Analogous for remaining files for top3, top5 and top10
- `GPT` - Results of GPT models
- `finetuned-gpt-3.5-turbo` - Results and dataset for finetuned-gpt-3.5-turbo
- `datasets` - Datasets used for fine-tuning, differing w.r.t. usage of lexicon and system prompt used
- `GPT/` - Results of GPT models
- `finetuned-gpt-3.5-turbo/` - Results and dataset for finetuned-gpt-3.5-turbo
- `datasets/` - Datasets used for fine-tuning, differing w.r.t. usage of lexicon and system prompt used
- `train_prompt1.jsonl` - Training dataset for prompt 1 without lexical entries in prompt
- `train_prompt1_lexicon.jsonl` - Training dataset for prompt 1 with lexical entries in prompt
- `valid_prompt1.jsonl` - Validation dataset for prompt 1 without lexical entries in prompt
Expand All @@ -28,12 +28,12 @@ This folder contains tables with results, generated queries etc. as well as the
- `QALD9_finetuned-gpt-3.5-turbo_0-shot_prompt1_test.csv` - Generated responses/queries for test dataset for model trained/finetuned and validated with prompt 1 dataset without lexical entries in prompt
- `QALD9_finetuned-gpt-3.5-turbo_0-shot_prompt1_test_lexicon.csv` - Generated responses/queries for test dataset for model trained/finetuned and validated with prompt 1 dataset with lexical entries in prompt
- Analogous for other prompts
- `gpt-3.5-turbo` - Results for gpt-3.5-turbo
- `gpt-3.5-turbo/` - Results for gpt-3.5-turbo
- `QALD9_gpt-3.5-turbo_0-shot_test.csv` - Generated responses/queries for QALD-9 test dataset without adding lexical entries to prompt
- `QALD9_gpt-3.5-turbo_0-shot_test_lexicon.csv` - Generated responses/queries for QALD-9 test dataset with adding lexical entries to prompt
- `QALD9_gpt-3.5-turbo_0-shot_train.csv` - Generated responses/queries for QALD-9 training dataset without adding lexical entries to prompt
- `QALD9_gpt-3.5-turbo_0-shot_train_lexicon.csv` - Generated responses/queries for QALD-9 training dataset with adding lexical entries to prompt
- `gpt-4` - Results for gpt-4
- `gpt-4/` - Results for gpt-4
- `QALD9_gpt-4_0-shot_test.csv` - Generated responses/queries for QALD-9 test dataset without adding lexical entries to prompt
- `QALD9_gpt-4_0-shot_test_lexicon.csv` - Generated responses/queries for QALD-9 test dataset with adding lexical entries to prompt
- `QALD9_gpt-4_0-shot_train.csv` - Generated responses/queries for QALD-9 training dataset without adding lexical entries to prompt
Expand Down

0 comments on commit de036f2

Please sign in to comment.