Skip to content

Commit

Permalink
Add command to open notebooks
Browse files Browse the repository at this point in the history
  • Loading branch information
motin committed Jan 25, 2024
1 parent d2767f8 commit 249e9f2
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 2 deletions.
2 changes: 1 addition & 1 deletion automation-api/notebooks/exploration-notebook.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
# extension: .py
# format_name: light
# format_version: '1.5'
# jupytext_version: 1.14.7
# jupytext_version: 1.15.2
# kernelspec:
# display_name: gapminder-ai-automation-api
# language: python
Expand Down
3 changes: 3 additions & 0 deletions automation-api/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,9 @@ test = ["pytest", "lint"]
pytest = "pytest --cov=lib --verbose tests/"
lint = { shell = "find . -type d -name '.venv*' -prune -o -type f -name '*.py' -print0 | xargs -0 pre-commit run --files" }
install_kernel = "python -m ipykernel install --user --name gapminder-ai-automation-api"
set_jupytext_as_default_viewer = "jupytext-config set-default-viewer"
jupyter = 'jupyter-notebook'
notebooks = ["install_kernel", "set_jupytext_as_default_viewer", "jupyter"]
fetch_questions = "python yival_experiments/scripts/fetch_questions.py"
generate_experiment_config = "python yival_experiments/scripts/generate_experiment_config.py"
generate_results_file = "python yival_experiments/scripts/generate_results.py"
Expand Down
10 changes: 9 additions & 1 deletion automation-api/yival_experiments/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,15 @@ TODO: We can add a custom evaluator in Yival to calculate the final scores.

## 8. Calculate scores, upload results to AI Eval Spreadsheet

Two notebooks in notebooks/ directory are provided for calculating scores.
Two notebooks in `./yival_experiments/notebooks/` directory are provided for calculating scores.

- final_scores.py: calculate a final score for each model and prompt
- upload_to_ai_eval_sheet.py: generate the result table and upload to the `Latest Results` sheet in AI Eval Spreadsheet

Start Jupyter:

```shell
poe notebooks
```

Then open the notebooks in the browser and run them.

0 comments on commit 249e9f2

Please sign in to comment.