Add automatic evaluation with LLM-as-a-Judge, LangSmith export, and SGI evaluation

[FelixTJDietrich]

Motivation and Context

We want an option to systematically evaluate the modules without manual labor.

closes #84
closes #36

Description

• Add /evaluation endpoint to Athena for returning arbitrary evaluation results
• Add example of how this /evaluation endpoint would like to module_example (for programming exercises)
• Add this new module request to the Playground
• Run evaluation for all healthy modules of the same type that support evaluation in evaluation mode
• Display progress of automatic evaluation (and move "Start Generating" button)
• Add LLM-as-a-judge for text modules
• Disable LLM-as-a-judge using environment variable
• Add token usage and response times in evaluation response (coming from LangSmith)
• Add eval code for SGI evaluation

Note: I omitted the "module_text_llm estimated Accepted with Minor modifications" text from the UI since it is kind of ugly to add and does not give immediate benefit. In the future one might make automatic results available for the inline feedback view but this depends also on future experiment designs.

Steps for Testing

1. Run an evaluation with module_text_llm
2. After everything is done click export, it should export the automatic_evaluation now as a separate file
3. File should contain LLM-as-a-judge, llm_statistics (from LangSmith, I added my key to the env), and feedback_statistics
4. Cancel experiment
5. Import results and automatic evaluation and check if it works

Screenshots

Docs:

Not Started

Sent Training Feedback

Generated Suggestions

Finished

[FelixTJDietrich]

Seems like I did not validate the structured_grading_instruction_id, it should be fine now.

[FelixTJDietrich]

Works for me on the test server now :)

[FelixTJDietrich]

I also added now some docs:

[pal03377]

The exported files look like expected. I also cannot find any issues (neither in the code nor in my tests) ✅