Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add automatic evaluation with LLM-as-a-Judge, LangSmith export, and SGI evaluation #174

Merged
merged 61 commits into from
Nov 12, 2023
Merged

Add automatic evaluation with LLM-as-a-Judge, LangSmith export, and SGI evaluation #174

merged 61 commits into from
Nov 12, 2023

Commits on Nov 5, 2023

  1. add endpoint

    @FelixTJDietrich
    FelixTJDietrich committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    3e336dc View commit details
    Browse the repository at this point in the history

  2. add evaluation_provider

    @FelixTJDietrich
    FelixTJDietrich committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    2fb28f0 View commit details
    Browse the repository at this point in the history

  3. add new line

    @FelixTJDietrich
    FelixTJDietrich committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    38f6055 View commit details
    Browse the repository at this point in the history

  4. add evaluation_provider to export

    @FelixTJDietrich
    FelixTJDietrich committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    92ca4ed View commit details
    Browse the repository at this point in the history

  5. add example evaluation endpoint

    @FelixTJDietrich
    FelixTJDietrich committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    433cd7f View commit details
    Browse the repository at this point in the history

  6. add playground ui

    @FelixTJDietrich
    FelixTJDietrich committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    9b4e2c9 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2023

  1. add automatic evaluation

    @FelixTJDietrich
    FelixTJDietrich committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    5f39b8c View commit details
    Browse the repository at this point in the history

  2. add automatic evaluation

    @FelixTJDietrich
    FelixTJDietrich committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    2667cab View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2023

  1. add UI changes

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    7dbb316 View commit details
    Browse the repository at this point in the history

  2. fix color

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    5053a4a View commit details
    Browse the repository at this point in the history

  3. add evaluation model

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    e9bcb26 View commit details
    Browse the repository at this point in the history

  4. add llm as a judge

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    cabed57 View commit details
    Browse the repository at this point in the history

  5. fix ui issue and some var naming

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    2595d5c View commit details
    Browse the repository at this point in the history

  6. fix line break

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    2739757 View commit details
    Browse the repository at this point in the history

  7. add langsmith logging

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    6b383e3 View commit details
    Browse the repository at this point in the history

  8. inline statistics

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    e46df2c View commit details
    Browse the repository at this point in the history

  9. add sgi evaluation

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    8d50922 View commit details
    Browse the repository at this point in the history

  10. refactor

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    a753b8a View commit details
    Browse the repository at this point in the history

  11. remove unused

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    fa0bde5 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'develop' of https://github.com/ls1intum/Athena into fea… 

    …ture/automatic-evaluation
    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    daff54a View commit details
    Browse the repository at this point in the history

  13. update ini

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    a4d7d8c View commit details
    Browse the repository at this point in the history

  14. only use selected modules

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    fd4fdab View commit details
    Browse the repository at this point in the history

  15. remove skip

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    f2c1736 View commit details
    Browse the repository at this point in the history

  16. add endpoint

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    33f2a2b View commit details
    Browse the repository at this point in the history

  17. add evaluation_provider

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    368ebcc View commit details
    Browse the repository at this point in the history

  18. add new line

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    5c68106 View commit details
    Browse the repository at this point in the history

  19. add evaluation_provider to export

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    7afd655 View commit details
    Browse the repository at this point in the history

  20. add example evaluation endpoint

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    1926144 View commit details
    Browse the repository at this point in the history

  21. add playground ui

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    db5e518 View commit details
    Browse the repository at this point in the history

  22. add automatic evaluation

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    2da4391 View commit details
    Browse the repository at this point in the history

  23. add automatic evaluation

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    a8589dc View commit details
    Browse the repository at this point in the history

  24. add UI changes

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    39c729d View commit details
    Browse the repository at this point in the history

  25. fix color

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    fdb073d View commit details
    Browse the repository at this point in the history

  26. add evaluation model

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    d0838f5 View commit details
    Browse the repository at this point in the history

  27. add llm as a judge

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    05608aa View commit details
    Browse the repository at this point in the history

  28. fix ui issue and some var naming

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    c68ba0f View commit details
    Browse the repository at this point in the history

  29. fix line break

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    afb1892 View commit details
    Browse the repository at this point in the history

  30. add langsmith logging

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    2a1d4b6 View commit details
    Browse the repository at this point in the history

  31. inline statistics

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    e845e1f View commit details
    Browse the repository at this point in the history

  32. add sgi evaluation

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    d17e486 View commit details
    Browse the repository at this point in the history

  33. refactor

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    5266461 View commit details
    Browse the repository at this point in the history

  34. remove unused

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    9f7494a View commit details
    Browse the repository at this point in the history

  35. update ini

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    d44178b View commit details
    Browse the repository at this point in the history

  36. only use selected modules

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    42f8210 View commit details
    Browse the repository at this point in the history

  37. remove skip

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    cc5693d View commit details
    Browse the repository at this point in the history

  38. add retries

    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    9462ec5 View commit details
    Browse the repository at this point in the history

  39. Merge branch 'feature/automatic-evaluation' of https://github.com/ls1… 

    …intum/Athena into feature/automatic-evaluation
    @FelixTJDietrich
    FelixTJDietrich committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    c6609af View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. enable example module evaluation support for now

    @FelixTJDietrich
    FelixTJDietrich committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    292d588 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into feature/automatic-evaluation

    @pal03377
    pal03377 authored Nov 8, 2023
    Configuration menu
    Copy the full SHA
    fea2d85 View commit details
    Browse the repository at this point in the history

  3. fix filter

    @FelixTJDietrich
    FelixTJDietrich committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    5548293 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2023

  1. Merge branch 'feature/automatic-evaluation' of https://github.com/ls1… 

    …intum/Athena into feature/automatic-evaluation
    @FelixTJDietrich
    FelixTJDietrich committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    22941ee View commit details
    Browse the repository at this point in the history

  2. implement feedbacl

    @FelixTJDietrich
    FelixTJDietrich committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    8de0ee6 View commit details
    Browse the repository at this point in the history

  3. update retry

    @FelixTJDietrich
    FelixTJDietrich committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    0edea59 View commit details
    Browse the repository at this point in the history

  4. validate grading instruction id

    @FelixTJDietrich
    FelixTJDietrich committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    b4b529e View commit details
    Browse the repository at this point in the history

  5. add additional check

    @FelixTJDietrich
    FelixTJDietrich committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    331f353 View commit details
    Browse the repository at this point in the history

  6. fix index

    @FelixTJDietrich
    FelixTJDietrich committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    4eac295 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2023

  1. add docs

    @FelixTJDietrich
    FelixTJDietrich committed Nov 11, 2023
    Configuration menu
    Copy the full SHA
    0fe6e6d View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' of https://github.com/ls1intum/Athena into fea… 

    …ture/automatic-evaluation
    @FelixTJDietrich
    FelixTJDietrich committed Nov 11, 2023
    Configuration menu
    Copy the full SHA
    c6e08df View commit details
    Browse the repository at this point in the history

  3. Merge branch 'develop' into feature/automatic-evaluation

    @FelixTJDietrich
    FelixTJDietrich authored Nov 11, 2023
    Configuration menu
    Copy the full SHA
    c384232 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2023

  1. fix text module

    @FelixTJDietrich
    FelixTJDietrich committed Nov 12, 2023
    Configuration menu
    Copy the full SHA
    2b7f212 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature/automatic-evaluation' of https://github.com/ls1… 

    …intum/Athena into feature/automatic-evaluation
    @FelixTJDietrich
    FelixTJDietrich committed Nov 12, 2023
    Configuration menu
    Copy the full SHA
    fe10281 View commit details
    Browse the repository at this point in the history