Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow dynamically created tasks in llm_as_judge #1292

Open
lilacheden opened this issue Oct 16, 2024 · 0 comments
Open

Allow dynamically created tasks in llm_as_judge #1292

lilacheden opened this issue Oct 16, 2024 · 0 comments

Comments

@lilacheden
Copy link
Member

Currently the task attribute of LLMAsJudgeBase is a string that accepts either the task suffix (for LLMAsJudge) or the task full path (for TaskBasedLLMAsJudge).
Need to change its type to Task and initialize with either the full task path or a Task object, as done with the template attribute of the same class.
This requires a small but non backward compatible change in LLMAsJudge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant