Add completeness judge #1410
Add completeness judge #1410
Conversation
Signed-off-by: Yoav Katz <[email protected]>
| }, | ||
| "generic_inference_engine": { | ||
| "model_name": "generic", | ||
| "inference_model": (GenericInferenceEngine()), |
There was a problem hiding this comment.
There is now a simpler way to support multiple inference engines in the judge using one consistent API, and that's with CrossProviderInferenceEngine.
https://www.unitxt.ai/en/latest/docs/inference.html#creating-a-cross-api-engine
There was a problem hiding this comment.
It uses the standard OpenAI params names.
There was a problem hiding this comment.
ok. I updated the engine to CrossProvider. Pls check now.
|
Please see these formatting errors: prepare/templates/response_assessment/judges/completeness/v5.py:6:6056: RUF001 String contains ambiguous |
Add LLM-based judge to check for completeness: if the response is complete with respect to the information in the document.