How to upload EvaluationResults via API/SDK?

[DevonPeroutky]

I have an existing set of evaluations that I would like to publish to Weave, but they don't fit in the Evaluation workflow that is outlined in the documentation like:

Evaluation(
    dataset=examples, scorers=[match_score1]
).evaluate()

For various reasons, I need to execute my predictions and scoring outside of of weave.Evaluation. I can wrap my prediction in a weave.op, but because the scoring for this evaluation is formal verification that involves spinning up a docker container and then writing results to a file locally, I can't simply pass in a dataset, model, and scorers to Evaluation.

Is there a way to create an evaluation and submit EvaluationResults via the SDK, outside of Evaluation(...).evaluate?

[gtarpenning]

Hi! Thanks for the feature request, this will be brought up for internal triage (I know that this is already in the backlog). There are possibly ways around this, like writing the results of your evaluation to disk, then spinning up a little simulation script (with weave ops) that reads those values in scorer functions and "predicts" just by looking at the source of truth.

Stay tuned for a first-class way of handling this though, it's a great idea.