Utilize source field for confidence and answered #264
Conversation
brandon-groundlight
requested review from
tyler-romero,
timmarkhuff and
f-wright
|
Looks good to me. Should we have a test for timing? For example, run The test you added ensures that the query is answered by the time the function call completes, but I don't see anything checking if the function call returned in a reasonable amount of time and didn't just wait the full patience time. |
| @@ -64,21 +64,16 @@ def iq_is_confident(iq: ImageQuery, confidence_threshold: float) -> bool: | |||
| The only subtletie here is that currently confidence of None means | |||
| human label, which is treated as confident. | |||
| """ | |||
| if iq.result.confidence is None: | |||
There was a problem hiding this comment.
not related to your PR but could you change subtletie to subtlety above?
There was a problem hiding this comment.
A subtle change
We've considered this before, but we've decided that such tests should live in our other testing code, and not inside the SDK tests. |
…thon-sdk into use_source_for_confidence
There's a bug currently for counting + multiclass where confidence can be below 0.5, which throws off previous assumptions.