Improve the plumbing and documentation for some of the ask* methods #125

tyler-romero · 2023-10-18T21:33:59Z

Address a some documentation papercuts for the new ask_* methods. Plus, improve plumbing of arguments to submit_image_query and to wait_for_confident_result.

tyler-romero · 2023-10-18T21:35:50Z

src/groundlight/client.py

-        """Evaluates an image with Groundlight waiting until an answer above the confidence threshold
-            of the detector is reached or the wait period has passed.
+        """
+        Evaluates an image with Groundlight waiting until an answer above the confidence threshold


Address some formatting issues
https://code.groundlight.ai/python-sdk/api-reference-docs/#groundlight.client.Groundlight.ask_confident

sunildkumar · 2023-10-18T21:45:01Z

src/groundlight/client.py

+        :param patience_time: How long to wait (in seconds) for a confident answer for this image query.
+            The longer the patience_time, the more likely Groundlight will arrive at a confident answer.
+            Within patience_time, Groundlight will update ML predictions based on stronger findings,
+            and, additionally, Groundlight will prioritize human review of the image query if necessary.
+            This is a soft server-side timeout. If not set, use the detector's patience_time.
+        :type patience_time: float
+
+        :param confidence_threshold: The confidence threshold to wait for.
+            If not set, use the detector's confidence threshold.
+        :type confidence_threshold: float
+


I absolutely love this description of patience time

src/groundlight/client.py

sunildkumar · 2023-10-18T21:50:44Z

src/groundlight/client.py

+        if confidence_threshold is None:
+            if isinstance(image_query, str):
+                image_query = self.get_image_query(image_query)
+            confidence_threshold = self.get_detector(image_query.detector_id).confidence_threshold


my only concern here is that this introduces another request from the server. I recall @robotrapta suggesting that we should try to minimize this to minimize overall latency, but maybe I am mistaken. I am guessing that the alternative would be to force the user to provide the detector as an argument to the function which is kinda ugly.

We actually call get_image_query if image_query is a string as the very first step in _wait_for_result, so this just brings the call forwards a bit.

sunildkumar · 2023-10-18T21:52:05Z

src/groundlight/client.py

-        def confidence_above_thresh(iq):
-            return iq_is_confident(iq, confidence_threshold=confidence_threshold)
-
+        confidence_above_thresh = partial(iq_is_confident, confidence_threshold=confidence_threshold)


thanks for teaching me about partial - this is very haskell-y.

Haha its honestly really nice sometimes. I love it because its a bit of a functional holdover

sunildkumar · 2023-10-18T21:53:10Z

src/groundlight/client.py

@@ -623,6 +662,7 @@ def add_label(self, image_query: Union[ImageQuery, str], label: Union[Label, str
        else:
            image_query_id = str(image_query)
            # Some old imagequery id's started with "chk_"
+            # TODO: handle iqe


I'm not sure what you mean here, so I'm concerned that others might not as well, but maybe I'm just out of the loop.

Edge detectors return image queries with the prefix iqe_, so we need to work on a way to handle that sanely

This what the edge server will return, right? This just opens up the sdk to work with the edge

Exactly Brandon, just updated the comment to be a bit more clear

sunildkumar · 2023-10-18T21:54:21Z

docs/docs/building-applications/5-async-queries.md

+image_query = gl.wait_for_confident_result(id=image_query.id)  # Poll for a confident result from Groundlight
+result = image_query.result


I like showing this as the default use case.

src/groundlight/client.py

sunildkumar

nice! Thanks for your feedback on all of the documentation this afternoon.

Co-authored-by: Sunil Kumar <[email protected]>

brandon-groundlight

Did we also want to attach confidence_threshold to the object sent to the BE, or does that require some BE work first?

brandon-groundlight · 2023-10-18T21:58:34Z

src/groundlight/client.py

@@ -623,6 +662,7 @@ def add_label(self, image_query: Union[ImageQuery, str], label: Union[Label, str
        else:
            image_query_id = str(image_query)
            # Some old imagequery id's started with "chk_"
+            # TODO: handle iqe


This what the edge server will return, right? This just opens up the sdk to work with the edge

tyler-romero · 2023-10-18T22:42:10Z

That will require a bit of BE work Brandon, this is just the intermediate solution

tyler-romero and others added 4 commits October 18, 2023 14:16

Improve the plumbing for some of the ask* methods

f8f3245

Small doc fixup

5b7c63e

Automatically reformatting code

35013f7

Fixups

c661b67

tyler-romero commented Oct 18, 2023

View reviewed changes

Touch ups

54885dd

tyler-romero requested review from sunildkumar, robotrapta and brandon-groundlight and removed request for sunildkumar October 18, 2023 21:39

lint fixes

945a4bc

sunildkumar reviewed Oct 18, 2023

View reviewed changes

tyler-romero marked this pull request as ready for review October 18, 2023 21:45

sunildkumar reviewed Oct 18, 2023

View reviewed changes

src/groundlight/client.py Outdated Show resolved Hide resolved

sunildkumar reviewed Oct 18, 2023

View reviewed changes

src/groundlight/client.py Outdated Show resolved Hide resolved

sunildkumar approved these changes Oct 18, 2023

View reviewed changes

Update src/groundlight/client.py

867ffac

Co-authored-by: Sunil Kumar <[email protected]>

brandon-groundlight approved these changes Oct 18, 2023

View reviewed changes

address PR

76212eb

increment version number

8de7f3a

tyler-romero merged commit ff6da32 into main Oct 18, 2023
8 checks passed

tyler-romero deleted the tyler/ask-method-improvements-and-documentation branch October 18, 2023 22:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve the plumbing and documentation for some of the ask* methods #125

Improve the plumbing and documentation for some of the ask* methods #125

tyler-romero commented Oct 18, 2023 •

edited

Loading

tyler-romero Oct 18, 2023

sunildkumar Oct 18, 2023

sunildkumar Oct 18, 2023

tyler-romero Oct 18, 2023

sunildkumar Oct 18, 2023

sunildkumar Oct 18, 2023

tyler-romero Oct 18, 2023

sunildkumar Oct 18, 2023

tyler-romero Oct 18, 2023

brandon-groundlight Oct 18, 2023

sunildkumar Oct 18, 2023

tyler-romero Oct 18, 2023

sunildkumar Oct 18, 2023

sunildkumar left a comment

brandon-groundlight left a comment

brandon-groundlight Oct 18, 2023

tyler-romero commented Oct 18, 2023

		image_query = gl.wait_for_confident_result(id=image_query.id) # Poll for a confident result from Groundlight
		result = image_query.result

Improve the plumbing and documentation for some of the ask* methods #125

Improve the plumbing and documentation for some of the ask* methods #125

Conversation

tyler-romero commented Oct 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sunildkumar left a comment

Choose a reason for hiding this comment

brandon-groundlight left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tyler-romero commented Oct 18, 2023

tyler-romero commented Oct 18, 2023 •

edited

Loading