Skip to content

Commit

Permalink
Adds a page on the ask_* methods
Browse files Browse the repository at this point in the history
  • Loading branch information
brandon-groundlight committed Aug 16, 2024
1 parent 2a62912 commit 0dd7e18
Show file tree
Hide file tree
Showing 2 changed files with 101 additions and 54 deletions.
47 changes: 47 additions & 0 deletions docs/docs/getting-started/6-ask-groundlight-methods.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Ask Groundlight Methods

The python-sdk provides a few different methods for submitting images to Groundlight depending on
your use case. All of these methods take an image and an identifier for the detector you want to
send the image to and return an `ImageQuery` object which contains the answer from Groundlight.
The different methods provide different ways to utilize Groundlight's escalation logic.

## `ask_ml`

`ask_ml` is the most direct method for submitting an image query and asks for the fastest available ML answer
from Groundlight. Groundlight's escalation logic will still run in order to improve the model's
confidence for future queries.

`ask_ml` is a good choice in most situations including live monitoring,

## `ask_confident`

`ask_confident` give you access to high quality and human reviewed answers from Groundlight. Rather
than get the first available answer, `ask_confident` will continue to wait as the image makes its
way through the escalation process until either a more powerful model can give a high confidence
answer or a human reviewer can provide a definitive answer. How confident the answer needs to be can
be configured through the `confidence_threshold` parameter.

This is
ideal in critical situations like quality control or safety monitoring where mistakes can be costly
and you can wait a little longer for a more accurate answer. You can also use `ask_confident` as a
filter after recieving an answer from `ask_ml` to provide a level of review before triggering any
operations or contingencies.

(You can program your own operations to be triggered by groundlight, but if you're interested in recieving text message or email notifications see if Groundlight alerts would meet your needs)

## `ask_async`

There are some cases where you may not need an answer from Groundlight right away, often when you're
submitting a batch of images. You may want to provide historical images you have in order to prepare
a detector for future use, or you may want to submit a set of images to multiple detectors
simultaneously before aggregating the answers. In these cases, you can use `ask_async` to submit the
image query and return immediately without waiting for an answer. You can then get an answer from
Groundlight later by calling `get_image_query` using the id from the `ImageQuery` object. returned
by ask_async.

## `submit_image_query`

`submit_image_query` provides the most configurability over submitting image queries. The other `ask_*` methods are actually wrappers around `submit_image_query` with some defaults set to produce their respective behavior.

See the details on the full function signature for `submit_image_query` and the other ask
groundlight methods in the [API Reference](/api-reference-docs/models.html#groundlight.Groundlight)
108 changes: 54 additions & 54 deletions src/groundlight/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -504,24 +504,26 @@ def ask_confident( # noqa: PLR0913 # pylint: disable=too-many-arguments
:param detector: the Detector object, or string id of a detector like `det_12345`
:param image: The image, in several possible formats:
- filename (string) of a jpeg file
- byte array or BytesIO or BufferedReader with jpeg bytes
- numpy array with values 0-255 and dimensions (H,W,3) in BGR order
(Note OpenCV uses BGR not RGB. `img[:, :, ::-1]` will reverse the channels)
- PIL Image
Any binary format must be JPEG-encoded already. Any pixel format will get
converted to JPEG at high quality before sending to service.
:param image: The image, in several possible formats: - filename (string) of a jpeg file -
byte array or BytesIO or BufferedReader with jpeg bytes - numpy array with values 0-255
and dimensions (H,W,3) in BGR order
(Note OpenCV uses BGR not RGB. `img[:, :, ::-1]` will reverse the channels) - PIL Image
Any binary format must be JPEG-encoded already. Any pixel format will get converted to
JPEG at high quality before sending to service.
:param confidence_threshold: The confidence threshold to wait for.
If not set, use the detector's confidence threshold.
:param confidence_threshold: The confidence threshold to wait for. If not set, use the
detector's confidence threshold.
:param wait: How long to wait (in seconds) for a confident answer.
:param metadata: A dictionary or JSON string of custom key/value metadata to associate with
the image query (limited to 1KB). You can retrieve this metadata later by calling
`get_image_query()`.
:param inspection_id: A premium feature allowing users to combine multiple image queries
together into an inspection report. If you are interested in learning more, please reach
out at [email protected] or through the chat widget on our website.
:return: ImageQuery
"""
return self.submit_image_query(
Expand All @@ -548,21 +550,23 @@ def ask_ml( # noqa: PLR0913 # pylint: disable=too-many-arguments, too-many-loca
:param detector: the Detector object, or string id of a detector like `det_12345`
:param image: The image, in several possible formats:
- filename (string) of a jpeg file
- byte array or BytesIO or BufferedReader with jpeg bytes
- numpy array with values 0-255 and dimensions (H,W,3) in BGR order
(Note OpenCV uses BGR not RGB. `img[:, :, ::-1]` will reverse the channels)
- PIL Image
Any binary format must be JPEG-encoded already. Any pixel format will get
converted to JPEG at high quality before sending to service.
:param image: The image, in several possible formats: - filename (string) of a jpeg file -
byte array or BytesIO or BufferedReader with jpeg bytes - numpy array with values 0-255
and dimensions (H,W,3) in BGR order
(Note OpenCV uses BGR not RGB. `img[:, :, ::-1]` will reverse the channels) - PIL Image
Any binary format must be JPEG-encoded already. Any pixel format will get converted to
JPEG at high quality before sending to service.
:param wait: How long to wait (in seconds) for any answer.
:param metadata: A dictionary or JSON string of custom key/value metadata to associate with
the image query (limited to 1KB). You can retrieve this metadata later by calling
`get_image_query()`.
:param inspection_id: A premium feature allowing users to combine multiple image queries
together into an inspection report. If you are interested in learning more, please reach
out at [email protected] or through the chat widget on our website.
:return: ImageQuery
"""
iq = self.submit_image_query(
Expand All @@ -588,38 +592,39 @@ def ask_async( # noqa: PLR0913 # pylint: disable=too-many-arguments
inspection_id: Optional[str] = None,
) -> ImageQuery:
"""
Convenience method for submitting an `ImageQuery` asynchronously. This is equivalent to calling
`submit_image_query` with `want_async=True` and `wait=0`. Use `get_image_query` to retrieve the `result` of the
ImageQuery.
Convenience method for submitting an `ImageQuery` asynchronously. This is equivalent to
calling `submit_image_query` with `want_async=True` and `wait=0`. Use `get_image_query` to
retrieve the `result` of the ImageQuery.
:param detector: the Detector object, or string id of a detector like `det_12345`
:param image: The image, in several possible formats:
- filename (string) of a jpeg file
- byte array or BytesIO or BufferedReader with jpeg bytes
- numpy array with values 0-255 and dimensions (H,W,3) in BGR order
(Note OpenCV uses BGR not RGB. `img[:, :, ::-1]` will reverse the channels)
- PIL Image: Any binary format must be JPEG-encoded already.
Any pixel format will get converted to JPEG at high quality before sending to service.
- numpy array with values 0-255 and dimensions (H,W,3) in BGR order (Note OpenCV uses BGR
not RGB. `img[:, :, ::-1]` will reverse the channels)
- PIL Image: Any binary format must be JPEG-encoded already. Any pixel format will get
converted to JPEG at high quality before sending to service.
:param patience_time: How long to wait (in seconds) for a confident answer for this image query.
The longer the patience_time, the more likely Groundlight will arrive at a confident answer.
Within patience_time, Groundlight will update ML predictions based on stronger findings,
and, additionally, Groundlight will prioritize human review of the image query if necessary.
This is a soft server-side timeout. If not set, use the detector's patience_time.
:param patience_time: How long to wait (in seconds) for a confident answer for this image
query. The longer the patience_time, the more likely Groundlight will arrive at a
confident answer. Within patience_time, Groundlight will update ML predictions based on
stronger findings, and, additionally, Groundlight will prioritize human review of the
image query if necessary. This is a soft server-side timeout. If not set, use the
detector's patience_time.
:param confidence_threshold: The confidence threshold to wait for.
If not set, use the detector's confidence threshold.
:param confidence_threshold: The confidence threshold to wait for. If not set, use the
detector's confidence threshold.
:param human_review: If `None` or `DEFAULT`, send the image query for human review
only if the ML prediction is not confident.
If set to `ALWAYS`, always send the image query for human review.
If set to `NEVER`, never send the image query for human review.
:param human_review: If `None` or `DEFAULT`, send the image query for human review only if
the ML prediction is not confident. If set to `ALWAYS`, always send the image query for
human review. If set to `NEVER`, never send the image query for human review.
:param inspection_id: Most users will omit this. For accounts with Inspection Reports enabled,
this is the ID of the inspection to associate with the image query.
:param inspection_id: A premium feature allowing users to combine multiple image queries
together into an inspection report. If you are interested in learning more, please reach
out at [email protected] or through the chat widget on our website.
:param metadata: A dictionary or JSON string of custom key/value metadata to associate with
the image query (limited to 1KB). You can retrieve this metadata later by calling
Expand All @@ -630,34 +635,29 @@ def ask_async( # noqa: PLR0913 # pylint: disable=too-many-arguments
**Example usage**::
gl = Groundlight()
detector = gl.get_or_create_detector(
name="door",
query="Is the door locked?",
confidence_threshold=0.9
gl = Groundlight() detector = gl.get_or_create_detector(
name="door", query="Is the door locked?", confidence_threshold=0.9
)
image_query = gl.ask_async(
detector=detector,
image="path/to/image.jpeg")
detector=detector, image="path/to/image.jpeg")
# the image_query will have an id for later retrieval
assert image_query.id is not None
# the image_query will have an id for later retrieval assert image_query.id is not None
# Do not attempt to access the result of this query as the result for all async queries
# will be None. Your result is being computed asynchronously and will be available later
assert image_query.result is None
# retrieve the result later or on another machine by calling gl.wait_for_confident_result()
# with the id of the image_query above. This will block until the result is available.
image_query = gl.wait_for_confident_result(image_query.id)
# retrieve the result later or on another machine by calling
gl.wait_for_confident_result() # with the id of the image_query above. This will block
until the result is available. image_query =
gl.wait_for_confident_result(image_query.id)
# now the result will be available for your use
assert image_query.result is not None
# now the result will be available for your use assert image_query.result is not None
# alternatively, you can check if the result is available (without blocking) by calling
# gl.get_image_query() with the id of the image_query above.
image_query = gl.get_image_query(image_query.id)
# gl.get_image_query() with the id of the image_query above. image_query =
gl.get_image_query(image_query.id)
"""
return self.submit_image_query(
detector,
Expand Down

0 comments on commit 0dd7e18

Please sign in to comment.