From 5cc24316248f4e5fddea89935d367bfb5a2ff061 Mon Sep 17 00:00:00 2001 From: paulina-positronix Date: Mon, 23 Oct 2023 23:32:14 +0000 Subject: [PATCH 1/3] Pull sample applications into its own page in docs --- .../1-sample-applications.md | 29 +++++++++++++ ...rabbing-images.md => 2-grabbing-images.md} | 0 ...tectors.md => 3-working-with-detectors.md} | 0 ...confidence.md => 4-managing-confidence.md} | 0 ...andling-errors.md => 5-handling-errors.md} | 0 ...{5-async-queries.md => 6-async-queries.md} | 0 .../{6-edge.md => 7-edge.md} | 0 .../{7-industrial.md => 8-industrial.md} | 0 .../building-applications.md | 43 ++++--------------- docs/docs/getting-started/getting-started.mdx | 2 +- 10 files changed, 38 insertions(+), 36 deletions(-) create mode 100644 docs/docs/building-applications/1-sample-applications.md rename docs/docs/building-applications/{1-grabbing-images.md => 2-grabbing-images.md} (100%) rename docs/docs/building-applications/{2-working-with-detectors.md => 3-working-with-detectors.md} (100%) rename docs/docs/building-applications/{3-managing-confidence.md => 4-managing-confidence.md} (100%) rename docs/docs/building-applications/{4-handling-errors.md => 5-handling-errors.md} (100%) rename docs/docs/building-applications/{5-async-queries.md => 6-async-queries.md} (100%) rename docs/docs/building-applications/{6-edge.md => 7-edge.md} (100%) rename docs/docs/building-applications/{7-industrial.md => 8-industrial.md} (100%) diff --git a/docs/docs/building-applications/1-sample-applications.md b/docs/docs/building-applications/1-sample-applications.md new file mode 100644 index 00000000..5fd84bad --- /dev/null +++ b/docs/docs/building-applications/1-sample-applications.md @@ -0,0 +1,29 @@ +# Sample Applications + +Explore these GitHub repositories to see examples of Groundlight-powered applications: + +## Groundlight Stream Processor + +Repository: [https://github.com/groundlight/stream](https://github.com/groundlight/stream) + +The Groundlight Stream Processor is an easy-to-use Docker container for analyzing RTSP streams or common USB-based cameras. You can run it with a single Docker command, such as: + +```bash +docker run stream:local --help +``` + +## Arduino ESP32 Camera Sample App + +Repository: [https://github.com/groundlight/esp32cam](https://github.com/groundlight/esp32cam) + +This sample application allows you to build a working AI vision detector using an inexpensive WiFi camera. With a cost of under $10, you can create a powerful and affordable AI vision system. + +## Raspberry Pi + +Repository: [https://github.com/groundlight/raspberry-pi-door-lock](https://github.com/groundlight/raspberry-pi-door-lock) + +This sample application demonstrates how to set up a Raspberry Pi-based door lock system. The application monitors a door and sends a notification if the door is observed to be unlocked during non-standard business hours. + +## Industrial and Manufacturing Applications + +Groundlight can be used to [apply modern natural-language-based computer vision to industrial and manufacturing applications](/docs/building-applications/industrial). diff --git a/docs/docs/building-applications/1-grabbing-images.md b/docs/docs/building-applications/2-grabbing-images.md similarity index 100% rename from docs/docs/building-applications/1-grabbing-images.md rename to docs/docs/building-applications/2-grabbing-images.md diff --git a/docs/docs/building-applications/2-working-with-detectors.md b/docs/docs/building-applications/3-working-with-detectors.md similarity index 100% rename from docs/docs/building-applications/2-working-with-detectors.md rename to docs/docs/building-applications/3-working-with-detectors.md diff --git a/docs/docs/building-applications/3-managing-confidence.md b/docs/docs/building-applications/4-managing-confidence.md similarity index 100% rename from docs/docs/building-applications/3-managing-confidence.md rename to docs/docs/building-applications/4-managing-confidence.md diff --git a/docs/docs/building-applications/4-handling-errors.md b/docs/docs/building-applications/5-handling-errors.md similarity index 100% rename from docs/docs/building-applications/4-handling-errors.md rename to docs/docs/building-applications/5-handling-errors.md diff --git a/docs/docs/building-applications/5-async-queries.md b/docs/docs/building-applications/6-async-queries.md similarity index 100% rename from docs/docs/building-applications/5-async-queries.md rename to docs/docs/building-applications/6-async-queries.md diff --git a/docs/docs/building-applications/6-edge.md b/docs/docs/building-applications/7-edge.md similarity index 100% rename from docs/docs/building-applications/6-edge.md rename to docs/docs/building-applications/7-edge.md diff --git a/docs/docs/building-applications/7-industrial.md b/docs/docs/building-applications/8-industrial.md similarity index 100% rename from docs/docs/building-applications/7-industrial.md rename to docs/docs/building-applications/8-industrial.md diff --git a/docs/docs/building-applications/building-applications.md b/docs/docs/building-applications/building-applications.md index d52b039d..ab206b5c 100644 --- a/docs/docs/building-applications/building-applications.md +++ b/docs/docs/building-applications/building-applications.md @@ -5,45 +5,18 @@ Groundlight provides a powerful "computer vision powered by natural language" sy In this page, we'll introduce you to some sample applications built using Groundlight and provide links to more detailed guides on various topics. ## Sample Applications - -Explore these GitHub repositories to see examples of Groundlight-powered applications: - -### Groundlight Stream Processor - -Repository: [https://github.com/groundlight/stream](https://github.com/groundlight/stream) - -The Groundlight Stream Processor is an easy-to-use Docker container for analyzing RTSP streams or common USB-based cameras. You can run it with a single Docker command, such as: - -```bash -docker run stream:local --help -``` - -### Arduino ESP32 Camera Sample App - -Repository: [https://github.com/groundlight/esp32cam](https://github.com/groundlight/esp32cam) - -This sample application allows you to build a working AI vision detector using an inexpensive WiFi camera. With a cost of under $10, you can create a powerful and affordable AI vision system. - -### Raspberry Pi - -Repository: [https://github.com/groundlight/raspberry-pi-door-lock](https://github.com/groundlight/raspberry-pi-door-lock) - -This sample application demonstrates how to set up a Raspberry Pi-based door lock system. The application monitors a door and sends a notification if the door is observed to be unlocked during non-standard business hours. - -### Industrial and Manufacturing Applications - -Groundlight can be used to [apply modern natural-language-based computer vision to industrial and manufacturing applications](/docs/building-applications/industrial). +- **[Sample Applications](1-sample-applications.md)**: Find repositories with examples of applications built with Groundlight ## Further Reading For more in-depth guides on various aspects of building applications with Groundlight, check out the following pages: -- **[Grabbing images](1-grabbing-images.md)**: Understand the intricacies of how to submit images from various input sources to Groundlight. -- **[Working with detectors](2-working-with-detectors.md)**: Learn how to create, configure, and use detectors in your Groundlight-powered applications. -- **[Confidence levels](3-managing-confidence.md)**: Master how to control the trade-off of latency against accuracy by configuring the desired confidence level for your detectors. -- **[Handling server errors](4-handling-errors.md)**: Understand how to handle and troubleshoot HTTP errors that may occur while using Groundlight. -- **[Asynchronous queries](5-async-queries.md)**: Groundlight makes it easy to submit asynchronous queries. Learn how to submit queries asynchronously and retrieve the results later. -- **[Using Groundlight on the edge](6-edge.md)**: Discover how to deploy Groundlight in edge computing environments for improved performance and reduced latency. -- **[Industrial applications](7-industrial.md)**: Learn how to apply modern natural-language-based computer vision to your industrial and manufacturing applications. +- **[Grabbing images](2-grabbing-images.md)**: Understand the intricacies of how to submit images from various input sources to Groundlight. +- **[Working with detectors](3-working-with-detectors.md)**: Learn how to create, configure, and use detectors in your Groundlight-powered applications. +- **[Confidence levels](4-managing-confidence.md)**: Master how to control the trade-off of latency against accuracy by configuring the desired confidence level for your detectors. +- **[Handling server errors](5-handling-errors.md)**: Understand how to handle and troubleshoot HTTP errors that may occur while using Groundlight. +- **[Asynchronous queries](6-async-queries.md)**: Groundlight makes it easy to submit asynchronous queries. Learn how to submit queries asynchronously and retrieve the results later. +- **[Using Groundlight on the edge](7-edge.md)**: Discover how to deploy Groundlight in edge computing environments for improved performance and reduced latency. +- **[Industrial applications](8-industrial.md)**: Learn how to apply modern natural-language-based computer vision to your industrial and manufacturing applications. By exploring these resources and sample applications, you'll be well on your way to building powerful visual applications using Groundlight's computer vision and natural language capabilities. diff --git a/docs/docs/getting-started/getting-started.mdx b/docs/docs/getting-started/getting-started.mdx index 2fd6d2bd..ac471e3f 100644 --- a/docs/docs/getting-started/getting-started.mdx +++ b/docs/docs/getting-started/getting-started.mdx @@ -18,7 +18,7 @@ _Note: The SDK is currently in "beta" phase. Interfaces are subject to change in ### How does it work? -Your images are first analyzed by machine learning (ML) models which are automatically trained on your data. If those models have high enough [confidence](docs/building-applications/3-managing-confidence.md), that's your answer. But if the models are unsure, then the images are progressively escalated to more resource-intensive analysis methods up to real-time human review. So what you get is a computer vision system that starts working right away without even needing to first gather and label a dataset. At first it will operate with high latency, because people need to review the image queries. But over time, the ML systems will learn and improve so queries come back faster with higher confidence. +Your images are first analyzed by machine learning (ML) models which are automatically trained on your data. If those models have high enough [confidence](docs/building-applications/4-managing-confidence.md), that's your answer. But if the models are unsure, then the images are progressively escalated to more resource-intensive analysis methods up to real-time human review. So what you get is a computer vision system that starts working right away without even needing to first gather and label a dataset. At first it will operate with high latency, because people need to review the image queries. But over time, the ML systems will learn and improve so queries come back faster with higher confidence. ### Escalation Technology From a64ac09f1bc8ef6339bd6b459d499ee97e9871a7 Mon Sep 17 00:00:00 2001 From: paulina-positronix Date: Tue, 24 Oct 2023 00:24:47 +0000 Subject: [PATCH 2/3] Add example live stream monitor --- docs/docs/getting-started/5-streaming.md | 76 ++++++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 docs/docs/getting-started/5-streaming.md diff --git a/docs/docs/getting-started/5-streaming.md b/docs/docs/getting-started/5-streaming.md new file mode 100644 index 00000000..c8e0db18 --- /dev/null +++ b/docs/docs/getting-started/5-streaming.md @@ -0,0 +1,76 @@ +# A Quick Example: Live Stream Alert + +A quick example to get used to setting up detectors and asking good questions: set up a monitor on a live stream. + +## Requirements + +- [Groundlight SDK](/docs/installation/) with Python 3.7 or higher +- The video ID of a YouTube live stream you'd like to monitor + +## Installation + +Ensure you have Python 3.7 or higher installed, and then install the Groundlight SDK and OpenCV library: + +```bash +pip install groundlight pillow ffmpeg yt-dlp typer +``` + +## Creating the Application + +1. Save this command as a shell script `get_last_frame.sh`: + +``` +#!/bin/bash + +ffmpeg -i "$(yt-dlp -g $1 | head -n 1)" -vframes 1 last.jpg -y +``` + +This will download the last frame from a YouTube live stream and save it to a local file `last.jpg`. + +2. Log in to the [Groundlight application](https://app.groundlight.ai) and get an [API Token](api-tokens). + +3. Next, we'll write the Python script for the application. + +```python notest +import os +import subprocess +import typer +from groundlight import Groundlight +from PIL import Image + +GROUNDLIGHT_API_TOKEN=os.getenv("GROUNDLIGHT_API_TOKEN") + + +def main(*, video_id: str = None, detector_name: str = None, query: str = None, confidence: float = 0.75, wait: int = 60): + """ + Run the script to get the stream's last frame as a subprocess, and submit result as an image query to a Groundlight detector + :param video_id: Video ID of the YouTube live stream + :param detector_name: Name for your Groundlight detector + :param query: Question you want to ask of the stream (we will alert on the answer of NO) + """ + gl = Groundlight(api_token=GROUNDLIGHT_API_TOKEN) + detector = gl.create_detector(name=detector_name, query=query, confidence_threshold=confidence) + + while True: + p = subprocess.run(["./get_last_frame.sh", video_id]) + if p.returncode != 0: + raise RuntimeError(f"Could not get image from video ID: {video_id}. Process exited with return code {p.returncode}.") + + image = Image.open("last.jpg").convert("RGB") + response = gl.submit_image_query(detector=detector, image=image, wait=wait) + + if response.result.label == "NO": + os.system("say 'Alert!'") # this may not work on all operating systems + + +if __name__ == "__main__": + typer.run(main) + +``` + +4. Save the script as `streaming_alert.py` and run it: + +```bash +python streaming_alert.py --detector_name --query +``` + From ce7d634757593f86c4cc9061bde7adb881b0c7f7 Mon Sep 17 00:00:00 2001 From: paulina-positronix Date: Tue, 24 Oct 2023 23:08:29 +0000 Subject: [PATCH 3/3] Improvements following code review: remove references to GROUNDLIGHT_API_TOKEN in the code sample; add instructions on how to get video_id; rename script to get most recent frame --- docs/docs/getting-started/5-streaming.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/docs/docs/getting-started/5-streaming.md b/docs/docs/getting-started/5-streaming.md index c8e0db18..07611998 100644 --- a/docs/docs/getting-started/5-streaming.md +++ b/docs/docs/getting-started/5-streaming.md @@ -17,7 +17,7 @@ pip install groundlight pillow ffmpeg yt-dlp typer ## Creating the Application -1. Save this command as a shell script `get_last_frame.sh`: +1. Save this command as a shell script `get_latest_frame.sh`: ``` #!/bin/bash @@ -25,7 +25,7 @@ pip install groundlight pillow ffmpeg yt-dlp typer ffmpeg -i "$(yt-dlp -g $1 | head -n 1)" -vframes 1 last.jpg -y ``` -This will download the last frame from a YouTube live stream and save it to a local file `last.jpg`. +This will download the most recent frame from a YouTube live stream and save it to a local file `last.jpg`. 2. Log in to the [Groundlight application](https://app.groundlight.ai) and get an [API Token](api-tokens). @@ -38,21 +38,19 @@ import typer from groundlight import Groundlight from PIL import Image -GROUNDLIGHT_API_TOKEN=os.getenv("GROUNDLIGHT_API_TOKEN") - def main(*, video_id: str = None, detector_name: str = None, query: str = None, confidence: float = 0.75, wait: int = 60): """ Run the script to get the stream's last frame as a subprocess, and submit result as an image query to a Groundlight detector - :param video_id: Video ID of the YouTube live stream + :param video_id: Video ID of the YouTube live stream (the URLs have the form https://www.youtube.com/watch?v=) :param detector_name: Name for your Groundlight detector :param query: Question you want to ask of the stream (we will alert on the answer of NO) """ - gl = Groundlight(api_token=GROUNDLIGHT_API_TOKEN) + gl = Groundlight() detector = gl.create_detector(name=detector_name, query=query, confidence_threshold=confidence) while True: - p = subprocess.run(["./get_last_frame.sh", video_id]) + p = subprocess.run(["./get_latest_frame.sh", video_id]) if p.returncode != 0: raise RuntimeError(f"Could not get image from video ID: {video_id}. Process exited with return code {p.returncode}.") @@ -68,7 +66,7 @@ if __name__ == "__main__": ``` -4. Save the script as `streaming_alert.py` and run it: +4. Save the script as `streaming_alert.py` in the same directory as `get_latest_frame.sh` above and run it: ```bash python streaming_alert.py --detector_name --query