Integrate YOLO model #6

sjay05 · 2024-03-24T20:03:57Z

Added command line option to run script with YOLO or finding largest contour
Uses 2023 CV model

TongguangZhang

Reviewed. Let's change it to have 2 functions - contour and yolo and main calls the function depending on the argument

TongguangZhang · 2024-03-24T20:50:46Z

blue_only.py

            if largest_contour is not None:
-                # Draw a rectangle around the largest circular contour
                x, y, w, h = cv2.boundingRect(largest_contour)


let's change this to early exiting like above

if largest_contour is None: continue

TongguangZhang · 2024-03-24T20:50:57Z

blue_only.py

+                if best_landing_pad is not None:
+                    x, y = best_landing_pad.x_1, best_landing_pad.y_1
+                    w = best_landing_pad.x_2 - best_landing_pad.x_1
+                    h = best_landing_pad.y_2 - best_landing_pad.y_1
+                    found_coordindates = True


let's change this to early exiting like above

if largest_contour is None: continue

TongguangZhang · 2024-03-24T20:51:26Z

blue_only.py

+                    h = best_landing_pad.y_2 - best_landing_pad.y_1
+                    found_coordindates = True
+
+        if found_coordindates:


remove this if condition (since we want to have early exiting above)

TongguangZhang · 2024-03-24T20:57:07Z

models/best-2n.pt

we try to not have models in the repositories (since their file size is really large) - let's add .pt files to the gitignore

TongguangZhang · 2024-03-24T20:57:37Z

yolo_decision.py

+            label = int(boxes.cls[i])
+            confidence = float(boxes.conf[i])
+            result, detection = Detection.create(bounds, label, confidence)
+            if result and detection:


remember to return nothing if the create method fails (similar to the bootcamp)

So if we have one good detection and one failed one, is it better to return nothing and continue? My original thought was it would be okay for the drone to land at the best landing pad among the valid detections.

For now, I have edited it to return nothing if it fails.

TongguangZhang · 2024-03-24T20:59:51Z

blue_only.py

+    if args.method == "yolo":
+        yolo_model = yolo_decision.DetectLandingPad("cpu", 0.5, "models/best-2n.pt")


move this to the section where you use the yolo model

TongguangZhang · 2024-03-24T21:00:20Z

blue_only.py

+    if args.method == "yolo":
+        yolo_model = yolo_decision.DetectLandingPad("cpu", 0.5, "models/best-2n.pt")


let's add these parameters to a config file (look at config.yaml in computer-vision-python for an example)

TongguangZhang · 2024-03-24T21:01:10Z

blue_only.py

-                last_image_time = current_milli_time()
-            # break
+                found_coordindates = True
+        else:


to be explicit let's make this
elif args.method == "yolo"

sjay05 · 2024-03-25T23:33:40Z

Updated.

Two separate functions run_with_countours() or yolo_detect (run with contours is identical to before)
Added YOLO model parameters to config.yml

TongguangZhang

Mostly revert to the old architecture - we just want the step of determining the landing pad to be in a function, not the whole setup and loop and mavlink message sending.

TongguangZhang · 2024-03-28T00:08:16Z

blue_only.py

+    if args.method == "contour":
+        return run_with_contours()
+
+    if args.method == "yolo":
+        model_device = config["yolo_detect_target"]["device"]
+        detect_confidence = config["yolo_detect_target"]["confidence"]
+        model_path = config["yolo_detect_target"]["model_path"]
+
+        return yolo_detect(model_device, detect_confidence, model_path)


sorry I didn't mean to copy the entire script into the functions - we want to have the while loop in main, send the mavlink in main, etc. but just get the x, y, w, h from the contour or yolo function depending on the argparse. I was thinking more like - the line numbers are from the main branch and this isn't exact but like architecturally this is what I was thinking

def detect_landing_pads_contour(): # basically copy paste lines 148-173 return x, y, w, h def detect_landing_pads_yolo(): # implement the class and functions in yolo_detection.py to get a landing pad x, y, w, h = ... # interpret the results return x, y, w, h def main(): # setup stuff and argparse while True: # lines 87 - 148 if args.method == "contour": x, y, w, h = detect_landing_pads_contour() if args.method == "yolo": x, y, w, h = detect_landing_pads_yolo() if x, y, w, h is None: # not exactly this but basically check that we have a bounding box return # lines 174 onward

TongguangZhang

Reviewed

TongguangZhang · 2024-03-29T19:45:51Z

blue_only.py

+    return message_id == pymavlink.mavutil.mavlink.MAVLINK_MSG_ID_RADIO_STATUS
+
+
+def detect_landing_pads_contour(image: np.ndarray) -> "None | tuple":


let's change the function to return False, None and True, output

TongguangZhang · 2024-03-29T19:46:06Z

blue_only.py

+    return tuple(cv2.boundingRect(largest_contour))
+
+
+def detect_landing_pads_yolo(image: np.ndarray, config: dict) -> "None | tuple":


same as above, let's change the function to return False, None and True, output

TongguangZhang · 2024-03-29T19:47:06Z

blue_only.py

+        if args.method == "contour":
+            bounding_box = detect_landing_pads_contour(image)
+        elif args.method == "yolo":
+            bounding_box = detect_landing_pads_yolo(image, config)
+
+        if bounding_box is not None:


from above, let's do result, bounding_box = detect(image) and check the result instead of the bounding box to see if it's valid

TongguangZhang · 2024-03-29T19:47:48Z

blue_only.py

+        if args.method == "contour":
+            bounding_box = detect_landing_pads_contour(image)
+        elif args.method == "yolo":
+            bounding_box = detect_landing_pads_yolo(image, config)
+
+        if bounding_box is not None:


let's do early exiting here - instead of if result, let's do

if not result: continue

TongguangZhang · 2024-03-29T19:48:36Z

blue_only.py

+            loop_counter = 0
+            last_time = current_milli_time()
+        if current_milli_time() - last_image_time > 100:
+            if is_pad_detected:


same as above, instead of checking this here, have it exit early so we don't have to use a variable here

TongguangZhang

LGTM

sjay05 added 4 commits March 24, 2024 15:49

add 2023 YOLO landing-pad model

ed671dd

integrate yolo decision

3995ef3

add changes from black formatter

4debc8a

edit argument field

ced3432

TongguangZhang reviewed Mar 24, 2024

View reviewed changes

sjay05 added 4 commits March 24, 2024 21:30

add .pt to gitignore

453bf93

refactor blue_only.py and add config.yml

fd243d8

add black changes

56a10ac

move parser to function

e8e1246

TongguangZhang reviewed Mar 28, 2024

View reviewed changes

modify blue_only

921e5bf

TongguangZhang reviewed Mar 29, 2024

View reviewed changes

add early exiting

a63b2df

TongguangZhang approved these changes Mar 29, 2024

View reviewed changes

sjay05 merged commit 8f2e2ed into main Mar 29, 2024
1 check passed

maxlou05 deleted the yolo-model branch August 29, 2024 15:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate YOLO model #6

Integrate YOLO model #6

sjay05 commented Mar 24, 2024

TongguangZhang left a comment •

edited

Loading

TongguangZhang Mar 24, 2024

TongguangZhang Mar 24, 2024

TongguangZhang Mar 24, 2024

TongguangZhang Mar 24, 2024

sjay05 Mar 25, 2024

TongguangZhang Mar 24, 2024

sjay05 Mar 25, 2024

sjay05 Mar 25, 2024

TongguangZhang Mar 24, 2024

TongguangZhang Mar 24, 2024

TongguangZhang Mar 24, 2024

sjay05 commented Mar 25, 2024

TongguangZhang left a comment

TongguangZhang Mar 28, 2024

TongguangZhang left a comment

TongguangZhang Mar 29, 2024

TongguangZhang Mar 29, 2024

TongguangZhang Mar 29, 2024

TongguangZhang Mar 29, 2024

TongguangZhang Mar 29, 2024

TongguangZhang left a comment

		if args.method == "yolo":
		yolo_model = yolo_decision.DetectLandingPad("cpu", 0.5, "models/best-2n.pt")

		return message_id == pymavlink.mavutil.mavlink.MAVLINK_MSG_ID_RADIO_STATUS


		def detect_landing_pads_contour(image: np.ndarray) -> "None \| tuple":

		return tuple(cv2.boundingRect(largest_contour))


		def detect_landing_pads_yolo(image: np.ndarray, config: dict) -> "None \| tuple":

Integrate YOLO model #6

Integrate YOLO model #6

Conversation

sjay05 commented Mar 24, 2024

TongguangZhang left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjay05 commented Mar 25, 2024

TongguangZhang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TongguangZhang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TongguangZhang left a comment

Choose a reason for hiding this comment

TongguangZhang left a comment •

edited

Loading