Skip to content

Orange-OpenSource/marine-detect

BranchesTags

Repository files navigation

Marine Detect 🌊🐟

This repository provides access to two YOLOv8 object detection models for identifying species of interest in underwater environments.

These models were developed in the context of the Let's Revive project in partnership with TΔ“naka. TΔ“naka emphasizes impact measurement through TΔ“naka Science, sharing monthly coral ecosystem data. To automate data collection, Orange Business and TΔ“naka partner to deploy an underwater research station with cameras. The system uses AI to identify and quantify 17 species in coral areas, streamlining monitoring and allowing marine biologists more time for restoration.

Images: @copyright TΔ“naka

🐟 Species Scope

The Fish and Invertebrates Object Detection Model detects the Fish and Invertebrates Species and the MegaFauna Object Detection Model detects MegaFauna and Rare Species.

  • MegaFauna and Rare Species: Sharks, Sea Turtles, Rays.
  • Fish Species: Butterfly Fish (Chaetodontidae), Grouper (Serranidae), Parrotfish (Scaridae), Snapper (Lutjanidae), Moray Eel (Muraenidae), Sweet Lips (Haemulidae), Barramundi Cod (Cromileptes altivelis), Humphead (Napoleon) Wrasse (Cheilinus undulatus), Bumphead Parrotfish (Bolbometopon muricatum), Fish (other than above or unrecognizable).
  • Invertebrates Species: Giant Clam, Urchin, Sea Cucumber, Lobster, Crown of Thorns.

These species are "bio-indicating" species, which serve as indicators of the ecosystem health. These bio-indicating species are of course dependent on each region - here the focus is for Malaysia/Indo-Pacific region.

πŸ“Š Datasets Details

The models utilize a combination of publicly available datasets (~ 90%) and TΔ“naka-based datasets (~ 10%). Some datasets were already annotated, and others undergo manual labeling.

References to the public datasets used can be found in the 'References' section of this README.

The images used with annotations (YOLO format) can be downloaded using the following links: FishInv dataset, MegaFauna dataset.

Datasets split details

Model Training + Validation Sets Test Set
FishInv 12,243 images (80%, 20%) 499 images
MegaFauna 8,130 images (80%, 20%) 253 images

Note

The rationale behind the development of two distinct models lies in the utilization of already annotated images available in public datasets. By having separate models, we sidestep the necessity of reannotating images that already encompass annotations for specific species with every Fish, Invertebrates and MegaFauna species. For example, we found a lot of images of turtles already annotated. If we were to adopt a single, all-encompassing model for both Fish and Invertebrates Species 🐟 and MegaFauna 🦈, it would necessitate the reannotation of all those turtle images to include species like urchins, fishes, ...

πŸ€– Model Details

You can download the trained models using the following links: FishInv model, MegaFauna model.

Performances on test sets

Important

Our models are currently undergoing enhancements for improved performance. More labeled images are on the way and will be used to retrain the models.

MegaFauna model performances
Class Images Instances mAP50 mAP50-95
ray 253 73 0.863 0.777
shark 253 111 0.741 0.627
turtle 253 109 0.948 0.887
FishInv model performances
Class Images Instances mAP50 mAP50-95
fish 499 259 0.616 0.501
serranidae 499 49 0.850 0.777
urchin 499 80 0.743 0.479
scaridae 499 48 0.828 0.794
chaetodontidae 499 65 0.891 0.827
giant_clam 499 102 0.870 0.602
lutjanidae 499 86 0.865 0.777
muraenidae 499 58 0.949 0.809
sea_cucumber 499 33 0.969 0.939
haemulidae 499 22 0.972 0.945
lobster 499 31 0.984 0.877
crown_of_thorns 499 28 0.981 0.790
bolbometopon_muricatum 499 19 0.993 0.936
cheilinus_undulatus 499 29 0.995 0.968
cromileptes_altivelis 499 30 0.995 0.945

πŸš— Usage

🏁 Environment Setup

There are 3 options to install the development environment.

Option 1 - Developing Inside a Docker Container with Visual Studio Code's Dev Containers Extension (recommended):

  • If you are using Windows, make sure that Windows Subsytem for Linux is installed and working on your machine (to do so, follow the instructions here).
  • Make sure Docker is installed on your machine.
  • Install the Dev Containers Extension in Visual Studio Code (ms-vscode-remote.remote-containers).
  • In VS Code, open the command palette (CTRL + SHIFT + P) and select Dev Containers: Rebuild and Reopen in Container (make sure Docker is running before executing this step). If the build seems to freeze, read the "Common Errors and Fixes" section below.

Note that the Dockerfile was created for CPU machines. If you wish to use GPU for inference, you can change the base image to nvidia/cuda:12.0.0-runtime-ubuntu22.04.

Option 2 - Developing on Your Host OS with Anaconda:

  • Make sure Conda is installed and working on your machine (to do so, click here).
  • Then, run the following commands in the project directory:
conda create --name your_env_name python=3.10
conda activate your_env_name
pip install -r requirements.txt

Option 3 - Developing on Your Host OS with PIP:

  • Make sure pyenv is installed and working
  • The, run the following commands in the project directory:
pyenv install 3.10
pyenv local 3.10
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

πŸš€ Inference

To make predictions on images or videos using the marine detection models, follow these simple steps:

  1. Models Download: Refer to the 'Model Details' section for information on the models.
  2. Prediction Functions: Utilize the following Python functions to generate predictions with bounding box annotations.
from src.marine_detect.predict import predict_on_images, predict_on_video

# Predict on a set of images using FishInv and MegaFauna models
predict_on_images(
    model_paths=["path/to/FishInv/model", "path/to/MegaFauna/model"],
    confs_threshold=[0.522, 0.6],
    images_input_folder_path="path/to/input/images",
    images_output_folder_path="path/to/output/folder",
)

# Predict on a video using FishInv and MegaFauna models
predict_on_video(
    model_paths=["path/to/FishInv/model", "path/to/MegaFauna/model"],
    confs_threshold=[0.522, 0.6],
    input_video_path="path/to/input/video.mp4",
    output_video_path="path/to/output/video.mp4",
)

Note

The optimal confidence thresholds for the FishInv model and the MegaFauna model are 0.522 and 0.6, respectively.

The resulting images or video files will have bounding boxes annotations, visually indicating the location and extent of the detected marine species within the original data.

For example:

πŸ“š References

Datasets

Model

About

Object detection models for identifying species in marine environments.

letsrevive.orange.com/technology-partner/

Topics

fish deep-learning object-detection marine-species marine-biology invertebrates megafauna yolov8

Resources

Readme

License

AGPL-3.0 license
Activity
Custom properties

Stars

14 stars

Watchers

8 watching

Forks

7 forks
Report repository

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •  

Languages