Images of marine mammals

[philpatton]

Search before asking

• I have searched the Pytorch-Wildlife issues and found no similar bug report.

Description

Are marine mammals within the scope of MegaDetector, or is it strictly for camera trap images? Photo-identification is a crucial tool for marine biologists studying space-use, movement, behavior, abundance, etc., since many marine mammals have natural or acquired marks. We have fairly robust algorithms for individual identification, but these algorithms require cropped images.

It would be great to be able to use MegaDetector to crop the images. I love the API since it's so easy to use and could be more widely distributed than anything we develop in house. In terms of training data, there is this Kaggle competion, where several competitors manually annotated images, and trained some older YOLO models. The results were imperfect, but reasonably effective.

Use case

This feature would be used the same way that biologists used the detection algorithms in MegaDetector. But these would be marine mammal biologists, rather than terrestrial folks using camera traps.

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[idchacon28]

Hi @philpatton,

Thank you for reaching out and bringing this to our attention. We are indeed interested in broadening the applications of MegaDetector, and marine mammals could certainly fall within the scope of our project.

We're currently in the process of training a new model that aims to detect multiple species in overhead data, which includes marine animals. This addition could potentially align with the requirements you've mentioned for photo-identification in marine biology research. This new model we're developing also includes a preprocessing and postprocessing pipeline designed to crop and stitch images, which should address the specific functionality you're interested in. Our aim is to release this new model early next year, and we're excited about the potential it has to aid marine mammal biologists in their important work.

As for the dataset from the Kaggle competition you've referenced, we will definitely review it to determine if it could be integrated into our training pipeline. Please keep an eye on our repository for updates, and we'll make sure to communicate our progress as we move forward.

[bw4sz]

Howdy, one of the maintainers of DeepForest here. We have a bird model that generalizes to other taxa (we've seen good performance on drones + manatees), and would be happy to collaborate, add datasets, be useful here as well. @philpatton, if you have additional images, you can annotate them and fine-tune starting from that backbone. We also have marine animal datasets to combine as well. @zhmiao, I sent you an email, would love to help find the right intersection here, cool stuff!

[idchacon28]

Hi @bw4sz ,

Thank you for your message and the offer to collaborate. It's exciting to hear about the bird model you have been developing and its versatility in generalizing to other species. Your expertise and resources could be incredibly valuable to our ongoing project, where we're in the process of training a new model to detect multiple species in overhead data. It seems there's a great potential for synergy here.

I'll follow up with you via email to discuss how we might work together and make the most of this opportunity. Looking forward to the possibility of combining our efforts to advance in this project.

Best regards,
@idchacon28