Feature: add YOLOv8 inference and finetunning

[mohwald]

Summary:

• adds new requirement for Ultralytics (which also installs PyTorch)
• adds both quantitative (precision, recall based on masks IoU) and qualitative evaluation
• adds data conversion for RAMP format into YOLOv5 format
• adds YOLOv8 support for training and inference

[mohwald]

When I installed ultralytics, for yolo8, from scratch, it conflicts with old opencv:

ERROR: Cannot install hot-fair-utilities and hot-fair-utilities==1.1.2 because these package versions have conflicting dependencies.

The conflict is caused by:
    hot-fair-utilities 1.1.2 depends on opencv-python==4.5.5.64
    ultralytics 8.0.157 depends on opencv-python>=4.6.0

I would not rush on updating opencv now, as this might require to fix more fair code. When I get to the point that yolo8 can be trained, so it has comparable outputs to ramp, then I would upgrade opencv.

[kshitijrajsharma]

Thanks @8ToThePowerOfMol , This is great ! I am thinking of experimenting yolo working with its training code and establish it as a basemodel if output is right and then we can upgrade opencv
Do you have any notebooks / resources for training or any recommendations ?

[mohwald]

As a next step, we need to be able to train/finetune yolo with our data. Ultralytics framework has their own training procedure, but for training data, it requires to transform the segmentation annotations to their format - I see two options:

1. Find out the way to transform geojson to Ultralytics YOLO format and then rely on their training pipeline.
2. Export their pretrained model (on COCO) into tensorflow checkpoint and then use the same tensorflow pipeline to train/finetune yolo the same way as ramp. However, I am not sure now how the architectures and callbacks work in tf.

I would first try the second option and if that won't work, we can try first one. The first option needs to convert the world geo-coordinates into the image coordinates.

When we get to the point, the training works on the sample_2, I would try to finetune yolo on the same data that were used for ramp. The resulting model, I would then use it as the yolo basemodel for fair.

[mohwald]

I exported yolo v8 model to keras, but then I wasn't successful with loading it.

[mohwald]

And I found solid evaluation script in ramp, so I added it as a function to the fair utils code, together with additional possibility to evaluate qualitatively from prediction masks.
With this evaluation, I created evaluation.ipynb where ramp is compared with FastSAM. FastSAM ends up unfortunately not usable at all. @kshitijrajsharma let me know if you want to include this to the repo, so anybody else can discover it.

[kshitijrajsharma]

Wonderful @mohwald , I have requested the collab notebook for the access ! Sorry for the late reply I was on vacation

[mohwald]

No worries. I made colab accessible now @kshitijrajsharma: https://colab.research.google.com/drive/1GxKpdWDYzmmOPJ3dY6FohFCjk2lvD_-w?usp=sharing

[kshitijrajsharma]

No worries. I made colab accessible now @kshitijrajsharma: https://colab.research.google.com/drive/1GxKpdWDYzmmOPJ3dY6FohFCjk2lvD_-w?usp=sharing

Excellent Analysis @mohwald ! Helped me visualize capabilities and I agree with you fastsam doesn't fit properly may be with different dataset and different feature type it might , I loved this , I might keep this on repo to explore more ! Thanks once again lovely piece of work !

[mohwald]

Major change with YOLO done. The source code is fully functional.

Example usage with RAMP comparison is there:
https://colab.research.google.com/drive/164H1o7VjflyucBu8xSw338bz6im8zgbn#scrollTo=hxyU2BQAjoT7

[kshitijrajsharma]

I am finally merging your PR ! Thank you so much for your contribution ! Soon Yolo will be available to UI