This is an additional implementation of Mask R-CNN for grapes mask detection, grapes bunches counting, and heat maps generation. We based this work on the main architecture implementation from main architecture, and the grape detection sample from grape sample detection. Also, you can read the original README for understanding our uptades.
The main contribution of this work is the addition of a tracker model (programmed with Pytorch). This implementation helps us to count the number of bunches without repetitions detected in a specific video. Another contribution is the heatmaps image generation from satellite images of one particular field. These images comprise the visual interpretability of the grapes bunches in an area.
This repository is composed of two main folders mrcnn
and samples
. The mrcnn
folder comprises the main architecture implementation in Keras, and the samples
comprises two samples of use. The use of coco
sample and demos can be found in README. To this work, we focus on the uvas sample. Review the README of that folder.
To execute this project you need:
- Python version: 3.6.9
- Create a python environment (using pip env or conda) with requirment.txt installed.
- File .h5 with the pretrained model weights of the mrcnn.
- File .t7 with the weights of the DeepSort model.
- A video for testing (in the work context of course).
- Create the environment with conda and python 3.6.9
conda create -n grapes python=3.6.9
- Install the requirements
pip install -r requirements.txt
-
Install cuda toolkit 10.0 and cudnn 7.+. (You must have the drivers already installed)
conda install -c anaconda cudatoolkit=10.0 conda install -c anaconda cudnn=7
-
Download pretrained weights .h5 from drive:
- Or request the file .h5 to a team colleague.
-
Download tracker pytorch checkpoint ckpt.t7 from drive in the folder pretrained weights
- Copy the file to the directory
maskrcnngrape/samples/uvas/deep_sort_pytorch/deep_sort/deep/checkpoint/
- Copy the file to the directory
-
Download a test video from drive:
- Or request the video to a team colleague.
Execution one splash grape prediction in a video
cd samples/uvas
python splash_uvas.py \
--weights=stuff/pretrained_weights/mask_rcnn_uvas_0029.h5 \
--video=stuff/input_videos/processed/DJI_0211.MOV
Here, you can find some problems with its respective solution that we found until now. Feel free to add a new problem and its respective solution.
-
(issues) The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then re-run cmake or configure script
- (solution)(shell): (https://stackoverflow.com/questions/14655969/opencv-error-the-function-is-not-implemented)
sudo apt-get install libgtk2.0-dev pkg-config
-
(issues) The cuda toolkit must be 10.0 and the cudnn > 7.5
-
To retrain the model, you can use all the available GPUs, changind the value
IMAGES_PER_GPU
. However, if you run the inference, we recomend to test first with only one GPUIMAGES_PER_GPU=1
, in the InferenceConfig of the file splash_uvas.py.