Object detection

[greenhawktobias]

I´m doing a project with urban tree detection in a Swedish municipality. We did semantic segmentation (RGB+elevation) using RV with good results.
I tried object detection using the same training data as with the semantic segmentation. I did 5 epochs using almost 300 tree annotations and set `merge_thresh=0.5, score_thresh=0.7). Script used was the template:
https://github.com/azavea/raster-vision/blob/master/rastervision_pytorch_backend/rastervision/pytorch_backend/examples/object_detection/cowc_potsdam.py#L86

My training data:

My predictions

Resnet18 is used as backbone (pretrained it says in cmd ). Is it prefered compared to resnet50 for this task?
Training chips looks good but predictions/precision metrics are bad. Using raster vision latest tag in Docker.
Do I just have to train for more epochs or maybe I´m doing some obviously wrong?

My aim is something like this:

[AdeelH]

Seems like most of the predicted boxes are not making it past the score threshold. I would consider lowering score_thresh until you can see more predicted boxes. If those look roughly correct, then it is just a case of the model not being confident enough yet and training for more epochs might fix it.

[greenhawktobias]

Running same docker image RV-latest tag, same script, same data as my collegue still he gets this traceback when trying to write the bundle in the end. Something with the model on his machine maybe? Maybe because he is a noon root docker user tryung to reach "/root/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth"?

Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 251, in <module>
    _main()
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 247, in _main
    main()
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 236, in run_command
    _run_command(
  File "/opt/src/rastervision_pipeline/rastervision/pipeline/cli.py", line 218, in _run_command
    command_fn()
  File "/opt/src/rastervision_core/rastervision/core/rv_pipeline/rv_pipeline.py", line 181, in predict
    self.backend.load_model()
  File "/opt/src/rastervision_pytorch_backend/rastervision/pytorch_backend/pytorch_learner_backend.py", line 123, in load_model
    self.learner = self._build_learner_from_bundle(training=False)
  File "/opt/src/rastervision_pytorch_backend/rastervision/pytorch_backend/pytorch_learner_backend.py", line 131, in _build_learner_from_bundle
    return Learner.from_model_bundle(
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 1022, in from_model_bundle
    learner: cls = cfg.build(
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/object_detection_learner_config.py", line 217, in build
    return ObjectDetectionLearner(
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 234, in __init__
    self.setup_model(
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/object_detection_learner.py", line 57, in setup_model
    self.load_init_weights(model_weights_path)
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 1146, in load_init_weights
    self.load_weights(uri=uri, **args)
  File "/opt/src/rastervision_pytorch_learner/rastervision/pytorch_learner/learner.py", line 1151, in load_weights
    self.model.load_state_dict(
  File "/opt/conda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for TorchVisionODAdapter:
        Missing key(s) in state_dict: "model.backbone.body.conv1.weight".
        Unexpected key(s) in state_dict: "model.backbone.body.conv1.1.0.weight", "model.backbone.body.conv1.1.1.weight".
make: *** [/opt/data/output/Makefile:9: 1] Error 1

[AdeelH]

That error is the same bug reported (#1625) and fixed (#1626) today. You will need to pull the latest image again.

[AdeelH]

Using other training images and labels gives
rastervision.pytorch_learner.dataset.object_detection_dataset: WARNING - ObjectDetectionRandomWindowGeoDataset: Failed to find negative window. Returning last sampled window. Why is that? Must be something with the data, code is the same as before. Maybe changing neg ratio to none could be a way? chip_options = ObjectDetectionChipOptions(neg_ratio=None, ioa_thresh=0.9)

It means that it could not find a "negative" chip i.e. a chip without any bounding boxes. This should only happen if neg_ratio is not None though. Are you setting neg_ratio in ObjectDetectionGeoDataWindowConfig? See docs for neg_ratio and neg_ioa_thresh.

I changed to method=GeoDataWindowMethod.sliding, seems to work. Wonder why?

If method=GeoDataWindowMethod.sliding, ObjectDetectionSlidingWindowGeoDataset is used. This generates chips in a sliding window fashion.

If method=GeoDataWindowMethod.random, ObjectDetectionRandomWindowGeoDataset is used. This tries to randomly sample positive chips (those with boxes) and negative chips (those w/o boxes) in a certain ratio if neg_ratio is specified.

[greenhawktobias]

RandomWindowGeoDataset give me a warning but keeps training the dataset. I set neg_ratio=0 since "none" is not accepted

Sliding works, but as I see it it does not have neg ratio?

I trained the data for 50 epochs using:

chip_options = ObjectDetectionChipOptions(neg_ratio=5.0, ioa_thresh=0.9)
if nochip:
window_opts = ObjectDetectionGeoDataWindowConfig(
method=GeoDataWindowMethod.sliding,

Map50 goes up slowly (19% after 50 epochs). I will try with more training data to increase precision.

[AdeelH]

Sliding works, but as I see it it does not have neg ratio?

Yes, sliding does not sample chips at random, but rather in a deterministic order starting from the top-left and ending at the bottom-right. So, in numpy-like syntax, the first chip is full_raster[0:size, 0:size], followed by full_raster[0:size, stride:stride+size], and so on.

I set neg_ratio=0 since "none" is not accepted

If you're using nochip=True, you can directly set neg_ratio in ObjectDetectionGeoDataWindowConfig to None (which is allowed) instead of doing neg_ratio=chip_options.neg_ratio (like here).