Predictions using Change Detection Model

[rogarmu8]

I wanted to train a change detection model and use it using rastervision 0.30.1 (pytorch-latest)
I am running everything localy using the docker image:
quay.io/azavea/raster-vision:pytorch-latest

Using this 2 discussions and the @AdeelH post:
#2119 #2114
https://www.azavea.com/blog/2022/04/18/change-detection-with-raster-vision/
I was able to get a model trained. I used basically the script provided in the discussion in a notebook:

https://colab.research.google.com/drive/1lZiyZhjgiLMK29Hk5rIBQv1TEL6_WNjV

With github_repo='AdeelH/pytorch-fpn:0.3' and batch size 4 due to computing limitations.

The problem comes when I try to make predictions on a new pair of pictures:

_RuntimeError: split_with_sizes expects split_sizes to sum exactly to 6 (input tensor's size at dimension 1), but got split_sizes=[3, 23]_

I have the following ScenePredictor script:

from pathlib import Path
from rastervision.core.data import (
    ClassConfig, Scene, RasterioSource, MultiRasterSource, StatsTransformer, ReclassTransformer, SemanticSegmentationLabelStore, StatsTransformer, ReclassTransformerConfig)
from rastervision.core.predictor import ScenePredictor
from rastervision.core.rv_pipeline import SemanticSegmentationPredictOptions
from rastervision.core.data.crs_transformer import IdentityCRSTransformer
from rastervision.core.raster_stats import RasterStats

def get_prediction_config(out_dir: str, bundle_path: str, uri_1: str, uri_2: str, scene_id: str, stats_uri:str) -> ScenePredictor:
    ######################################
    # Configure the Scene for New Data
    ######################################
    class_cfg = ClassConfig(
        names=['no change', 'change'],
        colors=['lightgray', 'red'],
        null_class='no change')

    new_scene = make_scene(scene_id, uri_1, uri_2, class_cfg, out_dir, stats_uri)

    ########################
    # Configure the Predictor
    ########################
    predict_options = SemanticSegmentationPredictOptions(
        chip_sz=256
    )

    predictor = ScenePredictor(
        model_bundle_uri=bundle_path,
        predict_options=predict_options
    )

    return predictor, new_scene


def make_scene(scene_id: str, uri_1: str, uri_2: str, class_cfg: ClassConfig, out_dir: str, stats_uri: str) -> Scene:
    raster_source_1 = RasterioSource(uris=[uri_1])
    raster_source_2 = RasterioSource(uris=[uri_2])

    stats = RasterStats.load(stats_uri)
    # Define transformer for the combined raster source
    combined_stats_transformer = StatsTransformer(means=stats.means, stds=stats.stds)
    
    raster_source = MultiRasterSource(
        raster_sources=[raster_source_1, raster_source_2],
        primary_source_idx=1,
        raster_transformers=[combined_stats_transformer]
    )
    crs_transformer = IdentityCRSTransformer()
    label_store = SemanticSegmentationLabelStore(
        uri=out_dir, crs_transformer=crs_transformer, class_config=class_cfg)

    scene = Scene(
        id=scene_id,
        raster_source=raster_source,
        label_store=label_store
    )
    
    return scene


if __name__ == '__main__':
    out_dir = './predictions_cd/out.json'
    bundle_path = './blog_0/bundle/model-bundle.zip'
    uri_1 = './change_imgs/pre.tif'
    uri_2 = './change_imgs/post.tif'
    scene_id = 'new_scene_id'
    stats_uri = './blog_0/analyze/stats/train_scenes/stats.json'
    # Set up the prediction configuration
    predictor, scene = get_prediction_config(out_dir, bundle_path, uri_1, uri_2, scene_id, stats_uri)
    # Run the prediction
    predictor.predict_scene(scene)

I am using the model bundled as described in the previous discussions. I am a little bit stuck, thanks for the help!

[AdeelH]

The error seems to suggest that your input to the model has 6 bands instead of the 26 that the model expects. Are you perhaps feeding in RGB images?

Also, unrelated to the error, but you should probably be passing raster_source.crs_transformer to SemanticSegmentationLabelStore instead of an IdentityCRSTransformer.

[rogarmu8]

Oh, yes my images are RGB tif images.
I thought that using the stats transformer, the images were being adapted to 26 channels... To be honest I am very new with satellite imagery and the different channels, etc.
I don't know if what I am trying to do is actually feasible.
I trained the model using the OCDC dataset as in the explanation, but I wanted to perform predictions in other types of imagery. Is there a way to transform this type of 3 band pictures to 26 band pictures?
And thank you for the suggestion on the transformer, I will use raster_source.crs_transformer.

Thank you very much!

[rogarmu8]

Moreover, I wanted to know, if there is an easy way to transform the output label.tif into a geojson with polygons to achieve a similar result as in your blog post at the end with the changes outlined in red. The only idea that comes to mind is using opencv but I suppose there is simpler! Thank you!

[AdeelH]

The stats transformer does not change the number of channels; it just normalizes the pixel values.

What you can do instead is train your model on just the RGB channels. When defining the raster sources in your training code, pass channel_order=[3, 2, 1]. This corresponds to the red, green, and blue channels in the 13-band images.

Also keep in mind that the images in the OSCD dataset are Sentinel-2 images. A model trained on those images will work best on other Sentinel-2 images. If you feed in images from a different satellite that have a different resolution, you will likely not get very good results.

[rogarmu8]

Thank you very much!! This helped me a lot. Just in case, I also was wonderig how to transform the labels.tif output file to a geojson with the result shapes. Thanks in advance!!!!

[AdeelH]

You can have RV output geojson predictions by passing vector_outputs to SemanticSegmentationLabels.save(). Example:

from rastervision.core.data import PolygonVectorOutputConfig

pred_labels.save(
    uri=f'out/',
    crs_transformer=crs_transformer,
    class_config=class_config,
    # VectorOutputConfigs for each class you want outputted as GeoJSON
    vector_outputs=[
        PolygonVectorOutputConfig(
            class_id=class_config.get_class_id('my_class'),
            # threshold to use for creating binary mask that will then be vectorized
            # (default: 1 / num_classes)
            threshold=0.5,
            # filter out positive regions in the binary mask smaller than
            # this diameter (in pixels)
            denoise=8,
        ),
    ],
)

See docs for PolygonVectorOutputConfig here.