Object Detection With Multiband Imagery

[natedailey7]

I want to perform object detection on 4 band imagery, and I'm not sure what model I can use. Based on this post, I tried using a FasterRCNN model with this line of code in my ObjectDetectionLearner:

model=TorchVisionODAdapter(fasterrcnn_resnet50_fpn_v2(num_classes=len(class_config), weights_backbone="DEFAULT"))

I end up with the following error:

RuntimeError: The size of tensor a (4) must match the size of tensor b (3) at non-singleton dimension 0

As seen in this tutorial, the parameter in_channels can be specified in with torch.hub.load. I wonder, is there something similar I could do for FasterRCNN? Or does anyone have a recommendation for a model I could use for multiband object detection? My code is as follows.

class_config = ClassConfig(
    names=['background', '1'],
    colors=['lightgray', 'darkred'],
    null_class='background')

val_image_uri   = "path"
val_label_uri   = "path"
train_image_uri = "path"
train_label_uri = "path"

window_width = 100
window_height = 100
	
train_ds = ObjectDetectionSlidingWindowGeoDataset.from_uris(
    class_config=class_config,
    image_uri=train_image_uri,
    label_vector_uri=train_label_uri,
    label_vector_default_class_id=class_config.get_class_id('1'),
    size=window_width,
    stride=window_width,
    transform=A.Resize(window_width, window_height),
)

val_ds = ObjectDetectionSlidingWindowGeoDataset.from_uris(
    class_config=class_config,
    image_uri=train_image_uri,
    label_vector_uri=train_label_uri,
    label_vector_default_class_id=class_config.get_class_id('1'),
    size=window_width,
    stride=window_width,
    transform=A.Resize(window_width, window_height),
)

data_cfg = ObjectDetectionGeoDataConfig(
    class_names=class_config.names,
    class_colors=class_config.colors,
    num_workers=0,
)

solver_cfg = SolverConfig(
    batch_sz=8,
    lr=0.005,
)

learner_cfg = ObjectDetectionLearnerConfig(data=data_cfg, solver=solver_cfg)
learner = ObjectDetectionLearner(
    cfg=learner_cfg,
    output_dir='path',
    model=TorchVisionODAdapter(fasterrcnn_resnet50_fpn_v2(num_classes=len(class_config), weights_backbone="DEFAULT")),
    train_ds=train_ds,
    valid_ds=val_ds,
)
learner.log_data_stats()

learner.train(epochs=20)

Thank you!

[AdeelH]

FasterRCNN is what RV uses by default for object detection, so instead of passing in a model you can let RV build the model for you by specifying a ModelConfig.

Here's how you would do it:

in_channels = 4
data_cfg = ObjectDetectionGeoDataConfig(
    class_names=class_config.names,
    class_colors=class_config.colors,
    img_channels=in_channels, # <---------- in_channels goes here
    num_workers=0,
)
solver_cfg = SolverConfig(
    batch_sz=8,
    lr=0.005,
)
model_cfg = ObjectDetectionModelConfig(backbone='resnet50', pretrained=True)
learner_cfg = ObjectDetectionLearnerConfig(
    data=data_cfg, 
    solver=solver_cfg,
    model=model_cfg, 
)
learner = ObjectDetectionLearner(
    cfg=learner_cfg,
    output_dir='path',
    train_ds=train_ds,
    valid_ds=val_ds,
)

[AdeelH]

Never seen that error before, but this stackoverflow post suggests uninstalling and reinstalling torchvision.

Can you try that out and see if it helps? You might need to change --index-url https://download.pytorch.org/whl/cu116 to --index-url https://download.pytorch.org/whl/cu118.

[AdeelH]

Can you also check your cuda version? Run this in a python console:

import torch
torch.version.cuda

[natedailey7]

I tried reinstalling, with the command pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 from Torchvision's interface. And torch.version.cuda is showing 11.8 for me. Is CUDA 11.8 typically compatible?

[natedailey7]

EDIT: I reinstalled the CUDA 11.8 toolkit from here and my object detection training is now working! Thank you so much @AdeelH 🙇‍♂️

[AdeelH]

Oh great! Yeah, it seemed like a CUDA version mismatch somewhere. It might be worth upgrading everything to CUDA 12 and re-trying (if you want).