F1 null

[asoxien]

Hi everyone;
After training object detection I get f1 null !!!
"overall": [
{
"class_id": 0,
"class_name": "car",
"gt_count": 2414.0,
"pred_count": 0.0,
"count_error": 2414.0,
"metrics": {
"recall": 0.0,
"precision": null,
"f1": null,
"sensitivity": 0.0,
"specificity": null
},
"true_pos": 0.0,
"false_pos": 0.0,
"false_neg": 2414.0
},

I used different backbone but the same result
The used configuration for training:
model=ObjectDetectionModelConfig(backbone=<Backbone.resnet50: 'resnet50'>, pretrained=True, init_weights=None, load_strict=True, external_def=None) solver=SolverConfig(lr=0.0001, num_epochs=100, test_num_epochs=2, test_batch_sz=4, overfit_num_steps=1, sync_interval=1, batch_sz=200, one_cycle=True, multi_stage=[], class_loss_weights=None, ignore_class_index=None, external_loss_def=None) data=ObjectDetectionGeoDataConfig(scene_dataset='<93 train_scenes, 41 validation_scenes, 0 test_scenes>', window_opts="method=<GeoDataWindowMethod.random: 'random'> size=412 stride=None padding=None size_lims=(412, 413) h_lims=None w_lims=None max_windows=500 max_sample_attempts=100 efficient_aoi_sampling=True ioa_thresh=0.75 clip=True neg_ratio=None neg_ioa_thresh=0.2") predict_mode=False test_mode=False overfit_mode=False eval_train=False save_model_bundle=True log_tensorboard=False run_tensorboard=False output_uri='/opt/src/output/model-object_detector_test/train'

[NoaMillsUSDA-ARS]

This is because the model predicts no true positives and no false positives, so when calculating precision and f1, we are dividing by zero. The formula for precision is TP/(TP+FP), which in this case would be 0/0 which is indeterminate, thus listed as null. The f1 value is based on both the values for recall and precision, and since precision is null, we can't find a value for f1 so f1 is then null.

Your model is predicting everything to be "not a car".

[AdeelH]

What dataset are you training on?

[AdeelH]

You should replace that URI with the path of your image.

That link does need to be updated though. Thanks for catching that!

[lewfish]

Actually, the predictions appear to be good in the training command, but the output of the predict command is empty JSON files.

[asoxien]

as you can see the bounding boxes show up correctly:

[asoxien]

I found something that can help when I put negative ratio in master it gets me this error:
ValueError: neg_ratio specified, but no bboxes found in scene.

[asoxien]

Find the result of cowc_potsdam to have a clear idea:
cowc.zip

[lewfish]

I opened an issue for this and have narrowed down the bug, but haven't figured it out yet. See #1517

[asoxien]

Hi;
The #1519 corrected the problem thank you.
But I was obliged to change the files in an old version before #1501 because this last (the last master version) throws me this error NameError: name 'ObjectDetectionConfig' is not defined

[lewfish]

PR #1519 has a fix for that problem too now.