Correct computation of recall in case of missing detection files #119
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…th file, without the need to add empty detection files before. - some minor refactorings for easier path handling
gboeer
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For me, a larger issue of the code is the handling of missing detection files.
The solution proposed so far in this case is to run intersect-gt-and-dr.py to sort those files out which do not have a detection file. However, this completely removes the respective groundtruth information from the evaluation, hence does not provide a correct evaluation of the detection, i.e. the false negatives are not taken into account in those cases. One solution is to add empty detection files for those images where no detections are provided, as has been proposed in #60.
With some small changes, this PR makes it unnecessary to perform the extra step of balancing the detection and groundtruth files. The script can now be run with any number of detection files which for me is much more intuitive. It will still be checked however, if there are groundtruth files for each detection file which may be arguable depending on the application (e.g. for not fully annotated datasets).
PR includes some minor changes for easier Path handling using pathlib.