add coarse salt/pepper to imgaug transforms

[themattinthehatt]

No description provided.

[ksikka]

Looks fine, a few questions

• Could you provide more context about why we're doing this change to the PR description?
  Is this a general enhancement (how do we test it) or is it to align with dlc (I noticed this gets run on imgaug = dlc or dlc-top-down).

• Should we add this to dali.py?

• Have you run any tests on model training, if so what were the results?

• Have you considered CoarseSaltAndPepper, or just SaltAndPepper?

• Why CoarseSalt instead of Salt?

• General Q: Does adding more imgaug steps increase the number of batches per epoch?

[themattinthehatt]

All great questions - running some models on data now. Answering a few of the specifics below:

• context: more transforms are generally better (results pending). I've seen in some of our IBL datasets that bright reflections on OOD data can often confuse the model into thinking they are in fact paws (which can also just be bright blobs). So the aim is to include some additional transforms that put bright blobs (and dark blobs) into the image as part of image augmentation.
• align with DLC? no, it does not. In general I think we should move away from the "DLC" naming convention (but probably need to wait until 2.0 to do this)
• DALI: has a salt_and_pepper noise transform, but not a coarse version. the coarse version is important here because it sets blocks of pixels to white(ish) or black(ish) rather than individual pixels, so it's a different type of augmentation (aimed at providing invariance to a different type of noise, i.e. environmental clutter rather than pixel noise)
• CoarseSaltAndPepper does something different; it creates a mask, then sets the pixels to black/white in a 50/50 ratio. Salt sets the pixels in the mask to something like 90 white/10 black (based on eyeballing some examples), where Pepper sets the pixels to ~10 white/90 black. Having mostly white or black blocks better approximates the environmental clutter that blocks of 50/50 mix (though maybe we can explore that at some point too)
• adding more imgaug steps does not increase the number of batches, it just slightly increases the amount of time required to process a given image (as it goes through additional augmentations)

[ksikka]

Thanks for the info! I think we should add some of the context you provided in comments.

# bright reflections can often confuse the model into thinking they are paws (which can also just be bright blobs).
# So include some additional transforms that put bright blobs (and dark blobs) into the image

I'm keeping track of a list of things for 2.0, it's in rough shape but I'll share it once it's good enough.