Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add regularization to loss #4

Open
hermancollin opened this issue Jul 22, 2023 · 1 comment
Open

Add regularization to loss #4

hermancollin opened this issue Jul 22, 2023 · 1 comment

Comments

@hermancollin
Copy link
Member

I would like to add some regularization to the loss function for robustness to discourage the model to produce "glitchy" segmentations. For a perfect illustration, see the image below, taken from the validation set of https://github.com/brainhack-school2023/collin_project/tree/main (first iteration of this project).

Screenshot_20230722_142152

  1. Pink axon has discontinuities.
  2. Brown axon is not complete

I am not yet entirely sure how to regularize the myelin prediction, but will update this issue later.

@hermancollin
Copy link
Member Author

hermancollin commented Aug 31, 2023

As a sidenote, SAM was trained with a compound loss: Focal + Dice + MSE. MSE is not computed on the mask, but on the predicted IoU and actual IoU of the mask. Maybe using a similar loss function to the one originally used could be useful.

We supervise mask prediction with a linear combination of focal loss [65] and dice loss [73] in a 20:1 ratio of
focal loss to dice loss, following [20, 14]. [...] The IoU prediction head is trained with mean-square-error loss between the IoU prediction and the predicted mask’s IoU with the ground truth mask. It is added to the mask loss with a constant scaling factor of 1.0.

This could be implemented with monai.losses.DiceFocalLoss with lambda_focal=20.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant