Adding more loss options during training #32
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The first commit is just a documentation error I noted.
The main changes are thoroughly described in the second commit. Briefly:
softmax
option for building the'avg_dice_no_reduce'
.DiceLoss
to return the loss of just a single class/tissue, this way we can monitor how individual tissues are progressing during the training process.This is probably still a bit of a work in progress - but I wanted to see if there are any thought or feedback as I work on this.
I've added the second commit message w/ the majority of the changes below to make the content easier to see in this pull request:
config.py
updated to take a list calledLOSS_METRICS
.This is a list of extra loss metrics to run during train/val steps.
Each item in the list is structured as:
[[loss_type, activation], weights]. The first item is a list that
mimics the current
cfg.LOSS
input intobuild_loss
and the second item is the weights.
losses.py
now explicitly includes "softmax" version of"avg_dice_no_reduce"
("avg_dice_no_reduce", "softmax")
which calls
DiceLoss
. Though, this might be redundant,becuase it does the exact same thing as calling
("avg_dice_no_reduce", "sigmoid")
. From this standpoint,it might be useful to break
LOSS
into the actual loss part(
'avg_dice_no_reduce'
) and the activation(
'softmax'
/'sigmoid'
).The current implementation was a bit confusing to me -
I thought I had to pass the
softmax
as the activation toDiceLoss
which caused issues. Breaking it up intothese parts would be clearer.
losses.py
got a new function that creates a one-hot-encodedset of
weights
if an integer is inputted instead ofa list of weights. This is useful it the goal is to just find the
loss of a single tissue/class during training.
build_loss
was updated to enable building the additionalloss functions mentioned in
config.py
above.trainer.py
was updated so that it builds a list of the loss metricsmentioned in
config.py
and enabled by the updates tobuild_loss
.reduce_tensor
inutils.py
was updated so that if the reduce was'none'
and the weights were one-hot encoded, it scales the weightsvalue so that it makes sense. Otherwise, when keras does its
built in reduce it averages over all of the zero dims making the loss
seem artificially low (by a factor of the number of categories/classes
in the loss).