Dataset with multi target

[MohammadJavadD]

Hi, my dataset returns a tuple of the target class and some more info about the sample, for example, y = [labels, age, gender]. What is the best way to use the Skorch classifier?

[BenjaminBossan]

(I converted this to a Discussion since we want to test the feature, I hope you don't mind).

This question is difficult to answer with so little information. Do you have a multiclass classification problem? In that case, it should just work if you have a 2d tensor as target. Of course, the labels should be encoded, so something like this:

>>> target
tensor([[0, 1, 2],
        [3, 0, 1],
        [2, 3, 0],
        ...
        [1, 2, 3],
        [0, 1, 2]])

Of course, your criterion should be able to deal with this kind of target.

If you have more details, I might be able to provide a more complete solution.

[BenjaminBossan]

@MohammadJavadD Any updates?

[MohammadJavadD]

Hi, @BenjaminBossan Sorry I missed this conversation. Actually, my dataloader returns the target as a tuple (i.e. [0, 45, 1]) you mentioned. but the criterion only needs to predict the first element. in PyTorch I would use target[:, 0] but here I'm getting error.

[BenjaminBossan]

I see, so your target contains multiple columns but you actually only need the first one. The best solution to this depends a bit on your data.

If you have control over X and y, and they are tensors or numpy arrays, this should just work: net.fit(X, y[:, 0]).

If you only have control over the dataset (e.g. because it comes from an external library), you could try to change its output. This could be through subclassing:

class MyDataset(ExternalDataset):
    def __getitem__(self, i):
        Xi, yi = super().__getitem__()
        return Xi, yi[0]

Or you could wrap the other dataset:

class MyDataset(torch.utils.data.Dataset):
    def __init__(self, external_dataset):
        self.external_dataset = external_dataset

    def __len__(self):
        return len(self.external_dataset)

    def __getitem__(self, i):
        Xi, yi = self.external_dataset[i]
        return Xi, yi[0]

(you can pass a dataset as X when calling net.fit)

If you cannot control the dataset, as a last resort, you could change the behavior of the net itself so that it ignores the extra targets when calculating the loss:

class MyNet(NeuralNetClassifier):
    def get_loss(self, y_pred, y_true, **kwargs):
        y_true = y_true[:, 0]
        return super().get_loss(y_pred, y_true, **kwargs)

[BenjaminBossan]

@MohammadJavadD Did you manage to try any of the proposals?

[MohammadJavadD]

@BenjaminBossan Thank you so much! The mix of the second and third solutions works for my case!

[MohammadJavadD]

As I'm implementing DANN (here), I need additional information on the target. Another obstacle is how can I implement the gradient reversal layer (GRL) on top of the representation.

[BenjaminBossan]

Another obstacle is how can I implement the gradient reversal layer (GRL) on top of the representation.

I haven't read the paper, but maybe adding a gradient hook can solve this. However, this is a different discussion and probably not skorch-specific, so please open a new discussion on the appropriate forum if you have more questions.