Hi all,
I am quite a newbie to Pytotch-Lighting, and I was wondering if there is an option similar to ‘class_weight’ in Keras that allows specifying a cost-factor during the training process.
My intuition tells me that this should a parameter for the Trainer, but I am not sure.
I would appreciate any help.
Best,
Weighting classes is usually performed with the loss function. In classification problems, often the CrossEntropyLoss is used. Take a look at this link for PyTorch’s implementation.
This used to be in the Keras docs:
class_weight : Optional dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function (during training only).
So basically there is no need for the trainer class to have a weight parameter, as the weighting highly depends on the task and the loss function itself.
Hi @johschmidt42,
Thanks for your quick answer. The thing is that I am using Hugging Face Transformer’s library, and they recommend to override the compute_loss method from the Trainer class here.
But I don’t know how to combine this with Pythorch Lighthing.