If I have a multi-task setting where my loss is a sum of various weighted objectives, is there a convenient way for me to track the gradient norms of each objective in Lightning individually? Perhaps some tweak on track_grad_norm
?
If I have a multi-task setting where my loss is a sum of various weighted objectives, is there a convenient way for me to track the gradient norms of each objective in Lightning individually? Perhaps some tweak on track_grad_norm
?