If I have a multi-task setting where my loss is a sum of various weighted objectives, is there a convenient way for me to track the gradient norms of each objective in Lightning individually? Perhaps some tweak on track_grad_norm ?
If I have a multi-task setting where my loss is a sum of various weighted objectives, is there a convenient way for me to track the gradient norms of each objective in Lightning individually? Perhaps some tweak on track_grad_norm ?