Hi, I am new to PyTorch Lightning and trying to integrate my workflow with the WandB logger.
I am using a LightningModule which retrieves the input data and labels. I have also associated each input/output pair with json configuration file which describes the capture environment (e.g., if it is multi host or single host, bandwidth, delay, BDP factor). I know how to log values such as F1 score and accuracy for each sample but I am confused how to associate each value with a configuration. Is there any guides available that addresses this or something similar?
For instance, in test_step from the LightningModule I have
def test_step(self, batch, batch_idx):
x, y, config_file = batch
y_pred = self.forward(x)
loss = self.loss(y_pred, y)
self.log("test/loss", loss)
return loss
I would like to do something like this:
def test_step(self, batch, batch_idx):
x, y, config_file = batch
y_pred = self.forward(x)
loss = self.loss(y_pred, y)
self.log(
"test/loss",
loss,
configuration = {
"delay": "10ms",
"BDP": 3,
# etc ...
}
)
return loss
The config_file is a json file with configuration.
My guess is that I could probably change "test/loss" to "test/loss/10ms/3", but I am not sure if this is the best way to go about it. I want to be able to compare different environment settings somehow.
I have looked a bit into torchmetrics, is it possible to address this issue with it?
Thanks in advance!