Hello, I have a question regarding automatic saving.
I’ve followed the description in the doc to save top-k best model weights, but I can’t get it to work.
I created a ModelCheckpoint as follows
from pytorch_lightning.callbacks import ModelCheckpoint # DEFAULTS used by the Trainer checkpoint_callback = ModelCheckpoint( save_top_k=1, verbose=True, monitor='val_acc', mode='max', ) trainer = Trainer(checkpoint_callback=checkpoint_callback)
And I created my
validation_epoch_end as follows:
def validation_step(self, batch, batch_idx): acc = self.calculate_acc(batch) result= pl.EvalResult() result.log('val_acc', acc) return result def validation_epoch_end(self, val_step_output): end_result = pl.EvalResult() end_result.val_acc = torch.sum(val_step_output.val_acc) return end_result
However, I am getting this message and my weights are not saved.
RuntimeWarning: Can save best model only with val_acc available, skipping.
Looking into the method
on_validation_end in the class
ModelCheckpoint (i.e., code), it seems I have to save ‘val_acc’ into the callback metrics of the EvalResult object, but I am not sure how I can do it and if this is the right way to do it.
Can anyone please help me with this?