Test The Model Every Epoch

implementation help
1

Hey all :slight_smile:
I have 3 type of datasets (Let’s say all of them contain labelled images)

  1. train
  2. validation
  3. special frames that I want to look at in Tensorboard (this is a third / different dataset).

I want every epoch to loop over the special frames in eval mode.

How can I achieve this using Pytorch lightning blocks?
Thanks in advance!

(Every training step I run over the training dataset, and every validation step I run over the validation dataset)

2

I think that you can either concatenate the special frames to your validation dataloader (way 1, probably better) or loop over the special frames in a post-validation callback step (way 2).

Way 1: Concatenate the special frames to your validation dataloader, literally - add code to the dataloader or (and possibly better from software design perspective) create a dataloader class/specialization which is initialized with the original validation dataloader and first loops over it then over the special frames (or even more, but probably too abstract/unnecesary - create a separate dataloader specialization for the new frames, and a dataloader class which joins any two dataloader classes…)

Way 2: Just subclass pytorch_lightning.Callback, override the post-validation step (on_validation_epoch_end, I think) to loop over the special frames, and send the callback to the Trainer costructor.

1 Like
3

Thank you very much Yuri. I’ll try your second approach tomorrow.