In a simple training setup, I would like to directly access the lists/dicts of losses and other metrics logged during training and validation so that I can make some custom plots. Where can I find these stored?
As far as I understand you need to use such callback:
from pytorch_lightning import Callback class MetricsCallback(Callback): """PyTorch Lightning metric callback.""" def __init__(self): super().__init__() self.metrics =  def on_validation_end(self, trainer, pl_module): self.metrics.append(trainer.callback_metrics)
Unfortunatelly this does not work:
Contains only whatever was passed to EvalResult as a
Interesting - this code worked for me when I used it with old API (dictionaries instead of EvalResult). I hope there will be a solution.
Yep, I just tried the old dict API, and it works. EvalResults do not.
can you try:
Since I am returning dictionaries for now instead of Result objects, the solution suggested by @Erlemar works.
But it does look like we need clarity around a solution for collecting metrics from Train or EvalResults going forward, instead of dicts. If I get to try it out, I will update this thread.
trainer.logged_metrics works in both
It is still unclear to me what metrics can be found under
trainer.logged_metrics in each callback, specifically are those reduced (by the
on_epoch=True param in
result.log) metrics or unreduced metrics.
It would be awesome to have this clearly described in the docs.