Pytorch lightning has an amazing number of logging options. Which is great, but also somewhat overwhelming. What I would like to do is quite basic I think. Logging metrics and after each epoch create + log some plots. So I would prefer somethink that is lightweight and well maintained (e.g. because pytorch lightning coredevs use it). Any recommendations?
I would personally suggest going with WandbLogger. It’s pretty amazing with no limitations + you will have the community support on wandb slack channel.
2 Likes
Thanks a lot for this opinion. I will check it out!
I tried out wandb and mlflow so far. Both are pretty amazing. When it comes to logging plots, wandb is the clear winner. You can directly log matplotlib plotly plots and in the UI it is easy to navigate between instances of a plot at certain epochs. With mlflow on the other hand it seems you need to save the plot say as png and log it as a generic artifact.
mlflow is self hosted out of the box, I really like this. wandb self hosting seems possible.
NICE!
BTW MLFlow still has some limitations: Using log_gpu_memory with MLFLow logger causes an exception. · Issue #4411 · Lightning-AI/lightning · GitHub
I hope they resolve it soon.
I settled on wandb now. Setting up self-hosting does require some extra steps (mainly installing docker), but it is not really hard and the slack community is super amazing and willing to help.