Testing Multi GPU training on a Single GPU

DDP/GPU
1

Hello,

is there a way how to test if your code runs on mutiple GPUs when you only have access to a machine with a single GPU?
I can only submit jobs to a cluster node (4-8GPUs) and can’t use the cluster for debugging.

The code runs fine on a single GPU workstation using the ddp trainer settings but fails when mutiple GPUs are used. A way how to emulating mutiple GPUs on a single GPU-machine would be great.

Best Jonas

3

in fact, you can play with ddp_cpu and later on multi-GPU system just use ddp