progressive GAN last iteration -> RuntimeError: CUDA out of memory. Tried to allocate 512.00 MiB

May you have any suggestions for solve the out of memory issue in the last iteration of the hybrid progressive GAN? what GPU did you use. 
I am trying with GCP 1-2 tier GPU and my RTX 2080 Ti, I also tried with Batch Size = 2 but still same issue. 
Any suggestions would be much appreciated