In the GAN example of the documentation, did we update D several times before updating G?
4 views (last 30 days)
In the following example of GAN for the deep learning toolbox ( https://www.mathworks.com/help/deeplearning/examples/train-generative-adversarial-network.html ), did we update Discriminator D several times before updating the generator G?
As per the GAN algorithm (say the original paper of Goodfellow 2014), we would update D k-times before updating G.
But I don't think the value of k is mentioned in the example. It looks like D and G are updated together in every loops.
The relevant codes:
% Update the discriminator network parameters.
[dlnetDiscriminator.Learnables,trailingAvgDiscriminator,trailingAvgSqDiscriminator] = ...
adamupdate(dlnetDiscriminator.Learnables, gradientsDiscriminator, ...
trailingAvgDiscriminator, trailingAvgSqDiscriminator, iteration, ...
learnRateDiscriminator, gradientDecayFactor, squaredGradientDecayFactor);
% Update the generator network parameters.
[dlnetGenerator.Learnables,trailingAvgGenerator,trailingAvgSqGenerator] = ...
adamupdate(dlnetGenerator.Learnables, gradientsGenerator, ...
trailingAvgGenerator, trailingAvgSqGenerator, iteration, ...
learnRateGenerator, gradientDecayFactor, squaredGradientDecayFactor);
Sourav Bairagya on 10 Jan 2020
GANs can be trained in many fashions. Here, discriminator, D is updated once before updating generator, G. Hence, first, discriminator, D is updated and then the generator, G is updated in every iteration.