Matlab does not currently support Generative Adversarial Neural Networks

I searched a lot to see if Matlab supports GAN but unfortunately it does not. I just found deconvolution layer. does anybody know how I can use that for designing a GAN. I am gonna integrate a GAN block within a DAG network and I am not sure if it is possible. I found also another open source lib 'MatConvNet'. is is possible to use it within DAG? Thank you very much I hope it is not wasting time working on deep learning with Matlab.

web address of matconvnet: http://www.vlfeat.org/matconvnet/

web address of DAG : https://blogs.mathworks.com/deep-learning/2018/03/19/creating-a-dag-network-from-dag-parts/

[Walter Roberson]

Did you have a look at https://github.com/layumi/2016_GAN_Matlab

[ashkan dashtban]

Thank you for your suggestion. I have already tried that. Actually the problem is we cannot embed such layers into a matlab DAG. a very simple sample code:

% from MaxconNet

   net = dagnn.DagNN();
  conv1Block = dagnn.ConvTranspose('size',[5 5 256 100],'hasBias',true,'upsample',[1,1],'crop',[0,0,0,0]);
  net.addLayer('G_conv1',conv1Block,{'data_rand'},{'G1'},{'G1f','G1b'});
  net.addLayer('G_bn1',dagnn.BatchNorm(),{'G1'},{'G1bn'},{'Gbn1f','Gbn1b','Gbn1c'});
  net.addLayer('Grelu1',GreluBlock,{'G1bn'},{'G1x'});

% from Matlab

      subdag1 = layerGraph([ ...
      convolution2dLayer([1 1],64,'NumChannels',192,'Name','inception_3a-1x1')
      reluLayer('Name','inception_3a-relu_1x1')]);

% create layer pool

   someLayers=[
     subdag1.Layers
     net];

%%%%%%%%%%%%% Error

Error using nnet.cnn.layer.Layer/vertcat
Cannot convert from 'dagnn.DagNN' to 'nnet.cnn.layer.Layer'.

If we update the above code with :

someLayers=[
 subdag1.Layers
 net.layers];

Error using nnet.cnn.layer.Layer/vertcat
Cannot convert from 'struct' to 'nnet.cnn.layer.Layer'.

I have no idea how to correct such errors.

Please ignore the logic behind this example since it is just for studding possibility

[Walter Roberson]

Those software products are not compatible. You will not be able to combine them.

[Jack Xiao]

so how to fulfill GAN in matlab? can not the official deep learning tool box of matlab support it?

[Walter Roberson]

I posted a link to GAN in MATLAB above.

However you cannot combine that GAN software with MATLAB's Deep Learning routines, and you cannot combine that GAN software with matconvnet .

[Jack Xiao]

yes, you are right. but here I want to know how to fulfill GAN with official deep learning tool box of matlab but not matconvnet?

[Walter Roberson]

Generative Adversarial Neural Networks are not available in any Mathworks product. They are not supported by the Neural Network toolbox.

[Walter Roberson]

ashkan dashtban comments to me:

Exactly and we hopefuly expect it to be available in near future!

[Kenta]

As of now, some kinds of GAN such as conditional GAN are available.

For example, https://jp.mathworks.com/matlabcentral/fileexchange/75441-conditional-gan-and-cnn-classification-with-fashion-mnist

https://jp.mathworks.com/help/deeplearning/ug/train-conditional-generative-adversarial-network.html

https://jp.mathworks.com/matlabcentral/fileexchange/74921-conditional-gan-generative-adversarial-network-with-mnist

Answer 1

[Yui Chun Leung]

I implemented different types of GANs with Matlab, including DCGAN, cGAN, CycleGAN and more.

You can find my files in FileExchange or Github (https://github.com/zcemycl/Matlab-GAN).

[Atallah Baydoun]

Thanks Yui,

I will take a look and see if I can adapt pix2pix to read .mat files instead of jpg and png.

Atallah

[Yui Chun Leung]

Atallah,

Here is the code used for transforming jpg and png to mat.

https://github.com/zcemycl/Matlab-GAN/blob/master/CycleGAN/LoadAppleOrangeDatabase.m https://github.com/zcemycl/Matlab-GAN/blob/master/Pix2Pix/LoadFacadeDatabase.m

But if you want a dataloader like in pytorch, you have to rewrite the for loop and put the jpg loader in it.

Answer 2

[Shounak Mitra]

In the next release, we're going to provide some flexbilities to design advanced networks such as GANs, Siamese Networks etc. Hang on :)

[ashkan dashtban]

• we hope Matlab will provide more flexibility in new release to enable us to build hybrid dynamic networks by combining different components of neural networks. Also, If we could add customized metric funciotions, say precision instead of accuracy.! such capability makes MatLab much more powerfull and handy than python!

[Shounak Mitra]

Great feedback. We're working on that too.

Answer 3

[KH TOHIDUL ISLAM]

It is avilabel now, please check this out. Train Generative Adversarial Network (GAN)

[Atallah Baydoun]

Yep, It was the first thing I checked when I got the email about Matlab 2019 b.

Thanks!

[Atallah Baydoun]

How extensible is the current model to 3D ?

Answer 4

[Atallah Baydoun]

Thanks Mitra for the info.

But when is the next release going to be ?

[Eric]

Going back to R2011b, all "b" releases except one have come out in September. The exception was 2014b, which was October 3. Hence September 2019 is a reasonable estimate.

[Shounak Mitra]

Eric is correct! It is planned for September at this point.

What kind of GANs are you interested in?

[Atallah Baydoun]

I am basically interested in DCGAN and cGAN. Will these types be supported ?

[Shounak Mitra]

Let's think of it in terms of what important features you need to create GANs - autodifferentiation, custom training loops, and may be custom layers too (if you're curious!). All those things are planned to be supported. And of course, the deep learning layer support will be there.

[Atallah Baydoun]

Hopefully then Matlab will include an example on GAN generation also.

Answer 5

[Wonho Song]

Hello.

Thanks for these nice codes.

Normally we do have checkpoints and model to save to a file.

1. I want to know how to save the model.Is it good to save the paramGen and paramDis? and if it is then how to use as a model.
2. 2 .And also I want to know how to test pix2pix.m to a specific image file.Thanks in advance.

[Atallah Baydoun]

Hey Wonho,

I ended up by using pix2pix from this link.

https://www.mathworks.com/matlabcentral/fileexchange/74779-pix2pix

It automatically saves the model. You can se from the link that many troubleshooting issues were resolved.

Answer 6

[David Willingham]

Since R2019b, MATLAB has supported the implementation of GAN's.

More information on this can be found here.

Regards,

David Willingham

Deep Learning Product Manager, MathWorks