Error in DeepLearning using googlenet

I am using transfer learning using Googlenet for binary classification of images. The following is the code.

imds = imageDatastore('RGBNewFrames_sameSz', ...

'IncludeSubfolders',true, ...

'LabelSource','foldernames');

%% Divide the data into training and validation data sets.

% Use 70% of the images for training and 30% for validation

[imdsTrain,imdsValidation] = splitEachLabel(imds,0.7,'randomized');

%% Load Pretrained Network

net = googlenet;

analyzeNetwork(net)

inputSize = net.Layers(1).InputSize

%% Replace Final Layers

% The last three layers of the pretrained network net are configured for 1000 classes.

% These three layers must be fine-tuned for the new classification problem.

% Extract all layers, except the last three, from the pretrained network.

layersTransfer = net.Layers(1:end-3);

% transfer the layers to the new classification task by replacing the last three layers with a fully connected layer, a softmax layer, and a classification output layer.

numClasses = numel(categories(imdsTrain.Labels))

layers = [

layersTransfer

fullyConnectedLayer(numClasses,'WeightLearnRateFactor',10,'BiasLearnRateFactor',10,'Name', 'FC_Layer')

softmaxLayer('Name', 'soft_mxLayer')

classificationLayer('Name', 'OutputLayer')];

%% Train Network

pixelRange = [-30 30];

imageAugmenter = imageDataAugmenter( ...

'RandXReflection',true, ...

'RandXTranslation',pixelRange, ...

'RandYTranslation',pixelRange);

augimdsTrain = augmentedImageDatastore(inputSize(1:2),imdsTrain, ...

'DataAugmentation',imageAugmenter);

augimdsValidation = augmentedImageDatastore(inputSize(1:3),imdsValidation);

options = trainingOptions('sgdm', ...

'MiniBatchSize',10, ...

'MaxEpochs',2, ...%6

'InitialLearnRate',1e-4, ...

'Shuffle','every-epoch', ...

'ValidationData',augimdsValidation, ...

'ValidationFrequency',3, ...

'Verbose',false, ...

'Plots','training-progress');

netTransfer = trainNetwork(augimdsTrain,layers,options);

%% Classify Validation Images

[YPred,scores] = classify(netTransfer,augimdsValidation);

YValidation = imdsValidation.Labels;

accuracy = mean(YPred == YValidation)

Was Getting the error

Error using trainNetwork (line 170)

Invalid network.

Error in Ex2_Transfer_learning_googlenet (line 73)

netTransfer = trainNetwork(augimdsTrain,lgraph,options);

Caused by:

Layer 'inception_3a-3x3_reduce': Input size mismatch. Size of input to this layer is different from the expected input size.

Inputs to this layer:

from layer 'inception_3a-relu_1x1' (28×28×64 output)

Layer 'inception_3a-output': Unused input. Each layer input must be connected to the output of another layer.

Detected unused inputs:

input 'in2'

input 'in3'

input 'in4'

Layer 'inception_3b-output': Unused input. Each layer input must be connected to the output of another layer.

Detected unused inputs:

input 'in2'

So used the solution provided in the link https://in.mathworks.com/matlabcentral/answers/411767-trainnetwork-invalid-network

Based on this made the following changes to the code

graph = layerGraph(layers);

netTransfer = trainNetwork(augimdsTrain,lgraph,options);

This resulted in the error,

Layer names in layer array must be non- empty.

So checked and found fcn, softmax did not have names and Explicity assigned names for these layers and executed the code.

The error on Layer names got sorted but now getting the old error again as below.(Error has also been mentioned above for reference)

Input size mismatch. Size of input to this layer is different from the expected input size.

Kindly help me sort this error. I tried all possible things I could think of to debug the error but could not solve it.

Getting the same error with resnet also. Presently it is working only with alexnet.

Any help on this will be greatly appreciated.

Thanks

Answer 1

[Srivardhan Gadila]

As David suggested you can use analyzeNetwork function to analyze your new network and debug the issue.

The issue with the workflow you are following is that, GoogleNet is a dagnetwork and when you are just collecting all the required layers excluding the last 3 layers in the "layersTransfer" array, you are only collecting the layers and information of the individual connections (Connections) is lost here.

layersTransfer = net.Layers(1:end-3);

So when you pass this new layers array to the trainNetwork function, it assumes that the network is a seriesNetwork and all the layers are connected serially. You can check the same using the analyzeNetwork function.

layers = [
    layersTransfer
    fullyConnectedLayer(numClasses,'WeightLearnRateFactor',10,'BiasLearnRateFactor',10,'Name', 'FC_Layer')
    softmaxLayer('Name', 'soft_mxLayer')
    classificationLayer('Name', 'OutputLayer')];
analyzeNetwork(layers)

Hence the correct workflow would be as follows:

Get the layerGraph instead of layers from the pretrained network:

lgraph = layerGraph(net);

Replace the fullyConnectedLayer and the classificationLayer with your newly defined layers:

fcLayer = fullyConnectedLayer(numClasses,'WeightLearnRateFactor',10,'BiasLearnRateFactor',10,'Name', 'FC_Layer');
clsLayer = classificationLayer('Name', 'OutputLayer');
lgraphNew = replaceLayer(lgraph,"loss3-classifier",fcLayer);
lgraphNew = replaceLayer(lgraphNew,"output",clsLayer)
analyzeNetwork(lgraphNew)

Now train this newly created layerGraph "lgraphNew" with the trainNetwork function.

For more information refer to SeriesNetwork, Layer, DAGNetwork, layerGraph, Object Functions of layerGraph.

[Sushma TV]

Hello Sir,

I am getting the following error

Error using trainNetwork (line 183)

Invalid training data. The output size (1000) of the last layer does not match the number of classes

(2).

Error in Ex2_Transfer_learning_googlenet (line 112)

netTransfer = trainNetwork(augimdsTrain,lgraph,options);

However I have included the following lines to make the changes to the number of classes

numClasses = numel(categories(imdsTrain.Labels))

fcLayer = fullyConnectedLayer(numClasses,'WeightLearnRateFactor',10,'BiasLearnRateFactor',10,'Name', 'FC_Layer');

[Srivardhan Gadila]

@Sushma TV, you should use "lgraphNew" with the trainNetwork function and not the "lgraph"

netTransfer = trainNetwork(augimdsTrain,lgraphNew,options);

[Sushma TV]

Hello Sir,

Yes I changed it and executed the code..Can you please explain a little further on how to view the details provided by the analyzeNetwork command.

The command displays the Deep Learning Network Analyzer window. It shows the network layout and the 'Analysis Result' displays the 'Name' of the layers, 'Type', 'Activations', 'Learnables', 'Total Learnables' and 'States'. The top rgith corner displays the No. of layers, Warnings and Errors.

When I run the command it shows 0 Errors and 0 Warnings.

What is the information I need to look for?? Could you please help me understand how I can look for errors generated and debug them from the layout generated.

[Srivardhan Gadila]

When you will execute the command w.r.t "layers" from the initial version of the code which you have posted in the question, you can see multiple errors

layers = [
    layersTransfer
    fullyConnectedLayer(numClasses,'WeightLearnRateFactor',10,'BiasLearnRateFactor',10,'Name', 'FC_Layer')
    softmaxLayer('Name', 'soft_mxLayer')
    classificationLayer('Name', 'OutputLayer')];
analyzeNetwork(layers)

whereas when you execture the command on the "lgraphNew" from the code in the answer I have posted, you will find no errors as it is the correct workflow

lgraph = layerGraph(net);
fcLayer = fullyConnectedLayer(numClasses,'WeightLearnRateFactor',10,'BiasLearnRateFactor',10,'Name', 'FC_Layer');
clsLayer = classificationLayer('Name', 'OutputLayer');
lgraphNew = replaceLayer(lgraph,"loss3-classifier",fcLayer);
lgraphNew = replaceLayer(lgraphNew,"output",clsLayer)
analyzeNetwork(lgraphNew)

[Sushma TV]

Yes sir got it!! I was using the analyzeNetwork command after I was reading the network itself and not after making modification to the layers.

With my earlier code it is displaying 10 errors as 'Unconnected input. Each layer input must be connected to the output of another layer'.

After making the corrections as suggested by you, analyzeNetwork(lgraphNew) is not giving any errors in the network but when it is giving the following error

Error using trainNetwork (line 183)

Invalid training data. The output size (1000) of the last layer does not match the number of classes

(2).

Error in Ex2_Transfer_learning_googlenet (line 112)

netTransfer = trainNetwork(augimdsTrain,lgraph,options);

I have however made the changes in the fullyConnected layer as indicated

fcLayer = fullyConnectedLayer(numClasses,'WeightLearnRateFactor',10,'BiasLearnRateFactor',10,'Name', 'FC_Layer')

with

numClasses = numel(categories(imdsTrain.Labels));

I checked the value of numClasses. It is rightly equal to 2.

What is causing the error sir??

[Srivardhan Gadila]

Like I said in this comment earlier, you should use "lgraphNew" with the trainNetwork function and not the "lgraph"

netTransfer = trainNetwork(augimdsTrain,lgraphNew,options);

[Sushma TV]

It is working now!! Thank you sir for your patience and help.

[Sushma TV]

With regard to use of deep learning networks through transfer learning,(i) how can the model be evaluated for metrics such as precision, recall, F1 score, Precision Recall Curve.

Should it be computed explicity from the confusion matrix or are there any commands for obtaining the same in deep learning models.?

(ii) What are the other metrics that need to be considered for evaluating and comparing Deep learning models.

Kindly provide some inputs on the same

[Srivardhan Gadila]

You can refer to the topics, examples and functions from the following pages:

1. Deep Learning Toolbox
2. Deep Learning Tuning and Visualization — Functions
3. Deep Learning with Images — Functions
4. Deep Learning Tuning and Visualization — Examples
5. Deep Learning with Images — Examples
6. Experiment Manager App

[Sushma TV]

Thank you sir..Actually found the command for finding the accuracy only..But for an unbalanced dataset that i have accuracy is not a good metric for evaluating model. So was looking for commands for other obtaining other metrics. Confusion matrix can be used for computing them. But wanted to check if I there are any inbuilt commands for it. Will check out in detail again. Thank you sir

Answer 2

[David Willingham]

Hi,

I'd recommend running the network analyser to see where the issues are with the modifications that have been made to the network. Here's the doc page for it:

https://www.mathworks.com/help/deeplearning/ref/analyzenetwork.html

Note: You can also visualize the network in the Deep Network Designer (the network analyser can also be launched from within it too): https://www.mathworks.com/help/deeplearning/ref/deepnetworkdesigner-app.html

What information does the network analyser provide you?

[Sushma TV]

I have included the following lines of code

net = googlenet;

analyzeNetwork(net);

The analyzeNetwork(net) opens up the Deep learning Network Analyzer with the visual structure of the network and details of the network. It is showing the no. of layers, 144 layers, 0 errors, 0 warnings.

No details are displayed regarding the issue in the Deep learning Network Analyzer window.

The error is displayed in the Command window, with the error message as reported above. PArt of it shown below.

Error using trainNetwork (line 183)

Invalid network.

Error in Ex2_Transfer_learning_googlenet (line 99)

netTransfer = trainNetwork(augimdsTrain,lgraph,options);

Caused by:

Layer 'inception_3a-3x3_reduce': Input size mismatch. Size of input to this layer is different from

the expected input size.

Inputs to this layer:

from layer 'inception_3a-relu_1x1' (output size 28×28×64)

Layer 'inception_3a-output': Unconnected input. Each layer input must be connected to the output of

another layer.

[Srivardhan Gadila]

@Sushma TV like I said in the answer below, to check the issues with your newly created network, you should use the analyzeNetwork function for it as well.

analyzeNetwork(layers)

As the pretrained network does not have any issues, using analyzeNetwork function on the pretrained network would not display any errors.