Clear Filters
Clear Filters

MATLAB Demo MerchData reproducibility Problem

2 views (last 30 days)
Dear all,
I am trying the MATLAB demo for Transfer Learning using Alexnet found here:
https://www.mathworks.com/help/deeplearning/ref/alexnet.html
However, I couldn't reproduce the same result when using the same dataset. My code as follows:
% Inputes
Number_of_Images_Each_Folder = 14;
Number_of_Classes = 5;
Starting_T = 1;
Experiment_Address = 'I:\DeepLearningDemos_1'; % parent directory of MerchData folder
%***********************************
Starting_T = Starting_T + 2; % because "files" strats from 3
Number_of_T = Number_of_Classes +2; % because "files" strats from 3
%*************************************
%
Root = pwd;
%
files = dir(Root);
net = alexnet;
layers = net.Layers;
Image_Data = '\MerchData';
cd (Experiment_Address) % go to the images address
layers(23) = fullyConnectedLayer(Number_of_Classes);
layers(25) = classificationLayer;
%%
allImages = imageDatastore('MerchData', 'IncludeSubfolders', true, 'LabelSource', 'foldernames');
[imdsTrain, imdsValidation] = splitEachLabel(allImages, 0.8, 'randomize');
%************************************************************************************************
%%
layersTransfer = net.Layers(1:end-3);
%%
layers = [
layersTransfer
fullyConnectedLayer(Number_of_Classes,'WeightLearnRateFactor',20,'BiasLearnRateFactor',20)
softmaxLayer
classificationLayer];
%%
inputSize = net.Layers(1).InputSize
%%
pixelRange = [-30 30];
imageAugmenter = imageDataAugmenter( ...
'RandXReflection',true, ...
'RandXTranslation',pixelRange, ...
'RandYTranslation',pixelRange);
augimdsTrain = augmentedImageDatastore(inputSize(1:2),imdsTrain, ...
'DataAugmentation',imageAugmenter);
%%
augimdsValidation = augmentedImageDatastore(inputSize(1:2),imdsValidation);
%%
options = trainingOptions('sgdm', ...
'MiniBatchSize',10, ...
'MaxEpochs',6, ...
'InitialLearnRate',1e-4, ...
'Shuffle','every-epoch', ...
'ValidationData',augimdsValidation, ...
'ValidationFrequency',3, ...
'Verbose',false, ...
'Plots','training-progress');
%%
rng(0)
netTransfer = trainNetwork(augimdsTrain,layers,options);
%%
save net
% Measure network accuracy
predictedLabels = classify(netTransfer, imdsValidation);
accuracy = mean(predictedLabels == imdsValidation.Labels)
% End of training
%**************************************************************************
%**************************************************************************
%%
Result_Table = {};
for T = Starting_T:Number_of_T
%
allImages_Group_B = imageDatastore('MerchData', 'IncludeSubfolders', true, 'LabelSource', 'foldernames');
dir_Group_B = allImages_Group_B.Files{Number_of_Images_Each_Folder*(T-2),1};
[Testing_Images_Path, name, ext] = fileparts(dir_Group_B);
cd (Testing_Images_Path)
for Tr = 1:Number_of_Images_Each_Folder
fileList = getAllFiles(Testing_Images_Path);
Testing_Image = imread(fileList{Tr,1});
First_Col_Table (Tr) = Tr;
p = predict(netTransfer, Testing_Image);
[p3,i3] = maxk(p,5); %Top 5 predictions
netTransfer.Layers(end)
All_posibilities = netTransfer.Layers(end).ClassNames(i3)
%***********************************************************
First_high_Posibility_1 = All_posibilities{1,1};
First_high_Posibility(Tr) = {First_high_Posibility_1};
Second_high_Posibility_2 = All_posibilities{2,1};
Second_high_Posibility(Tr) = {Second_high_Posibility_2};
Third_high_Posibility_3 = All_posibilities{3,1};
Third_high_Posibility (Tr) = {Third_high_Posibility_3};
Fourth_high_Posibility_4 = All_posibilities{4,1};
Fourth_high_Posibility (Tr) = {Fourth_high_Posibility_4};
Fifth_high_Posibility_5 = All_posibilities{5,1};
Fifth_high_Posibility (Tr) = {Fifth_high_Posibility_5};
end
First_high_Pos = First_high_Posibility';
Second_high_Pos = Second_high_Posibility';
Third_high_Pos = Third_high_Posibility';
Fourth_high_Pos = Fourth_high_Posibility';
Fifth_high_Pos = Fifth_high_Posibility';
New_ResultsX = [First_high_Pos, Second_high_Pos, Third_high_Pos, Fourth_high_Pos, Fifth_high_Pos];
cd (Root) % go back to the parent folder
xlswrite('All_possibilities_MerchData', New_ResultsX, T-2)
end
You will need (getAllFiles) function to import all the images in the current folder
function fileList = getAllFiles(dirName)
dirData = dir(dirName); %# Get the data for the current directory
dirIndex = [dirData.isdir]; %# Find the index for directories
fileList = {dirData(~dirIndex).name}'; %'# Get a list of the files
if ~isempty(fileList)
fileList = cellfun(@(x) fullfile(dirName,x),... %# Prepend path to files
fileList,'UniformOutput',false);
end
subDirs = {dirData(dirIndex).name}; %# Get a list of the subdirectories
validIndex = ~ismember(subDirs,{'.','..'}); %# Find index of subdirectories
%# that are not '.' or '..'
for iDir = find(validIndex) %# Loop over valid subdirectories
nextDir = fullfile(dirName,subDirs{iDir}); %# Get the subdirectory path
fileList = [fileList; getAllFiles(nextDir)]; %# Recursively call getAllFiles
end
end
If you run this code two times and save the excel file results with different names, you can compare and see that results are different from each other. Any idea on how to solve this problem?
Any comment will be appreciated.
Meshoo

Accepted Answer

Naoya
Naoya on 15 Mar 2019
Unfortunately, there is no way to obtain reproducible results on GPU mode, even though the user specfied same seed for random number on MATLAB side.

More Answers (1)

Naoya
Naoya on 14 Mar 2019
The irreproducibility that you reported is due to the non-determinism of the cuDNN routines which Deep Learning Toolbox uses when training on the GPU. The non-deterministic routines are restricted to the backward methods, so a forward pass should always be reproducible.
The simplest way to guarantee reproducible training runs is to train on the CPU, though the tranfer learning script did not take long time to train on CPU.
  1 Comment
Meshooo
Meshooo on 15 Mar 2019
I have too many samples and doing the processing in CPU takes really a lot of time.
Is there any alternative way to optain reproducible results using GPU?
Many thanks in advance.

Sign in to comment.

Categories

Find more on Deep Learning Toolbox in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!