Transfer Learning With Alexnet & RCNN Object Detection

Hi Professionals,

I've manage to get this far but I am experiencing a challenge understanding what the errors really mean to proceed forward!

can some one help me with my code please?

I have a small dataset of gun images I am trying to Training the network using alexnet and then test the network utilising RCNN!!

I am not sure if I am clear please look at my code it will give you the idea!

Thank you in advance for responding to my challenges!

My code:

%%  Entering Image Folder Name/Data Upload Path
gunfolder = '/Users/mmgp/Desktop/gunsGT'; 
  
%% Specifying Image Amount In Specified Folder
total_images = numel(gunfolder)

%% Accessing Content of Folder TrainingSet Using Datastore
imds = imageDatastore(gunfolder,'IncludeSubFolders',0,'LabelSource','Foldernames')

%% Count Images In Each Category "If not equal this will create issues"
tbl=countEachLabel(imds)
%Make Category The Same Number Of Images
minSetCount=min(tbl{:,2})
[imdsTrain,imdsValidation] = splitEachLabel(imds,0.7,'randomized')

%% Resizing images/ Height By Weight By The Amount Of Images In The Dataset
size(imdsTrain)
size(imdsValidation)
%% Counting Total Number Of Images Including Subfolders **IF AVAILABLE**
imgTotal = length(imds.Files)

%% Displaying Multiple Randomized Images Within The Dataset
a = 4;
b = 4;
n = randperm(imgTotal, a*b);

figure(),
Idx = 1;
for j=1:a
    for k=1:b
        img=readimage(imds,n(Idx));
        subplot(a,b,Idx)
        imshow(img);
        Idx=Idx+1;
    end
end

%% Loading Transfer Learning Network Alexnet
net = alexnet;
net.Layers(1);
net.Layers(end);
numel(net.Layers(end).ClassNames);


%% Spliting Network Into 70% Training & 30% Testing
[trainingSet, testSet] =splitEachLabel(imds, 0.7, 'randomize')
imageSize = net.Layers(1).InputSize;

%% Resizing Images, Assists With Preventing Overfitting 
augmentedTrainingSet = augmentedImageDatastore(imageSize,trainingSet,'ColorPreprocessing', 'gray2rgb')
augmentedTestSet = augmentedImageDatastore(imageSize,testSet,'ColorPreprocessing', 'gray2rgb')

%% Initialising Layer Weight Properties & Displaying Feature Extractions
weight1=net.Layers(2).Weights;
weight1=mat2gray(weight1);
figure
montage(weight1),title('First Convolutional Layer Weight');
featureLayer='fc8';

%% Specifying Training Features
trainingFeatures = activations(net,augmentedTrainingSet,featureLayer,'MiniBatchSize',20,'OutputAs','columns');
trainingLabels = trainingSet.Labels;
Classifier = fitcecoc(trainingFeatures,trainingLabels,'Learner'...
,'Linear','Coding','onevsall','ObservationsIn','columns');

%% Specifying Testing Features
testFeatures = activations(net,augmentedTestSet,featureLayer,'MiniBatchSize', 32, 'OutputAs','columns');
predictLabels = predict(Classifier, testFeatures,'ObservationsIn','columns');
testLabels = testSet.Labels;

%% Specifying Training Options For Model
% Set network training options to use mini-batch size of 32 to reduce
% GPU/CPU memory usage. Lower the InitialLearnRate to reduce the rate at which
% network parameters are changed. This is beneficial when fine-tuning a
% pre-trained network and prevents the network from changing too rapidly.

opts = trainingOptions('sgdm', ...
    'Momentum', 0.9, ...
    'InitialLearnRate', 0.004, ...
    'LearnRateSchedule', 'piecewise', ...
    'LearnRateDropFactor', 0.1, ...
    'LearnRateDropPeriod', 8, ...
    'L2Regularization', 0.004, ...
    'MaxEpochs', 10, ...
    'MiniBatchSize', 128, ...
    'Verbose', true, 'Plots','training-progress');
doTraining = true;

[height,width,numChannels, ~] = size(trainingSet);
imageSize = [height width numChannels];
inputLayer = imageInputLayer(imageSize);


% %% Train the R-CNN detector. Training can take a few minutes to complete.
rcnn = trainRCNNObjectDetector(gunfolder, net, opts, 'NegativeOverlapRange', [0 0.3]);

My errors: for which I am having challenge interpreting to move forward

Error using trainRCNNObjectDetector
Expected input number 1, trainingData, to be one of these types:

table

Error in vision.internal.cnn.validation.checkGroundTruth (line 2)
validateattributes(gt, {'table'},{'nonempty'}, name, 'trainingData',1);

Error in trainRCNNObjectDetector>parseInputs (line 311)
vision.internal.cnn.validation.checkGroundTruth(trainingData, fname);

Error in trainRCNNObjectDetector (line 248)
[network, params] = parseInputs(trainingData, network, options, mfilename, varargin{:});

Error in guntest (line 96)
rcnn = trainRCNNObjectDetector(gunfolder, net, opts, 'NegativeOverlapRange', [0 0.3]);
 
>> 

Answer 1

[Raunak Gupta]

Hi,

If it is required to Transfer learning with the alexnet you may need to retrain the alexnet with images that are there in the beginning. Since the CNN is created with alexnet, you can use trainNetwork for training the pretrained network as here. This will now be a classifier for the new images that have provided in beginning. In the code that is provided in question I can see you have used trainRCNNObjectDetector which is a standalone object detector. For training an object detector you need to give training data with bounding boxes and need to add rcnnBoxRegressionLayer at the end.

The error that is shown is because you are using trainRCNNObjectDetector but the training data is a string that is stored in variable gunfolder. Training data should be a table in the format mentioned here. For this it is required to have bounding boxes for the training images.

[Matpar]

Hi Gupta,

I am really trying to understand the approach taken can you point me to a location that gives an example of loading my own data from a specific folder? I would like to follow this step by step please!

I tried the approach with the trainRCNNObjectDetector but that ran me into more challenges as there things that are happening before the code bellow but is not mentioned anywhere that i have searched thus far. Maybe I am phrasing my key words incorrectly?

% Load the ground truth data
data = load('stopSignsAndCars.mat', 'stopSignsAndCars');

these files are taken from the root folder in matlab but is there some where i can go to see how they are creating this from the start so that i can see if this is my problem?

I went into the workflow and I saw the same details that you are suggesting that is missing from my file!!!

the table is gTruth is 40x2 please attachment and the problem is still occuring where the bounding box is being draw now and then. I am having a challenging seeing where the issue is, based on you professionals I am following as you lead.