How to train resnet50 model on multiple input?

I am following the belw link to make the multiple input network

https://www.mathworks.com/matlabcentral/fileexchange/74760-image-classification-using-cnn-with-multi-input-cnn

but my dataset is different from the above link. I have time series data with shape (1,1024,2) in .mat file and data with 2 channel (224,224,2) .mat file. How do i make the image datastore as upperhalf and bottom half

Answer 1

[yanqi liu]

yes，sir，may be use addLayers、connectLayers to merge net layers

[john karli]

I have tried but get the error my code is

trainpath1 = fullfile("D:\folder\");
trainpath2 = fullfile("E:\Classes\");

%imds1 = imageDatastore(trainpath1, 'IncludeSubfolders',true, 'FileExtensions','.PNG','LabelSource','foldernames');
imds1 = imageDatastore(trainpath1, 'FileExtensions', '.mat', 'IncludeSubfolders',true, ...
                      'LabelSource','foldernames',...
                      'ReadFcn',@matReader);
imds2 = signalDatastore(trainpath2,'SignalVariableNames',["frame","label"],'IncludeSubfolders',true,'FileExtensions','.mat');

%imds2 = imageDatastore(trainpath2, 'IncludeSubfolders',true, 'FileExtensions','.PNG','LabelSource','foldernames');
labelds = fileDatastore('labels.mat','ReadFcn',@myReadFcn,'ReadMode','partialfile');
cds = combine(imds1,imds2,labelds);
% Change the image sizes accordingly
imsize1 = [656 875 2];
imsize2 = [1 1024 2];
numClasses = 11;
%% Define muliple input network
layers1 = [
    imageInputLayer(imsize1,'Name','input1')  
    convolution2dLayer(3,16,'Padding','same','Name','conv_1')
    reluLayer('Name','relu_1')
    fullyConnectedLayer(10,'Name','fc11')
    additionLayer(2,'Name','add')
    fullyConnectedLayer(numClasses,'Name','fc12')
    softmaxLayer('Name','softmax')
    classificationLayer('Name','classOutput')];
lgraph = layerGraph(layers1);
layers2 = [imageInputLayer(imsize2,'Name','input2')
    convolution2dLayer(3,16,'Padding','same','Name','conv_2')
    reluLayer('Name','relu_2')
    fullyConnectedLayer(10,'Name','fc21')];
lgraph = addLayers(lgraph,layers2);
lgraph = connectLayers(lgraph,'fc21','add/in2');
plot(lgraph)
%% Define trainingOptions and also set 'Shuffle' to 'never' for this workaround to work
options = trainingOptions('adam', ...
    'InitialLearnRate',0.001, ...
    'LearnRateSchedule','piecewise',...
    'MaxEpochs',3, ...
    'MiniBatchSize',128, ...
    'Verbose',1, ...
    'Plots','training-progress');
net = trainNetwork(cds,lgraph,options);

error

Error using trainNetwork (line 184)
Dimensions of arrays being concatenated are not consistent.

Caused by:
    Error using horzcat
    Dimensions of arrays being concatenated are not consistent.