ResNet50 on multi-spectral image segmentation

Is there a way to use any pretrain network (not necessarily Resnet) to segment multispectral images in MATLAB?

deeplabv3plusLayers

only allows [height width 3] or [height width] input images. While I tried bypassing the error deeplabv3plusLayers returns, when I used trainNetwork I get an error referring to the wrong input data 224x224xN.

Can the first convolutional layer of the pretrained network be replaced to process more than 3 channels? An example done in python can be found here.

Answer 1

[Srivardhan Gadila]

You can copy the layerGraph of the pretrained network and change the imageInputLayer, the first convolutionLayer to match the input image channel dimension & convolution filter dimensions. Then you can freeze/unfreeze the existing pretrained weights during training the new network accordingly.

You can do something like below:(N=50)

imageSize = [224 224 3];

% Specify the number of classes.
numClasses = 10;

N = 50;
% Create DeepLab v3+.
lgraph = deeplabv3plusLayers(imageSize, numClasses, "resnet50");
analyzeNetwork(lgraph)
layers = lgraph.Layers

%%
newlgraph = replaceLayer(lgraph,'input_1',imageInputLayer([224 224 N],'Name','input'));
newlgraph = replaceLayer(newlgraph,'conv1',convolution2dLayer(7,64,'stride',[2 2],'padding',[3 3 3 3],'Name','conv1'))
analyzeNetwork(newlgraph)

You can refer to Get Started with Transfer Learning & Deep Learning Toolbox