How to match format of data for 3D (4D) CNN ??

Hi,

I want to process 3D time series by CNN in sequence-to-sequence mode. But I am unable to fit format of Patterns and Responses for 3D CNN.

Example of the problem:

function testInputCNN_4D



% some data - any random numbers for formal check

N=20;  % number of datasets

for n = 1:N    
     
   % data sizes:
   % size_1 x size_2 x number of samples  x 3 channels ( eg. [R G B])

    signalPattern{n}=rand(4,4,10,3);  
    signalRespons{n}=rand(4,4,10,3);  

end


% simple Net to demonstrate the problem
layers = layerGraph();

layers1 = [
    sequenceInputLayer([4 4 10 3],"Name","sequence")
    convolution3dLayer([3 3 3],3,"Name","conv3d_1","Padding","same")
    softmaxLayer("Name","softmax")
    regressionLayer("Name","regressionoutput")];

lgraph = addLayers(layers,layers1);
plot(lgraph);


% training

maxEpochs = 5;
miniBatchSize = 4;
options = trainingOptions('adam', ...
    'MaxEpochs',maxEpochs, ...
    'MiniBatchSize',miniBatchSize, ...
    'InitialLearnRate',0.022, ...
    'GradientThreshold',0.7, ...
    'Shuffle','never', ...
    'Plots','training-progress',...
    'ExecutionEnvironment','cpu',...
    'Verbose',0);

CNNtest4D = trainNetwork(signalPattern',signalRespons',lgraph,options);


end
%=======eof====

the trainign process starts, but imediattely finshes with errors:

Error using trainNetwork

Array inputs have incompatible channel dimensions.

Error in testInputCNN_4D (line 47)

CNNtest4D = trainNetwork(signalPattern',signalRespons',lgraph,options);

Caused by:

Error using builtin

Array inputs have incompatible channel dimensions.

even if the formal check by Deep Network Designer is OK.

Welcome any idea or working example ....

Answer 1

[Kausthub]

Hi Petr Kolar,

I understand that you are facing errors while trying to fit a 3D CNN model and unsure about the format of data.

From the error message it looks like a case of channel dimension mismatch and the example that you have provided runs successfully. I suggest you to:

• Check the dimensions of your input arrays and ensure that the channel dimensions are compatible for the operation or function you are using.
• Verify that you are indexing or slicing arrays correctly along the channel dimension.

Also, if your data consists of 3D images it is recommended to use ‘image3dInputLayer’ instead of ‘sequenceInputLayer’. Refer the following documentation for a better understanding: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.image3dinputlayer.html

Below mentioned are some references that might be useful for you:

• size(): https://www.mathworks.com/help/matlab/ref/size.html and ndims(): https://www.mathworks.com/help/matlab/ref/double.ndims.html to inspect the dimensions of the arrays and confirm they match as expected.
• reshape(): https://www.mathworks.com/help/matlab/ref/reshape.html to rearrange or reshape your arrays to match the required channel dimensions.

Hope this helps and clarifies your issue regarding format of data for 3D CNN!

[Petr Kolar]

Hallo,

thank you for the answer - generally you are of course rigth, but above given data are only an example. My real data are time-series (so I need 'sequensInputLayer'), and aso the spatial dimensions and configuration play their role. However my problem was meanwhile (partly) solved by using a "macro" function 'unet3dLayers' which designs working NN sturcture based on available functions. In addition (my) time-series have to be fold ('sequenceFoldingLayer') and unfold then ('sequenceUnfoldingLayer') to persist original data structure.

Regard Petr