How to store images and vectors in CNN image-to-vector regression problems

Using CNN regression, I want to predict a vector parameter with three components from an RGB image. The training images(*.tif) and the corresponding vector files(*.dat) are saved in the same folder. I loaded them in the Matlab workspace as a cell array for images and a matrix for vector parameters.

%Import files
Imgs = cell(1,length(files));
Vpara = zeros(3,length(files));
for n=1:length(files)
    Imgs{n} = imread("each image");
    Vpara(:,3) = load("each vector");
end

I constructed a simple CNN, and performed training.

[net, info] = trainNetwork(Imgs,Vpara,layers,options);

However, an error occurred, stating that the predictor and the number of observed values in the response must be the same.

The actual size of Imgs and Vpara are 1x5400 cell and 3x5400 double, respectively.

How do I solve this problem?

Answer 1

[Govind KM]

Hi @Hiroki Fukuda,

The reasons for the error are:

• When passing 2D images in the form of a numeric array as input to the trainNetwork function, they need to be passed in a specific format, i.e. h-by-w-by-c-by-n numeric array, where "h", "w", and "c" are the height, width, and number of channels of the images, respectively, and "n" is the number of images.
• For 2D image regression, responses need to be passed as an n-by-r matrix, where "n" is the number of images and "r" is the number of responses.
• Although unrelated to the error, only the third column of "Vpara" is being updated in the provided code. All other columns will remain zero.

For reference, here is the corrected code:

%Import files
%Let each image be of dimensions h-by-w-by-c
Imgs = zeros(h,w,c,length(files));
Vpara = zeros(length(files),3);
for n=1:length(files)
    Imgs(:,:,:,n) = imread("each image");
    Vpara(n,:) = load("each vector");
end
[net, info] = trainNetwork(Imgs,Vpara,layers,options);

Input images can also be passed as a Datastore or a Table. More details can be found in the documentation below:

https://www.mathworks.com/help/deeplearning/ref/trainnetwork.html#mw_199f8be9-53f8-4bac-acdf-2846778f5904

Hope this is helpful!