Multiple-Input and Multiple-Output Networks

In Deep Learning Toolbox™, you can define network architectures with multiple inputs (for example, networks trained on multiple sources and types of data) or multiple outputs (for example, networks that predicts both classification and regression responses).

Multiple-Input Networks

Define networks with multiple inputs when the network requires data from multiple sources or in different formats. For example, networks that require image data captured from multiple sensors at different resolutions.

To define and train a deep learning network with multiple inputs, specify the network architecture using a dlnetwork object and train using the trainnet function.

To make predictions on a trained deep learning network with multiple inputs, use the minibatchpredict function. Specify multiple inputs using one of the following:

• combinedDatastore object

• transformedDatastore object

• multiple numeric arrays

For an example showing how to train a network with both image and feature input, see Train Network on Image and Feature Data.

Multiple-Output Networks

Define networks with multiple outputs for tasks requiring multiple responses in different formats. For example, tasks requiring both categorical and numeric output.

To train a deep learning network with multiple outputs, use the trainnet function with a custom loss function. For example, to define a loss that corresponds to the sum of the cross-entropy loss of predicted and target labels and the mean squared error of the predicted and target numeric responses, use this loss function:

lossFcn = @(Y1,Y2,T1,T2) crossentropy(Y1,T1) + mse(Y2,T2);

Train the neural network with the custom loss function using the trainnet function.

net = trainnet(dsTrain,net,lossFcn,options);

To make predictions on a trained deep learning network with multiple outputs, use the minibatchpredict function.

For an example, see Train Network with Multiple Outputs.

Use Datastores for Multiple-Input and Multiple-Output Networks

To train a network with multiple input layers or multiple outputs, use the combine and transform functions to create a datastore that outputs a cell array with (numInputs + numOutputs) columns, where numInputs is the number of network inputs and numOutputs is the number of network outputs. The first numInputs columns specify the predictors for each input, and the last numOutputs columns specify the responses. The InputNames and OutputNames properties of the neural network determine the order of the inputs and outputs, respectively.

For inference using the minibatchpredict function, the datastore is valid as long as the read function of the datastore returns columns corresponding to the predictors. The minibatchpredict function uses the first numInputs columns and ignores the subsequent columns, where numInputs is the number of network input layers.