Deep Network Customization for Images

Customize deep learning training loops and loss functions

If the trainingOptions function does not provide the training options that you need for your task, or custom output layers do not support the loss functions that you need, then you can define a custom training loop. For networks that cannot be created using layer graphs, you can define custom networks as a function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.

Functions

expand all

Custom Training Loops

`dlnetwork`	Deep learning network for custom training loops
`forward`	Compute deep learning network output for training
`predict`	Compute deep learning network output for inference
`adamupdate`	Update parameters using adaptive moment estimation (Adam)
`rmspropupdate`	Update parameters using root mean squared propagation (RMSProp)
`sgdmupdate`	Update parameters using stochastic gradient descent with momentum (SGDM)
`dlupdate`	Update parameters using custom function
`minibatchqueue`	Create mini-batches for deep learning
`onehotencode`	Encode data labels into one-hot vectors
`onehotdecode`	Decode probability vectors into class labels
`initialize`	Initialize learnable and state parameters of a `dlnetwork`
`plot`	Plot neural network architecture
`addLayers`	Add layers to layer graph or network
`removeLayers`	Remove layers from layer graph or network
`connectLayers`	Connect layers in layer graph or network
`disconnectLayers`	Disconnect layers in layer graph or network
`replaceLayer`	Replace layer in layer graph or network
`summary`	Print network summary
`trainingProgressMonitor`	Monitor and plot training progress for deep learning custom training loops

Automatic Differentiation

`dlarray`	Deep learning array for custom training loops
`dlgradient`	Compute gradients for custom training loops using automatic differentiation
`dlfeval`	Evaluate deep learning model for custom training loops

Layers

Input Layers

`imageInputLayer`	Image input layer
`image3dInputLayer`	3-D image input layer

Convolution and Fully Connected Layers

`convolution2dLayer`	2-D convolutional layer
`convolution3dLayer`	3-D convolutional layer
`groupedConvolution2dLayer`	2-D grouped convolutional layer
`transposedConv2dLayer`	Transposed 2-D convolution layer
`transposedConv3dLayer`	Transposed 3-D convolution layer
`fullyConnectedLayer`	Fully connected layer

Activation Layers

`reluLayer`	Rectified Linear Unit (ReLU) layer
`leakyReluLayer`	Leaky Rectified Linear Unit (ReLU) layer
`clippedReluLayer`	Clipped Rectified Linear Unit (ReLU) layer
`eluLayer`	Exponential linear unit (ELU) layer
`tanhLayer`	Hyperbolic tangent (tanh) layer
`swishLayer`	Swish layer
`geluLayer`	Gaussian error linear unit (GELU) layer
`functionLayer`	Function layer

Normalization, Dropout, and Cropping Layers

`batchNormalizationLayer`	Batch normalization layer
`groupNormalizationLayer`	Group normalization layer
`instanceNormalizationLayer`	Instance normalization layer
`layerNormalizationLayer`	Layer normalization layer
`crossChannelNormalizationLayer`	Channel-wise local response normalization layer
`dropoutLayer`	Dropout layer
`crop2dLayer`	2-D crop layer
`crop3dLayer`	3-D crop layer

Pooling and Unpooling Layers

`averagePooling2dLayer`	Average pooling layer
`averagePooling3dLayer`	3-D average pooling layer
`globalAveragePooling2dLayer`	2-D global average pooling layer
`globalAveragePooling3dLayer`	3-D global average pooling layer
`globalMaxPooling2dLayer`	Global max pooling layer
`globalMaxPooling3dLayer`	3-D global max pooling layer
`maxPooling2dLayer`	Max pooling layer
`maxPooling3dLayer`	3-D max pooling layer
`maxUnpooling2dLayer`	Max unpooling layer

Combination Layers

`additionLayer`	Addition layer
`multiplicationLayer`	Multiplication layer
`concatenationLayer`	Concatenation layer
`depthConcatenationLayer`	Depth concatenation layer

Topics

Train Network Using Custom Training Loop
This example shows how to train a network that classifies handwritten digits with a custom learning rate schedule.
Multiple-Input and Multiple-Output Networks
Learn how to define and train deep learning networks with multiple inputs or multiple outputs.
Train Network with Multiple Outputs
This example shows how to train a deep learning network with multiple outputs that predict both labels and angles of rotations of handwritten digits.
Train a Siamese Network to Compare Images
This example shows how to train a Siamese network to identify similar images of handwritten characters.
Example Deep Learning Networks Architectures
This example shows how to define simple deep learning neural networks for various tasks.
Data Sets for Deep Learning
Discover data sets for various deep learning tasks.