Normalize across each channel for each observation independently
The instance normalization operation normalizes the input data
across each channel for each observation independently. To improve the convergence of training
the convolutional neural network and reduce the sensitivity to network hyperparameters, use
instance normalization between convolution and nonlinear operations such as
After normalization, the operation shifts the input by a learnable offset β and scales it by a learnable scale factor γ.
instancenorm function applies the layer normalization operation
dlarray objects makes working with high
dimensional data easier by allowing you to label the dimensions. For example, you can label
which dimensions correspond to spatial, time, channel, and batch dimensions using the
"B" labels, respectively. For unspecified and other dimensions, use the
"U" label. For
dlarray object functions that operate
over particular dimensions, you can specify the dimension labels by formatting the
dlarray object directly, or by using the
The function normalizes over grouped subsets of the
'T' (time), and
'U' (unspecified) dimensions of
dlX for each observation in the
'C' (channel) and
'B' (batch) dimensions, independently.
For unformatted input data, use the
applies the instance normalization operation to the unformatted
dlY = instancenorm(
dlX with format specified by
FMT using any of
the previous syntaxes. The output
dlY is an unformatted
dlarray object with dimensions in the same order as
dlX. For example,
data for 2-D image input with format
'SSCB' (spatial, spatial, channel,
Create randomized input data with two spatial, one channel, and one observation dimension.
width = 12; height = 12; channels = 6; numObservations = 16; X = randn(width,height,channels,numObservations); dlX = dlarray(X,'SSCB');
Create the learnable parameters.
offset = dlarray(zeros(channels,1)); scaleFactor = dlarray(ones(channels,1));
Calculate the instance normalization.
dlZ = instancenorm(dlX,offset,scaleFactor);
View the size and format of the normalized data.
ans = 1×4 12 12 6 16
ans = 'SSCB'
dlX— Input data
dlarray| numeric array
Input data, specified as a formatted
dlarray, an unformatted
dlarray, or a numeric array.
dlX is an unformatted
dlarray or a numeric
array, then you must specify the format using the
'DataFormat' option. If
dlX is a numeric array,
offset must be a
dlX must have a
dlarray| numeric array
Offset β, specified as a formatted
dlarray, or a numeric array with one nonsingleton dimension
with size matching the size of the
'C' (channel) dimension of the
offset is a formatted
dlarray object, then
the nonsingleton dimension must have label
scaleFactor— Scale factor
dlarray| numeric array
Scale factor γ, specified as a formatted
dlarray, or a numeric array with one nonsingleton
dimension with size matching the size of the
'C' (channel) dimension
of the input
scaleFactor is a formatted
then the nonsingleton dimension must have label
comma-separated pairs of
the argument name and
Value is the corresponding value.
Name must appear inside quotes. You can specify several name and value
pair arguments in any order as
'Epsilon',3e-5sets the variance offset to
DataFormat— Dimension order of unformatted data
Dimension order of unformatted input data, specified as a character vector or string
FMT that provides a label for each dimension of the data.
When you specify the format of a
dlarray object, each character provides a
label for each dimension of the data and must be one of the following:
"S" — Spatial
"C" — Channel
"B" — Batch (for example, samples and observations)
"T" — Time (for example, time steps of sequences)
"U" — Unspecified
You can specify multiple dimensions labeled
"U". You can use the labels
"T" at most once.
You must specify
DataFormat when the input data is not a
Epsilon— Variance offset
1e-5(default) | numeric scalar
Variance offset for preventing divide-by-zero errors, specified as the comma-separated pair
'Epsilon' and a numeric scalar greater than or equal to
dlY— Normalized data
Normalized data, returned as a
dlarray. The output
dlY has the same underlying data type as the input
If the input data
dlX is a formatted
dlY has the same dimension format as
dlX. If the
input data is not a formatted
dlY is an
dlarray with the same dimension order as the input data.
The instance normalization operation normalizes the elements xi of the input by first calculating the mean μI and variance σI2 over the spatial and time dimensions for each channel in each observation independently. Then, it calculates the normalized activations as
where ϵ is a constant that improves numerical stability when the variance is very small.
To allow for the possibility that inputs with zero mean and unit variance are not optimal for the operations that follow instance normalization, the instance normalization operation further shifts and scales the activations using the transformation
where the offset β and scale factor γ are learnable parameters that are updated during network training.
Usage notes and limitations:
When at least one of the following input arguments is a
dlarray with underlying data of type
gpuArray, this function runs on the GPU:
For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).