fullyConnectedLayer

Fully connected layer

Description

A fully connected layer multiplies input vectors by a weight matrix and then adds a bias vector.

Creation

Syntax

layer = fullyConnectedLayer(outputSize)

layer = fullyConnectedLayer(outputSize,dim)

layer = fullyConnectedLayer(outputSize,Name=Value)

Description

layer = fullyConnectedLayer(outputSize) returns a fully connected layer and sets the OutputSize property.

layer = fullyConnectedLayer(outputSize,dim) also sets the OperationDimension property. (since R2026a)

layer = fullyConnectedLayer(outputSize,Name=Value) sets optional properties using one or more name-value arguments.

Input Arguments

outputSize — Output size

positive integer

Output size for the fully connected layer, specified as a positive integer.

The layer operation flattens the dimensions specified by the OperationDimension property, then multiplies by the weights matrix and adds the bias vector for each element in remaining dimensions, independently. (since R2026a)

Before R2026a: The layer operation flattens the "S" (spatial), "C" (channel), and "U" (unspecified) dimensions of the input data, then multiplies by the weights matrix and adds the bias vector for each element in the "B" (batch) and "T" (time) dimensions, independently.

This argument sets the OutputSize property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

dim — Operation dimension

"spatial-channel" (default) | positive integer

Since R2026a

Operation dimension, specified as one of these values:

• "spatial-channel" — Flatten the "S" (spatial) and "C" (channel) dimensions of the input data, then multiply by the weights matrix and add the bias vector for each element in the "B" (batch), "T" (time), and "U" (unspecified) dimensions, independently.

• positive integer — Use the specified dimension of the layer input data X as the inner dimension of the matrix multiplication Weights*X in the layer operation, and apply the operation independently for each of the remaining dimensions.

This argument sets the OperationDimension property.

Data Types: single | double | char | string

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: fullyConnectedLayer(10,Name="fc1") creates a fully connected layer with an output size of 10 and the name "fc1".

InputLearnables — Learnable parameters from other layers

{} (default) | string array | character vector | cell array of character vectors

Since R2026a

Learnable parameters from other layers, specified as a string array, character vector, or cell array of character vectors of zero or more of these values:

• "weights" — Receive weights from another layer. If the InputLearnables property specifies that the layer receives weights from another layer, then the layer has an additional input with the name "weights". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Weights property.

• "bias" — Receive biases from another layer. If the InputLearnables property specifies that the layer receives biases from another layer, then the layer has an additional input with the name "bias". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Bias property.

If InputLearnables specifies that the layer receives weights or biases from another layer, then the corresponding properties that contain the initializer, learnable parameters, learning rate factor, and L₂ regularization factor must be [].

For more information, see Neural Network Weight Tying.

This argument sets the InputLearnables property.

OutputLearnables — Learnable parameters to output to other layers

{} (default) | string array | character vector | cell array of character vectors

Since R2026a

Learnable parameters to output to other layers, specified as a string array, character vector, or cell array of character vectors of zero or more of these values:

• "weights" — Output weights to another layer. If the OutputLearnables property specifies that the layer outputs weights to another layer, then the layer has an additional output with the name "weights". In this case, the layer uses this output to pass the weights to other layers.

• "bias" — Output biases to another layer. If the OutputLearnables property specifies that the layer outputs biases to another layer, then the layer has an additional output with the name "bias". In this case, the layer uses this output to pass the bias to other layers.

For more information, see Neural Network Weight Tying.

This argument sets the OutputLearnables property.

WeightsInitializer — Function to initialize weights

"glorot" (default) | "he" | "orthogonal" | "narrow-normal" | "zeros" | "ones" | function handle | [] (since R2026a)

Function to initialize the weights, specified as one of these values:

• "glorot" — Initialize the weights with the Glorot initializer [1] (also known as Xavier initializer). The Glorot initializer independently samples from a normal distribution with zero mean and variance 2/(InputSize + OutputSize).

• "he" — Initialize the weights with the He initializer [2]. The He initializer samples from a normal distribution with zero mean and variance 2/InputSize.

• "orthogonal" — Initialize the input weights with Q, the orthogonal matrix given by the QR decomposition of Z = QR for a random matrix Z sampled from a unit normal distribution. [3]

• "narrow-normal" — Initialize the weights by independently sampling from a normal distribution with zero mean and standard deviation 0.01.

• "zeros" — Initialize the weights with zeros.

• "ones" — Initialize the weights with ones.

• Function handle — Initialize the weights with a custom function. If you specify a function handle, then the function syntax must be of the form weights = func(sz), where sz is the size of the weights. For an example, see Specify Custom Weight Initialization Function.

• [] (since R2026a) — Do not initialize the weights. Use the weights from the 'weights' input instead.

The layer only initializes the weights when the Weights property is [] and the InputLearnables property does not specify that the layer receives weights from another layer.

This argument sets the WeightsInitializer property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | char | string | function_handle

BiasInitializer — Function to initialize biases

"zeros" (default) | "narrow-normal" | "ones" | function handle | [] (since R2026a)

Function to initialize the biases, specified as one of these values:

• "zeros" — Initialize the biases with zeros.

• "ones" — Initialize the biases with ones.

• "narrow-normal" — Initialize the biases by independently sampling from a normal distribution with a mean of zero and a standard deviation of 0.01.

• Function handle — Initialize the biases with a custom function. If you specify a function handle, then the function must have the form bias = func(sz), where sz is the size of the biases.

• [] (since R2026a) — Do not initialize the bias. Use the bias from the 'bias' input instead.

The layer only initializes the weights when the Bias property is [] and the InputLearnables property does not specify that the layer receives biases from another layer.

This argument sets the BiasInitializer property.

Data Types: char | string | function_handle

Weights — Layer weights

[] (default) | matrix

Initial layer weights, specified as a matrix.

If the InputLearnables property specifies that the layer receives weights from another layer, then the layer has an additional input with the name "weights". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Weights property. (since R2026a)

The layer weights are learnable parameters. You can specify the initial value of the weights directly using the Weights property of the layer. When you train a network, if the Weights property is nonempty, then the trainnet function uses the Weights property as the initial value. If the Weights property is empty and the InputLearnables property does not specify that the layer receives weights from another layer, then the software uses the initializer specified by the WeightsInitializer property.

At training time, Weights is an OutputSize-by-InputSize matrix.

If InputLearnables specifies that the layer receives weights from another layer, then the WeightsInitializer, Weights, WeightLearnRateFactor, and WeightL2Factor properties must be [].

This argument sets the Weights property.

Data Types: single | double

Bias — Layer biases

[] (default) | vector

Initial layer biases, specified as a vector.

If the InputLearnables property specifies that the layer receives biases from another layer, then the layer has an additional input with the name "bias". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Bias property. (since R2026a)

The layer biases are learnable parameters. When you train a neural network, if the Bias property is nonempty, then the trainnet function uses the Bias property as the initial value. If Bias is empty and the InputLearnables property does not specify that the layer receives biases from another layer, then software uses the initializer specified by the BiasInitializer property.

At training time, Bias is an OutputSize-by-1 vector.

If InputLearnables specifies that the layer receives biases from another layer, then the BiasInitializer, Bias, BiasLearnRateFactor, and BiasL2Factor properties must be [].

This argument sets the Bias property.

Data Types: single | double

WeightLearnRateFactor — Learning rate factor for weights

1 (default) | nonnegative scalar | [] (since R2026a)

Learning rate factor for the weights, specified as one of these values:

• Nonnegative scalar — Use the specified learning rate factor.

• [] (since R2026a) — Use learning rate factor from another layer.

The software multiplies this factor by the global learning rate to determine the learning rate for the weights in this layer. For example, if WeightLearnRateFactor is 2, then the learning rate for the weights in this layer is twice the current global learning rate. The software determines the global learning rate based on the settings you specify using the trainingOptions function.

If InputLearnables is "weights", then the Weights, WeightsInitializer, WeightLearnRateFactor, and WeightL2Factor properties must be empty.

This argument sets the WeightLearnRateFactor property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

BiasLearnRateFactor — Learning rate factor for biases

1 (default) | nonnegative scalar | [] (since R2026a)

Learning rate factor for the biases, specified as one of these:

• Nonnegative scalar — Use the specified learning rate factor.

• [] (since R2026a) — Use learning rate factor from another layer.

The software multiplies this factor by the global learning rate to determine the learning rate for the biases in this layer. For example, if BiasLearnRateFactor is 2, then the learning rate for the biases in the layer is twice the current global learning rate. The software determines the global learning rate based on the settings you specify using the trainingOptions function.

If InputLearnables specifies that the layer receives biases from another layer, then the BiasInitializer, Bias, BiasLearnRateFactor, and BiasL2Factor properties must be [].

This argument sets the BiasLearnRateFactor property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

WeightL2Factor — L₂ regularization factor for weights

1 (default) | nonnegative scalar | [] (since R2026a)

L₂ regularization factor for the weights, specified as one of these values:

• Nonnegative scalar — Use the specified regularization factor.

• [] (since R2026a) — Use regularization factor from another layer.

The software multiplies this factor by the global L₂ regularization factor to determine the L₂ regularization for the weights in this layer. For example, if WeightL2Factor is 2, then the L₂ regularization for the weights in this layer is twice the global L₂ regularization factor. You can specify the global L₂ regularization factor using the trainingOptions function.

If InputLearnables specifies that the layer receives weights from another layer, then the WeightsInitializer, Weights, WeightLearnRateFactor, and WeightL2Factor properties must be [].

This argument sets the WeightL2Factor property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

BiasL2Factor — L₂ regularization factor for biases

0 (default) | nonnegative scalar | [] (since R2026a)

L₂ regularization factor for the biases, specified as one of these:

• Nonnegative scalar — Use the specified regularization factor.

• [] (since R2026a) — Use regularization factor from another layer.

The software multiplies this factor by the global L₂ regularization factor to determine the L₂ regularization for the biases in this layer. For example, if BiasL2Factor is 2, then the L₂ regularization for the biases in this layer is twice the global L₂ regularization factor. The software determines the global L₂ regularization factor based on the settings you specify using the trainingOptions function.

If InputLearnables specifies that the layer receives biases from another layer, then the BiasInitializer, Bias, BiasLearnRateFactor, and BiasL2Factor properties must be [].

This argument sets the BiasL2Factor property.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

Name — Layer name

"" (default) | character vector | string scalar

Layer name, specified as a character vector or a string scalar. For Layer array input, the trainnet and dlnetwork functions automatically assign names to unnamed layers.

This argument sets the Name property.

Data Types: char | string

Properties

Fully Connected

OutputSize — Output size

positive integer

This property is read-only after object creation. To set this property, use the corresponding positional input argument when you create the FullyConnectedLayer object.

Output size for the fully connected layer, specified as a positive integer.

The layer operation flattens the dimensions specified by the OperationDimension property, then multiplies by the weights matrix and adds the bias vector for each element in remaining dimensions, independently. (since R2026a)

Before R2026a: The layer operation flattens the "S" (spatial), "C" (channel), and "U" (unspecified) dimensions of the input data, then multiplies by the weights matrix and adds the bias vector for each element in the "B" (batch) and "T" (time) dimensions, independently.

Data Types: double

OperationDimension — Operation dimension

'spatial-channel' (default) | positive integer

Since R2026a

This property is read-only after object creation. To set this property, use the corresponding positional input argument when you create the FullyConnectedLayer object.

Operation dimension, specified as one of these values:

• 'spatial-channel' — Flatten the "S" (spatial) and "C" (channel) dimensions of the input data, then multiply by the weights matrix and add the bias vector for each element in the "B" (batch), "T" (time), and "U" (unspecified) dimensions, independently.

• positive integer — Use the specified dimension of the layer input data X as the inner dimension of the matrix multiplication Weights*X in the layer operation, and apply the operation independently for each of the remaining dimensions.

Data Types: double | char

InputSize — Input size

'auto' (default) | positive integer

Input size for the layer, specified as a positive integer or 'auto'. If InputSize is 'auto', then the software automatically determines the input size during training.

Data Types: double | char

Weight Tying

InputLearnables — Learnable parameters from other layers

{} (default) | cell array of character vectors

Since R2026a

This property is read-only after object creation. To set this property, use the corresponding name-value argument when you create the FullyConnectedLayer object.

Learnable parameters from other layers, specified as a cell array of zero or more of these values:

• 'Weights' — Receive weights from another layer. If the InputLearnables property specifies that the layer receives weights from another layer, then the layer has an additional input with the name "weights". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Weights property.

• 'Bias' — Receive biases from another layer. If the InputLearnables property specifies that the layer receives biases from another layer, then the layer has an additional input with the name "bias". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Bias property.

If InputLearnables specifies that the layer receives weights or biases from another layer, then the corresponding properties that contain the initializer, learnable parameters, learning rate factor, and L₂ regularization factor must be [].

For more information, see Neural Network Weight Tying.

Data Types: cell

OutputLearnables — Learnable parameters to output to other layers

{} (default) | cell array of character vectors

Since R2026a

This property is read-only after object creation. To set this property, use the corresponding name-value argument when you create the FullyConnectedLayer object.

Learnable parameters to output to other layers, specified as a cell array of character vectors of zero or more of these values:

• 'Weights' — Output weights to another layer. If the OutputLearnables property specifies that the layer outputs weights to another layer, then the layer has an additional output with the name "weights". In this case, the layer uses this output to pass the weights to other layers.

• 'Bias' — Output biases to another layer. If the OutputLearnables property specifies that the layer outputs biases to another layer, then the layer has an additional output with the name "bias". In this case, the layer uses this output to pass the bias to other layers.

For more information, see Neural Network Weight Tying.

Data Types: cell

Parameters and Initialization

WeightsInitializer — Function to initialize weights

'glorot' (default) | 'he' | 'orthogonal' | 'narrow-normal' | 'zeros' | 'ones' | function handle | [] (since R2026a)

Function to initialize the weights, specified as one of these values:

• 'glorot' — Initialize the weights with the Glorot initializer [1] (also known as Xavier initializer). The Glorot initializer independently samples from a normal distribution with zero mean and variance 2/(InputSize + OutputSize).

• 'he' — Initialize the weights with the He initializer [2]. The He initializer samples from a normal distribution with zero mean and variance 2/InputSize.

• 'orthogonal' — Initialize the input weights with Q, the orthogonal matrix given by the QR decomposition of Z = QR for a random matrix Z sampled from a unit normal distribution. [3]

• 'narrow-normal' — Initialize the weights by independently sampling from a normal distribution with zero mean and standard deviation 0.01.

• 'zeros' — Initialize the weights with zeros.

• 'ones' — Initialize the weights with ones.

• Function handle — Initialize the weights with a custom function. If you specify a function handle, then the function syntax must be of the form weights = func(sz), where sz is the size of the weights. For an example, see Specify Custom Weight Initialization Function.

• [] (since R2026a) — Do not initialize the weights. Use the weights from the 'weights' input instead.

The layer only initializes the weights when the Weights property is [] and the InputLearnables property does not specify that the layer receives weights from another layer.

Data Types: double | char | string | function_handle

BiasInitializer — Function to initialize biases

'zeros' (default) | 'narrow-normal' | 'ones' | function handle | [] (since R2026a)

Function to initialize the biases, specified as one of these values:

• 'zeros' — Initialize the biases with zeros.

• 'ones' — Initialize the biases with ones.

• 'narrow-normal' — Initialize the biases by independently sampling from a normal distribution with a mean of zero and a standard deviation of 0.01.

• Function handle — Initialize the biases with a custom function. If you specify a function handle, then the function must have the form bias = func(sz), where sz is the size of the biases.

• [] (since R2026a) — Do not initialize the bias. Use the bias from the 'bias' input instead.

The layer only initializes the weights when the Bias property is [] and the InputLearnables property does not specify that the layer receives biases from another layer.

Data Types: double | char | string | function_handle

Weights — Layer weights

[] (default) | matrix

Initial layer weights, specified as a matrix.

If the InputLearnables property specifies that the layer receives weights from another layer, then the layer has an additional input with the name "weights". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Weights property. (since R2026a)

The layer weights are learnable parameters. You can specify the initial value of the weights directly using the Weights property of the layer. When you train a network, if the Weights property is nonempty, then the trainnet function uses the Weights property as the initial value. If the Weights property is empty and the InputLearnables property does not specify that the layer receives weights from another layer, then the software uses the initializer specified by the WeightsInitializer property.

At training time, Weights is an OutputSize-by-InputSize matrix.

If InputLearnables specifies that the layer receives weights from another layer, then the WeightsInitializer, Weights, WeightLearnRateFactor, and WeightL2Factor properties must be [].

Data Types: single | double

Bias — Layer biases

[] (default) | matrix

Initial layer biases, specified as a vector.

If the InputLearnables property specifies that the layer receives biases from another layer, then the layer has an additional input with the name "bias". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Bias property. (since R2026a)

The layer biases are learnable parameters. When you train a neural network, if the Bias property is nonempty, then the trainnet function uses the Bias property as the initial value. If Bias is empty and the InputLearnables property does not specify that the layer receives biases from another layer, then software uses the initializer specified by the BiasInitializer property.

At training time, Bias is an OutputSize-by-1 vector.

If InputLearnables specifies that the layer receives biases from another layer, then the BiasInitializer, Bias, BiasLearnRateFactor, and BiasL2Factor properties must be [].

Data Types: single | double

Learning Rate and Regularization

WeightLearnRateFactor — Learning rate factor for weights

1 (default) | nonnegative scalar | [] (since R2026a)

Learning rate factor for the weights, specified as one of these values:

• Nonnegative scalar — Use the specified learning rate factor.

• [] (since R2026a) — Use learning rate factor from another layer.

The software multiplies this factor by the global learning rate to determine the learning rate for the weights in this layer. For example, if WeightLearnRateFactor is 2, then the learning rate for the weights in this layer is twice the current global learning rate. The software determines the global learning rate based on the settings you specify using the trainingOptions function.

If InputLearnables is "weights", then the Weights, WeightsInitializer, WeightLearnRateFactor, and WeightL2Factor properties must be empty.

Data Types: double

BiasLearnRateFactor — Learning rate factor for biases

1 (default) | nonnegative scalar | [] (since R2026a)

Learning rate factor for the biases, specified as one of these:

• Nonnegative scalar — Use the specified learning rate factor.

• [] (since R2026a) — Use learning rate factor from another layer.

The software multiplies this factor by the global learning rate to determine the learning rate for the biases in this layer. For example, if BiasLearnRateFactor is 2, then the learning rate for the biases in the layer is twice the current global learning rate. The software determines the global learning rate based on the settings you specify using the trainingOptions function.

If InputLearnables specifies that the layer receives biases from another layer, then the BiasInitializer, Bias, BiasLearnRateFactor, and BiasL2Factor properties must be [].

Data Types: double

WeightL2Factor — L₂ regularization factor for weights

1 (default) | nonnegative scalar | [] (since R2026a)

L₂ regularization factor for the weights, specified as one of these values:

• Nonnegative scalar — Use the specified regularization factor.

• [] (since R2026a) — Use regularization factor from another layer.

The software multiplies this factor by the global L₂ regularization factor to determine the L₂ regularization for the weights in this layer. For example, if WeightL2Factor is 2, then the L₂ regularization for the weights in this layer is twice the global L₂ regularization factor. You can specify the global L₂ regularization factor using the trainingOptions function.

If InputLearnables specifies that the layer receives weights from another layer, then the WeightsInitializer, Weights, WeightLearnRateFactor, and WeightL2Factor properties must be [].

Data Types: double

BiasL2Factor — L₂ regularization factor for biases

0 (default) | nonnegative scalar | [] (since R2026a)

L₂ regularization factor for the biases, specified as one of these:

• Nonnegative scalar — Use the specified regularization factor.

• [] (since R2026a) — Use regularization factor from another layer.

The software multiplies this factor by the global L₂ regularization factor to determine the L₂ regularization for the biases in this layer. For example, if BiasL2Factor is 2, then the L₂ regularization for the biases in this layer is twice the global L₂ regularization factor. The software determines the global L₂ regularization factor based on the settings you specify using the trainingOptions function.

If InputLearnables specifies that the layer receives biases from another layer, then the BiasInitializer, Bias, BiasLearnRateFactor, and BiasL2Factor properties must be [].

Data Types: double

Layer

Name — Layer name

'' (default) | character vector

Layer name, specified as a character vector. For Layer array input, the trainnet and dlnetwork functions automatically assign names to unnamed layers.

Data Types: char

NumInputs — Number of inputs

Read-only: 1 (default) | 2 | 3

This property is read-only.

Number of inputs to the layer, stored as a positive integer.

The number of inputs depends on the InputLearnables property:

• The layer, by default, has an input with the name "in" that corresponds to the layer input data.

• If the InputLearnables property specifies that the layer receives weights from another layer, then the layer has an additional input with the name "weights". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Weights property.

• If the InputLearnables property specifies that the layer receives biases from another layer, then the layer has an additional input with the name "bias". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Bias property.

Data Types: double

InputNames — Input names

Read-only: {'in'} (default) | cell array of character vectors

This property is read-only.

Input names, stored as a cell array of character vectors.

The number of inputs depends on the InputLearnables property:

• The layer, by default, has an input with the name "in" that corresponds to the layer input data.

• If the InputLearnables property specifies that the layer receives weights from another layer, then the layer has an additional input with the name "weights". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Weights property.

• If the InputLearnables property specifies that the layer receives biases from another layer, then the layer has an additional input with the name "bias". In this case, the layer uses the values passed to this input for the layer operation. Otherwise, the layer uses the weights in the Bias property.

Data Types: cell

NumOutputs — Number of outputs

1 (default) | 2 | 3

Number of outputs from the layer, stored as a positive integer.

The number of outputs depends on the OutputLearnables property:

• The layer, by default, has an output with the name "out" that corresponds to the layer output data.

• If the OutputLearnables property specifies that the layer outputs weights to another layer, then the layer has an additional output with the name "weights". In this case, the layer uses this output to pass the weights to other layers.

• If the OutputLearnables property specifies that the layer outputs biases to another layer, then the layer has an additional output with the name "bias". In this case, the layer uses this output to pass the bias to other layers.

Data Types: double

OutputNames — Output names

Read-only: {'out'} (default) | cell array of character vectors

This property is read-only.

Output names, stored as a cell array of character vectors.

The number of outputs depends on the OutputLearnables property:

• The layer, by default, has an output with the name "out" that corresponds to the layer output data.

• If the OutputLearnables property specifies that the layer outputs weights to another layer, then the layer has an additional output with the name "weights". In this case, the layer uses this output to pass the weights to other layers.

• If the OutputLearnables property specifies that the layer outputs biases to another layer, then the layer has an additional output with the name "bias". In this case, the layer uses this output to pass the bias to other layers.

Data Types: cell

Examples

Create Fully Connected Layer

Create a fully connected layer with an output size of 10 and the name fc1.

layer = fullyConnectedLayer(10,Name="fc1")

layer = 
  FullyConnectedLayer with properties:

                  Name: 'fc1'

   Hyperparameters
             InputSize: 'auto'
            OutputSize: 10
    OperationDimension: 'spatial-channel'

   Learnable Parameters
               Weights: []
                  Bias: []

  Show all properties

Include a fully connected layer in a Layer array.

layers = [ ...
    imageInputLayer([28 28 1])
    convolution2dLayer(5,20)
    reluLayer
    maxPooling2dLayer(2,Stride=2)
    fullyConnectedLayer(10)
    softmaxLayer]

layers = 
  6×1 Layer array with layers:

     1   ''   Image Input       28×28×1 images with 'zerocenter' normalization
     2   ''   2-D Convolution   20 5×5 convolutions with stride [1  1] and padding [0  0  0  0]
     3   ''   ReLU              ReLU
     4   ''   2-D Max Pooling   2×2 max pooling with stride [2  2] and padding [0  0  0  0]
     5   ''   Fully Connected   Fully connected layer with output size 10
     6   ''   Softmax           Softmax

Specify Initial Weights and Biases in Fully Connected Layer

To specify the weights and bias initializer functions, use the WeightsInitializer and BiasInitializer properties respectively. To specify the weights and biases directly, use the Weights and Bias properties respectively.

Specify Initialization Function

Create a fully connected layer with an output size of 10 and specify the weights initializer to be the He initializer.

outputSize = 10;
layer = fullyConnectedLayer(outputSize,'WeightsInitializer','he')

layer = 
  FullyConnectedLayer with properties:

                  Name: ''

   Hyperparameters
             InputSize: 'auto'
            OutputSize: 10
    OperationDimension: 'spatial-channel'

   Learnable Parameters
               Weights: []
                  Bias: []

  Show all properties

Note that the Weights and Bias properties are empty. At training time, the software initializes these properties using the specified initialization functions.

Specify Custom Initialization Function

To specify your own initialization function for the weights and biases, set the WeightsInitializer and BiasInitializer properties to a function handle. For these properties, specify function handles that take the size of the weights and biases as input and output the initialized value.

Create a fully connected layer with output size 10 and specify initializers that sample the weights and biases from a Gaussian distribution with a standard deviation of 0.0001.

outputSize = 10;
weightsInitializationFcn = @(sz) rand(sz) * 0.0001;
biasInitializationFcn = @(sz) rand(sz) * 0.0001;

layer = fullyConnectedLayer(outputSize, ...
    'WeightsInitializer',@(sz) rand(sz) * 0.0001, ...
    'BiasInitializer',@(sz) rand(sz) * 0.0001)

layer = 
  FullyConnectedLayer with properties:

                  Name: ''

   Hyperparameters
             InputSize: 'auto'
            OutputSize: 10
    OperationDimension: 'spatial-channel'

   Learnable Parameters
               Weights: []
                  Bias: []

  Show all properties

Again, the Weights and Bias properties are empty. At training time, the software initializes these properties using the specified initialization functions.

Specify Weights and Bias Directly

Create a fully connected layer with an output size of 10 and set the weights and bias to W and b in the MAT file FCWeights.mat respectively.

outputSize = 10;
load FCWeights

layer = fullyConnectedLayer(outputSize, ...
    'Weights',W, ...
    'Bias',b)

layer = 
  FullyConnectedLayer with properties:

                  Name: ''

   Hyperparameters
             InputSize: 720
            OutputSize: 10
    OperationDimension: 'spatial-channel'

   Learnable Parameters
               Weights: [10×720 double]
                  Bias: [10×1 double]

  Show all properties

Here, the Weights and Bias properties contain the specified values. At training time, if these properties are non-empty, then the software uses the specified values as the initial weights and biases. In this case, the software does not use the initializer functions.

Algorithms

Fully Connect Layer

A fully connected layer multiplies input vectors by a weight matrix and then adds a bias vector.

The layer operation flattens the dimensions specified by the OperationDimension property, then multiplies by the weights matrix and adds the bias vector for each element in remaining dimensions, independently. (since R2026a)

Before R2026a: The layer operation flattens the "S" (spatial), "C" (channel), and "U" (unspecified) dimensions of the input data, then multiplies by the weights matrix and adds the bias vector for each element in the "B" (batch) and "T" (time) dimensions, independently.

For each of the flattened vectors x of the input, the corresponding output vector y is given by

$$y_k={\displaystyle \sum _i{W}_{k,i}}{x}_i+b_k$$

where W and b denote the weights and bias, respectively, and i and k index over the flattened input operation dimension and the output dimension, respectively.

For example, for an array that represents a batch of sequences where the first, second, and third dimensions correspond to channels, observations, and time steps, respectively, and a fully connect operation that operates over the channel dimension only, the output is given by

$$Y_{k,n,t}={\displaystyle \sum _i{W}_{k,i}}{X}_{i,n,t}+b_k,$$

where X and Y are the input and output data, respectively, i, n, and t index over the input channels, observations, and time steps, respectively, and k indexes over the output dimension.

Layer Input and Output Formats

Most layers in a layer array or layer graph pass data to subsequent layers as formatted dlarray objects. The format of a dlarray object is a string of characters in which each character describes the corresponding dimension of the data. The format consists of one or more of these characters:

• "S" — Spatial

• "C" — Channel

• "B" — Batch

• "T" — Time

• "U" — Unspecified

For example, you can describe 2-D image data that is represented as a 4-D array, where the first two dimensions correspond to the spatial dimensions of the images, the third dimension corresponds to the channels of the images, and the fourth dimension corresponds to the batch dimension, as having the format "SSCB" (spatial, spatial, channel, batch).

You can interact with these dlarray objects in automatic differentiation workflows, such as those for:

• developing a custom layer

• using a functionLayer object

• using the forward and predict functions with dlnetwork objects

The output format depends on the input data and the OperationDimension property. This table shows the supported input formats of FullyConnectedLayer objects when the OperationDimension property is "spatial-channel" and the corresponding output format. If the software passes the output of the layer to a custom layer that does not inherit from the nnet.layer.Formattable class, or a FunctionLayer object with the Formattable property set to 0 (false), then the layer receives an unformatted dlarray object with dimensions ordered according to the formats in this table. The formats listed here are only a subset. The layer may support additional formats such as formats with additional "S" (spatial) or "U" (unspecified) dimensions.

Input Format	Output Format
"CB" (channel, batch)	"CB" (channel, batch)
"SCB" (spatial, channel, batch)
"SSCB" (spatial, spatial, channel, batch)
"SSSCB" (spatial, spatial, spatial, channel, batch)
"CBT" (channel, batch, time)	"CBT" (channel, batch, time)
"SC" (spatial, channel)	"CB" (channel, batch)
"SSC" (spatial, spatial, channel)
"SSSC" (spatial, spatial, spatial, channel, batch)
"SCBT" (spatial, channel, batch, time)	"CBT" (channel, batch, time)
"SSCBT" (spatial, spatial, channel, batch, time)
"SSSCBT" (spatial, spatial, spatial, channel, batch, time)
"CT" (channel, time)	"CT" (channel, time)
"SCT" (spatial, channel, time)
"SSCT" (spatial, spatial, channel, time)
"SSSCT" (spatial, spatial, spatial, channel, time)

The layer supports data with additional "U" (unspecified) dimensions. If the "U" (unspecified) dimensions are nonsingleton, then you must set OperationDimension to a positive integer. (since R2026a)

Before R2026a: Additional "U" (unspecified) dimensions must be singleton.

If the input data is a formatted dlarray object and the OperationDimension property is a positive integer, then the output format matches the input format.

This layer also supports operating on unformatted data directly and returns unformatted data. In this case, the OperationDimension property value must be a positive integer. (since R2026a)

If InputLearnables specifies that the layer receives weights or biases from another layer, then the corresponding input weights or biases must be unformatted dlarray objects, or formatted dlarray objects with formats that contain only "U" (unspecified) dimensions.

If OutputFormats specifies that the layer outputs weights or biases to another layer, then the format of the corresponding output weights or biases depend on the values of InputLearnables:

• For any values of OutputLearnables that match values of InputLearnables, then the corresponding output data is identical to the corresponding input data.

• For any values of OutputLearnables that do not match values of InputLearnables, then the corresponding output data is unformatted.

References

[1] Glorot, Xavier, and Yoshua Bengio. "Understanding the Difficulty of Training Deep Feedforward Neural Networks." In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 249–356. Sardinia, Italy: AISTATS, 2010. https://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf

[2] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification." In 2015 IEEE International Conference on Computer Vision (ICCV), 1026–34. Santiago, Chile: IEEE, 2015. https://doi.org/10.1109/ICCV.2015.123

[3] Saxe, Andrew M., James L. McClelland, and Surya Ganguli. "Exact Solutions to the Nonlinear Dynamics of Learning in Deep Linear Neural Networks.” Preprint, submitted February 19, 2014. https://arxiv.org/abs/1312.6120.

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

• Code generation does not support passing dlarray objects with "U" (unspecified) dimensions to this layer.

• The OperationDimension property must be "spatial-channel".

• The InputLearnables and OutputLearnables properties must be empty.

GPU Code Generation

Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Refer to the usage notes and limitations in the C/C++ Code Generation section. The same usage notes and limitations apply to GPU code generation.

Version History

Introduced in R2016a

R2026a: Share learnable parameters with other layers

To share learnable parameters with other layers (also known as weight tying), use the InputLearnables and OutputLearnables properties:

• To create an input that receives learnables from other layers, use the InputLearnables property.

• To create an output that passes learnables to other layers, use the OuputLearnables property.

For more information, see Neural Network Weight Tying.

R2026a: Specify operation dimension

Specify the dimension of the input data that the layer operates over using the dim argument.

R2024a: DAGNetwork and SeriesNetwork objects are not recommended

Starting in R2024a, DAGNetwork and SeriesNetwork objects are not recommended, use dlnetwork objects instead.

There are no plans to remove support for DAGNetwork and SeriesNetwork objects. However, dlnetwork objects have these advantages and are recommended instead:

• dlnetwork objects are a unified data type that supports network building, prediction, built-in training, visualization, compression, verification, and custom training loops.

• dlnetwork objects support a wider range of network architectures that you can create or import from external platforms.

• The trainnet function supports dlnetwork objects, which enables you to easily specify loss functions. You can select from built-in loss functions or specify a custom loss function.

• Training and prediction with dlnetwork objects is typically faster than LayerGraph and trainNetwork workflows.

To convert a trained DAGNetwork or SeriesNetwork object to a dlnetwork object, use the dag2dlnetwork function.

Fully connected layers behave slightly differently in dlnetwork objects when compared to DAGNetwork and SeriesNetwork objects. Fully connected layers flatten the output. They encode the spatial data in the channel dimension by reshaping the output data. Fully connected layers in SeriesNetwork and DAGNetwork objects output data with the same number of the spatial dimensions as the input by outputting data with spatial dimensions of size one. Fully connected layers in dlnetwork objects remove the spatial dimensions of the output.

R2019a: Default weights initialization is Glorot

Starting in R2019a, the software, by default, initializes the layer weights of this layer using the Glorot initializer. This behavior helps stabilize training and usually reduces the training time of deep networks.

In previous releases, the software, by default, initializes the layer weights by sampling from a normal distribution with zero mean and variance 0.01. To reproduce this behavior, set the 'WeightsInitializer' option of the layer to 'narrow-normal'.