If you create a custom deep learning layer, then you can use
the checkLayer
function
to check that the layer is valid. The function checks layers for validity, GPU compatibility,
correctly defined gradients, and code generation compatibility. To check that a layer is valid,
run the following command:
checkLayer(layer,validInputSize)
layer
is an instance of the layer, validInputSize
is a
vector or cell array specifying the valid input sizes to the layer. To check with multiple
observations, use the ObservationDimension
option. To check for code generation compatibility, set the
CheckCodegenCompatibility
option to 1
(true). For large input sizes, the gradient checks take longer to run. To speed up the tests, specify a smaller valid input size.Check the validity of the example custom layer preluLayer
.
The custom layer preluLayer
, attached to this is example as a supporting file, applies the PReLU operation to the input data. To access this layer, open this example as a live script.
Create an instance of the layer and check that it is valid using checkLayer
. Set the valid input size to the typical size of a single observation input to the layer. For a single input, the layer expects observations of size hbywbyc, where h, w, and c are the height, width, and number of channels of the previous layer output, respectively.
Specify validInputSize
as the typical size of an input array.
layer = preluLayer(20); validInputSize = [5 5 20]; checkLayer(layer,validInputSize)
Skipping multiobservation tests. To enable tests with multiple observations, specify the 'ObservationDimension' option. For 2D image data, set 'ObservationDimension' to 4. For 3D image data, set 'ObservationDimension' to 5. For sequence data, set 'ObservationDimension' to 2. Skipping GPU tests. No compatible GPU device found. Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options. Running nnet.checklayer.TestLayerWithoutBackward .......... .. Done nnet.checklayer.TestLayerWithoutBackward __________ Test Summary: 12 Passed, 0 Failed, 0 Incomplete, 16 Skipped. Time elapsed: 0.18741 seconds.
The results show the number of passed, failed, and skipped tests. If you do not specify the ObservationsDimension
option, or do not have a GPU, then the function skips the corresponding tests.
Check Multiple Observations
For multiobservation input, the layer expects an array of observations of size hbywbycbyN, where h, w, and c are the height, width, and number of channels, respectively, and N is the number of observations.
To check the layer validity for multiple observations, specify the typical size of an observation and set the ObservationDimension
option to 4.
layer = preluLayer(20); validInputSize = [5 5 20]; checkLayer(layer,validInputSize,ObservationDimension=4)
Skipping GPU tests. No compatible GPU device found. Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options. Running nnet.checklayer.TestLayerWithoutBackward .......... ........ Done nnet.checklayer.TestLayerWithoutBackward __________ Test Summary: 18 Passed, 0 Failed, 0 Incomplete, 10 Skipped. Time elapsed: 0.19972 seconds.
In this case, the function does not detect any issues with the layer.
The checkLayer
function checks the validity of a custom layer by performing a series of tests.
The checkLayer
function uses these tests to check the validity of custom
intermediate layers (layers of type nnet.layer.Layer
).
Test  Description 

functionSyntaxesAreCorrect  The syntaxes of the layer functions are correctly defined. 
predictDoesNotError  predict function does not error. 
forwardDoesNotError  When specified, the 
forwardPredictAreConsistentInSize  When 
backwardDoesNotError  When specified, backward does not error. 
backwardIsConsistentInSize  When

predictIsConsistentInType  The outputs of 
forwardIsConsistentInType  When 
backwardIsConsistentInType  When 
gradientsAreNumericallyCorrect  When backward is specified, the gradients computed
in backward are consistent with the numerical
gradients. 
backwardPropagationDoesNotError  When backward is not specified, the derivatives
can be computed using automatic differentiation. 
predictReturnsValidStates  For layers with state properties, the predict
function returns valid states. 
forwardReturnsValidStates  For layers with state properties, the forward
function, if specified, returns valid states. 
resetStateDoesNotError  For layers with state properties, the resetState
function, if specified, does not error and resets the states to valid
states. 
codegenPragmaDefinedInClassDef  The pragma "%#codegen" for code generation is
specified in class file. 
checkForSupportedLayerPropertiesForCodegen  The layer properties support code generation. 
predictIsValidForCodeGeneration  predict is valid for code generation. 
doesNotHaveStateProperties  For code generation, the layer does not have state properties. 
supportedFunctionLayer  For code generation, the layer is not a
FunctionLayer object. 
Some tests run multiple times. These tests also check different data types and for GPU compatibility:
predictIsConsistentInType
forwardIsConsistentInType
backwardIsConsistentInType
To execute the layer functions on a GPU, the functions must support inputs and outputs of
type gpuArray
with the underlying data type
single
.
The checkLayer
function uses these tests to check the
validity of custom output layers (layers of type
nnet.layer.ClassificationLayer
or
nnet.layer.RegressionLayer
).
Test  Description 

forwardLossDoesNotError  forwardLoss does not error. 
backwardLossDoesNotError  backwardLoss does not error. 
forwardLossIsScalar  The output of forwardLoss is scalar. 
backwardLossIsConsistentInSize  When backwardLoss is specified, the output of
backwardLoss is consistent in size:
dLdY is the same size as the predictions
Y . 
forwardLossIsConsistentInType  The output of 
backwardLossIsConsistentInType  When 
gradientsAreNumericallyCorrect  When backwardLoss is specified, the gradients computed
in backwardLoss are numerically correct. 
backwardPropagationDoesNotError  When backwardLoss is not specified, the derivatives
can be computed using automatic differentiation. 
The forwardLossIsConsistentInType
and
backwardLossIsConsistentInType
tests also check for GPU compatibility. To
execute the layer functions on a GPU, the functions must support inputs and outputs of type
gpuArray
with the underlying data type single
.
To check the layer validity, the checkLayer
function generates data depending on the type of layer:
Layer Type  Description of Generated Data 

Intermediate  Values in the range [1,1] 
Regression output  Predictions and targets with values in the range [1,1] 
Classification output  Predictions with values in the range [0,1]. If
you specify the If you do not specify
the 
To check for multiple observations, specify the observation
dimension using the ObservationDimension
option. If you specify the
observation dimension, then the checkLayer
function checks that the layer
functions are valid using generated data with minibatches of size 1 and 2. If you do not
specify this namevalue pair, then the function skips the tests that check that the layer
functions are valid for multiple observations.
If a test fails when you use checkLayer
,
then the function provides a test diagnostic and a framework diagnostic. The test
diagnostic highlights any issues found with the layer. The framework diagnostic provides
more detailed information.
The test functionSyntaxesAreCorrect
checks that the layer
functions have correctly defined syntaxes.
Test Diagnostic  Description  Possible Solution 

Incorrect number of input arguments for 'predict' in
Layer .  The syntax for the predict function is not
consistent with the number of layer inputs.  Specify the correct number of input and output
arguments in The
You can adjust the syntaxes for layers with multiple inputs, multiple outputs, or multiple state parameters:
Tip If the number of inputs to the layer can vary, then use If the number of outputs can vary, then use Tip If the custom layer has a 
Incorrect number of output arguments for 'predict' in
Layer  The syntax for the predict function is not
consistent with the number of layer outputs.  
Incorrect number of input arguments for 'forward' in
Layer  The syntax for the optional forward function
is not consistent with the number of layer inputs.  Specify the correct number of input and output
arguments in The
You can adjust the syntaxes for layers with multiple inputs, multiple outputs, or multiple state parameters:
Tip If the number of inputs to the layer can vary, then use If the number of outputs can vary, then use Tip If the custom layer has a 
Incorrect number of output arguments for 'forward' in
Layer  The syntax for the optional forward function
is not consistent with the number of layer outputs.  
Incorrect number of input arguments for 'backward' in
Layer  The syntax for the optional backward function
is not consistent with the number of layer inputs and
outputs.  Specify the correct number of input and output
arguments in The
You can adjust the syntaxes for layers with multiple inputs, multiple outputs, multiple learnable parameters, or multiple state parameters:
To reduce memory usage by preventing unused variables being saved between the forward and
backward pass, replace the corresponding input arguments with Tip If the number of inputs to If the number of outputs can vary, then use Tip If the layer forward functions support 
Incorrect number of output arguments for 'backward' in
Layer  The syntax for the optional backward function
is not consistent with the number of layer outputs. 
For layers with multiple inputs or outputs, you must set the values of the layer
properties NumInputs
(or alternatively,
InputNames
) and NumOutputs
(or
alternatively, OutputNames
) in the layer constructor function,
respectively.
The checkLayer
function checks that the layer functions are
valid for single and multiple observations. To check for multiple observations, specify the observation
dimension using the ObservationDimension
option. If you specify the
observation dimension, then the checkLayer
function checks that the layer
functions are valid using generated data with minibatches of size 1 and 2. If you do not
specify this namevalue pair, then the function skips the tests that check that the layer
functions are valid for multiple observations.
Test Diagnostic  Description  Possible Solution 

Skipping multiobservation tests. To enable checks with
multiple observations, specify the 'ObservationDimension'
parameter in checkLayer .  If you do not specify the
'ObservationDimension' parameter in
checkLayer , then the function skips the
tests that check data with multiple observations.  Use the command
For more information, see Layer Input Sizes. 
These tests check that the layers do not error when passed input data of valid size.
Intermediate Layers. The tests predictDoesNotError
,
forwardDoesNotError
, and
backwardDoesNotError
check that the layer functions do
not error when passed inputs of valid size. If you specify an observation
dimension, then the function checks the layer for both a single observation and
multiple observations.
Test Diagnostic  Description  Possible Solution 

The function 'predict' threw an
error:  The predict function errors when
passed data of size
validInputSize .  Address the error described in the
Tip If the layer forward functions support 
The function 'forward' threw an
error:  The optional forward function errors
when passed data of size
validInputSize .  
The function 'backward' threw an
error:  The optional backward function errors
when passed the output of
predict . 
Output Layers. The tests forwardLossDoesNotError
and
backwardLossDoesNotError
check that the layer functions
do not error when passed inputs of valid size. If you specify an observation
dimension, then the function checks the layer for both a single observation and
multiple observations.
Test Diagnostic  Description  Possible Solution 

The function 'forwardLoss' threw an
error:  The forwardLoss function errors when
passed data of size
validInputSize .  Address the error described in the
Tip If the 
The function 'backwardLoss' threw an
error:  The optional backwardLoss function
errors when passed data of size
validInputSize . 
These tests check that the layer function outputs are consistent in size.
Intermediate Layers. The test backwardIsConsistentInSize
checks that the
backward
function outputs derivatives of the correct
size.
The backward
function syntax depends on the type of layer.
dLdX = backward(layer,X,Z,dLdZ,memory)
returns the derivatives
dLdX
of the loss with respect to the layer input, where
layer
has a single input and a single output.
Z
corresponds to the forward function output and
dLdZ
corresponds to the derivative of the loss with respect
to Z
. The function input memory
corresponds to
the memory output of the forward function.
[dLdX,dLdW] = backward(layer,X,Z,dLdZ,memory)
also returns the
derivative dLdW
of the loss with respect to the learnable
parameter, where layer
has a single learnable parameter.
[dLdX,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory)
also
returns the derivative dLdSin
of the loss with respect to the
state input using any of the previous syntaxes, where layer
has a
single state parameter and dLdSout
corresponds to the derivative
of the loss with respect to the layer state output.
[dLdX,dLdW,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory)
also returns the derivative dLdW
of the loss with respect to the
learnable parameter and returns the derivative dLdSin
of the loss
with respect to the layer state input using any of the previous syntaxes, where
layer
has a single state parameter and single learnable
parameter.
You can adjust the syntaxes for layers with multiple inputs, multiple outputs, multiple learnable parameters, or multiple state parameters:
For layers with multiple inputs, replace X
and
dLdX
with X1,...,XN
and
dLdX1,...,dLdXN
, respectively, where N
is the number of inputs.
For layers with multiple outputs, replace Z
and
dLdZ
with Z1,...,ZM
and
dLdZ1,...,dLdZM
, respectively, where M
is the number of outputs.
For layers with multiple learnable parameters, replace dLdW
with dLdW1,...,dLdWP
, where P
is the
number of learnable parameters.
For layers with multiple state parameters, replace dLdSin
and dLdSout
with dLdSin1,...,dLdSinK
and
dLdSout1,...,dLdSoutK
, respectively, where
K
is the number of state parameters.
To reduce memory usage by preventing unused variables being saved between the forward and
backward pass, replace the corresponding input arguments with ~
.
Tip
If the number of inputs to backward
can vary, then use
varargin
instead of the input arguments after
layer
. In this case, varargin
is a cell array
of the inputs, where the first N
elements correspond to the
N
layer inputs, the next M
elements correspond
to the M
layer outputs, the next M
elements
correspond to the derivatives of the loss with respect to the M
layer
outputs, the next K
elements correspond to the K
derivatives of the loss with respect to the K
states outputs, and the
last element corresponds to memory
.
If the number of outputs can vary, then use varargout
instead of the
output arguments. In this case, varargout
is a cell array of the
outputs, where the first N
elements correspond to the
N
the derivatives of the loss with respect to the
N
layer inputs, the next P
elements correspond
to the derivatives of the loss with respect to the P
learnable
parameters, and the next K
elements correspond to the derivatives of
the loss with respect to the K
state inputs.
The derivatives dLdX1
, …, dLdXn
must be
the same size as the corresponding layer inputs, and
dLdW1,…,dLdWk
must be the same size as the corresponding
learnable parameters. The sizes must be consistent for input data with single
and multiple observations.
Test Diagnostic  Description  Possible Solution 

Incorrect size of 'dLdX' for
'backward' .  The derivatives of the loss with respect to the layer inputs must be the same size as the corresponding layer input.  Return the derivatives

Incorrect size of the derivative of the loss
with respect to the input 'in1' for
'backward'  
The size of 'Z' returned from 'forward' must be
the same as for 'predict' .  The outputs of predict must be the
same size as the corresponding outputs of
forward .  Return the outputs 
Incorrect size of the derivative of the loss
with respect to 'W' for 'backward' .  The derivatives of the loss with respect to the learnable parameters must be the same size as the corresponding learnable parameters.  Return the derivatives

Tip
If the layer forward functions support dlarray
objects, then the software automatically determines the backward function and you do not need to specify the backward
function. For a list of functions that support dlarray
objects, see List of Functions with dlarray Support.
Output Layers. The test forwardLossIsScalar
checks that the output of the
forwardLoss
function is scalar. When the
backwardLoss
function is specified, the test
backwardLossIsConsistentInSize
checks that the outputs of
forwardLoss
and backwardLoss
are of
the correct size.
The syntax for forwardLoss
is loss
= forwardLoss(layer,Y,T)
. The input Y
corresponds to the
predictions made by the network. These predictions are the output of the previous layer. The
input T
corresponds to the training targets. The output
loss
is the loss between Y
and T
according to the specified loss function. The output loss
must be
scalar.
If the forwardLoss
function supports
dlarray
objects, then the software automatically determines
the backward loss function and you do not need to specify the
backwardLoss
function. For a list of functions that
support dlarray
objects, see List of Functions with dlarray Support.
The syntax for backwardLoss
is dLdY
= backwardLoss(layer,Y,T)
. The input Y
contains the predictions
made by the network and T
contains the training targets. The output
dLdY
is the derivative of the loss with respect to the predictions
Y
. The output dLdY
must be the same size as the layer
input Y
.
Test Diagnostic  Description  Possible Solution 

Incorrect size of 'loss' for
'forwardLoss' .  The output loss of
forwardLoss must be a scalar.  Return the output 
Incorrect size of the derivative of loss 'dLdY'
for 'backwardLoss' .  When backwardLoss is specified, the
derivatives of the loss with respect to the layer input must
be the same size as the layer input.  Return derivative If the

These tests check that the layer function outputs are consistent in type and that the layer functions are GPU compatible.
If the layer forward functions fully support dlarray
objects, then the layer
is GPU compatible. Otherwise, to be GPU compatible, the layer functions must support inputs
and return outputs of type gpuArray
(Parallel Computing Toolbox).
Many MATLAB^{®} builtin functions support gpuArray
(Parallel Computing Toolbox) and dlarray
input arguments. For a list of
functions that support dlarray
objects, see List of Functions with dlarray Support. For a list of functions
that execute on a GPU, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
To use a GPU for deep
learning, you must also have a supported GPU device. For information on supported devices, see
GPU Support by Release (Parallel Computing Toolbox). For more information on working with GPUs in MATLAB, see GPU Computing in MATLAB (Parallel Computing Toolbox).
Intermediate Layers. The tests predictIsConsistentInType
,
forwardIsConsistentInType
, and
backwardIsConsistentInType
check that the layer functions
output variables of the correct data type. The tests check that the layer
functions return consistent data types when given inputs of the data types
single
, double
, and
gpuArray
with the underlying types
single
or double
.
Tip
If you preallocate arrays using functions such as
zeros
, then you must ensure that the data types of these arrays are
consistent with the layer function inputs. To create an array of zeros of the same data type as
another array, use the "like"
option of zeros
. For
example, to initialize an array of zeros of size sz
with the same data type
as the array X
, use Z = zeros(sz,"like",X)
.
Test Diagnostic  Description  Possible Solution 

Incorrect type of 'Z' for
'predict' .  The types of the outputs
Z1,…,Zm of the
predict function must be consistent
with the inputs X1,…,Xn .  Return the outputs

Incorrect type of output 'out1' for
'predict' .  
Incorrect type of 'Z' for
'forward' .  The types of the outputs
Z1,…,Zm of the optional
forward function must be consistent
with the inputs X1,…,Xn .  
Incorrect type of output 'out1' for
'forward' .  
Incorrect type of 'dLdX' for
'backward' .  The types of the derivatives
dLdX1,…,dLdXn of the optional
backward function must be consistent
with the inputs X1,…,Xn .  Return the derivatives

Incorrect type of the derivative of the loss
with respect to the input 'in1' for
'backward' .  
Incorrect type of the derivative of loss with
respect to 'W' for 'backward' .  The type of the derivative of the loss of the learnable parameters must be consistent with the corresponding learnable parameters.  For each learnable parameter, return the derivative with the same type as the corresponding learnable parameter. 
Tip
If the layer forward functions support dlarray
objects, then the software automatically determines the backward function and you do not need to specify the backward
function. For a list of functions that support dlarray
objects, see List of Functions with dlarray Support.
Output Layers. The tests forwardLossIsConsistentInType
and
backwardLossIsConsistentInType
check that the layer
functions output variables of the correct data type. The tests check that the
layers return consistent data types when given inputs of the data types
single
, double
, and
gpuArray
with the underlying types
single
or double
.
Test Diagnostic  Description  Possible Solution 

Incorrect type of 'loss' for
'forwardLoss' .  The type of the output loss of the
forwardLoss function must be
consistent with the input Y .  Return 
Incorrect type of the derivative of loss 'dLdY'
for 'backwardLoss' .  The type of the output dLdY of the
optional backwardLoss function must be
consistent with the input Y .  Return 
Tip
If the forwardLoss
function supports dlarray
objects, then the software automatically determines the backward loss function and you do not need to specify the backwardLoss
function. For a list of functions that support dlarray
objects, see List of Functions with dlarray Support.
The test gradientsAreNumericallyCorrect
checks that the
gradients computed by the layer functions are numerically correct. The test
backwardPropagationDoesNotError
checks that the derivatives
can be computed using automatic differentiation.
Intermediate Layers. When the optional backward
function is not specified, the
test backwardPropagationDoesNotError
checks that the
derivatives can be computed using automatic differentiation. When the optional
backward
function is specified, the test
gradientsAreNumericallyCorrect
tests that the gradients
computed in backward
are numerically correct.
Test Diagnostic  Description  Possible Solution 

Expected a dlarray with no dimension labels, but
instead found labels .  When the optional backward function is
not specified, the layer forward functions must output
dlarray objects without dimension
labels.  Ensure that any dlarray objects created
in the layer forward functions do not contain dimension
labels. 
Unable to backward propagate through the layer.
Check that the 'forward' function fully supports
automatic differentiation. Alternatively, implement the
'backward' function manually .  One or more of the following:
 Check that the forward functions
support Check that the derivatives of the input
Alternatively, define a custom backward
function by creating a function named

Unable to backward propagate through the layer.
Check that the 'predict' function fully supports
automatic differentiation. Alternatively, implement the
'backward' function manually .  
The derivative 'dLdX' for 'backward' is
inconsistent with the numerical
gradient .  One or more of the following:
 If the layer forward functions support
Check that the derivatives in
If the derivatives are correctly
computed, then in the If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic. 
The derivative of the loss with respect to the
input 'in1' for 'backward' is inconsistent with the
numerical gradient .  
The derivative of loss with respect to 'W' for
'backward' is inconsistent with the numerical
gradient . 
Tip
If the layer forward functions support dlarray
objects, then the software automatically determines the backward function and you do not need to specify the backward
function. For a list of functions that support dlarray
objects, see List of Functions with dlarray Support.
Output Layers. When the optional backwardLoss
function is not specified,
the test backwardPropagationDoesNotError
checks that the
derivatives can be computed using automatic differentiation. When the optional
backwardLoss
function is specified, the test
gradientsAreNumericallyCorrect
tests that the gradients
computed in backwardLoss
are numerically correct.
Test Diagnostic  Description  Possible Solution 

Expected a dlarray with no dimension labels, but
instead found labels  When the optional backwardLoss
function is not specified, the
forwardLoss function must output
dlarray objects without dimension
labels.  Ensure that any dlarray objects created
in the forwardLoss function does not
contain dimension labels. 
Unable to backward propagate through the layer.
Check that the 'forwardLoss' function fully supports
automatic differentiation. Alternatively, implement the
'backwardLoss' function manually  One or more of the following:
 Check that the Check that the derivatives of the input
Alternatively, define a custom backward
loss function by creating a function named

The derivative 'dLdY' for 'backwardLoss' is
inconsistent with the numerical
gradient .  One or more of the following:
 Check that the derivatives in
If the derivatives are correctly
computed, then in the If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic. 
Tip
If the forwardLoss
function supports dlarray
objects, then the software automatically determines the backward loss function and you do not need to specify the backwardLoss
function. For a list of functions that support dlarray
objects, see List of Functions with dlarray Support.
For layers with state properties, the test
predictReturnsValidStates
checks that the predict function
returns valid states. When forward
is specified, the test
forwardReturnsValidStates
checks that the forward function
returns valid states. The test resetStateDoesNotError
checks that
the resetState
function returns a layer with valid state
properties.
Test Diagnostic  Description  Possible Solution 

Error using 'predict' in Layer. 'State' must be
realvalues numeric array or unformatted dlarray
object .  State outputs must be realvalued numeric arrays or
unformatted dlarray objects.  Ensure that the states identified in the
Framework Diagnostic are realvalued
numeric arrays or unformatted dlarray
objects. 
Error using 'resetState' in Layer. 'State' must be
realvalues numeric array or unformatted dlarray
object  State properties of returned layer must be realvalued
numeric arrays or unformatted dlarray
objects. 
If you set the CheckCodegenCompatibility
option to
1
(true), then the checkLayer
function
checks the layer for code generation compatibility.
The test codegenPragmaDefinedInClassDef
checks that the layer
definition contains the code generation pragma %#codegen
. The
test checkForSupportedLayerPropertiesForCodegen
checks that the
layer properties support code generation. The test
predictIsValidForCodegeneration
checks that the outputs of
predict
are consistent in dimension and batch size.
Code generation supports intermediate layers with 2D image or feature input only. Code generation does not support layers with state properties (properties with attribute State
).
The checkLayer
function does not check that functions used by the layer
are compatible with code generation. To check that functions used by the custom layer also
support code generation, first use the Code Generation Readiness app. For more
information, see Check Code by Using the Code Generation Readiness Tool (MATLAB Coder).
Test Diagnostic  Description  Possible Solution 

Specify '%#codegen' in the class definition of
custom layer  The layer definition does not include the pragma
"%#codegen" for code generation. 
Add the 
Nonscalar layer properties must be type single or
double or character array for custom layer  The layer contains nonscalar properties of type other than single, double, or character array.  Convert nonscalar properties to use a representation of type single, double, or character array. For
example, convert a categorical array to an array of integers
of type 
Scalar layer properties must be numeric, logical, or
string for custom layer  The layer contains scalar properties of type other than numeric, logical, or string.  Convert scalar properties to use a numeric representation, or a representation of type logical or string. For example, convert a categorical
scalar to an integer of type 
For code generation, 'Z' must have the same number
of dimensions as the layer input .  The number of dimensions of the output
 In the 
For code generation, 'Z' must have the same batch
size as the layer input .  The size of the batch size of the output
 In the 