createMLPNetwork

Create and initialize a Multi-Layer Perceptron (MPL) network to be used within a neural state-space system

Since R2022b

Syntax

dlnet = createMLPNetwork(nss,type)

dlnet = createMLPNetwork(___,Name=Value)

Description

dlnet = createMLPNetwork(nss,type) creates a multi-layer perceptron (MLP) network dlnet of type type to approximate either the state or the (non-trivial part of) the output function of the neural state space object nss. For example, to specify the network for the state function, use

nss.StateNetwork = createMLPNetwork(nss, 'state',...)

To specify the network for the non-trivial part of the output function, use

nss.OutputNetwork(2) = createMLPNetwork(nss, 'output',...)

dlnet = createMLPNetwork(___,Name=Value) specifies name-value pair arguments after any of the input argument in the previous syntax. You can use name-value pair arguments to set the number of layers, the number of neurons per layer, or the type of their activation function.

For example, dlnet = createMLPNetwork(nss,'output',LayerSizes=[4 3],Activations="sigmoid") creates an output network with two hidden layers having four and three sigmoid-activated neurons, respectively.

Examples

Create MLP Network for Neural State-Space Object

Use idNeuralStateSpace to create a continuous-time neural state-space object with three states and one input. By default, the state network has two hidden layers each with 64 neurons and an hyperbolic tangent activation function.

nss = idNeuralStateSpace(3,NumInputs=1)

Continuous-time Neural ODE in 3 variables
     dx/dt = f(x(t),u(t))
      y(t) = x(t) + e(t)

f(.) network:
  Deep network with 2 fully connected, hidden layers
  Activation function: Tanh
g(.) network:
  Deep network with 0 fully connected, hidden layers
  Activation function: 

Variables: x1, x2, x3

Status:                                                         
Created by direct construction or transformation. Not estimated.
More information in model's "Report" property.

Use createMLPNetwork and dot notation, to re-configure the state network. Specify three hidden layers of 4, 8 and 4 neurons, respectively, and use a sigmoid as activation function.

nss.StateNetwork = createMLPNetwork(nss,'state', ...
    LayerSizes=[4 8 4],Activations="sigmoid");

You can now use time-domain data to perform estimation and validation.

Input Arguments

nss — Neural state-space object

idNeuralStateSpace object

Neural state-space object, specified as an idNeuralStateSpace object.

Example: idNeuralStateSpace(2,NumInputs=1)

type — Network type

"state" | "output"

Network type, specified as one of the following:

• "state" or 'state' — creates a network to approximate the state function of nss. For continuous state-space systems the state function returns the system state derivative with respect to time, while for discrete-time state-space systems it returns the next state. The inputs of the state function are time (if IsTimeInvariant is false), the current state, and the current input (if NumInputs is positive).

• "output" or 'output' — creates a network to approximate the non-trivial part of the output function of nss. This network returns the non-trivial system output, y₂(t) = H(t,x,u), as a function of time (if IsTimeInvariant is false), the current state, and the current input (if NumInputs is positive). For more information, see idNeuralStateSpace.

Example: "output"

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: LayerSizes=[16 32 16]

Use name-value pair arguments to specify network properties such as the number of hidden layers, the size of each hidden layer, the activation functions, and the weights and bias initialization methods.

LayerSizes — Layer sizes

[64 64] (default) | vector of positive integers

Layer sizes, specified as a vector of positive integers. Each number specifies the number of neurons (network nodes) for each hidden layer (each layer is fully-connected). For example, [10 20 8] specifies a network with three hidden layers, the first (after the network input) having 10 neurons, the second having 20 neurons, and the last (before the network output), having 8 neurons. Note that the output layer is also a fully-connected, and you cannot change its size.

Example: [32 32]

Activations — Activation function type

"tanh" (default) | "sigmoid"

Activation function type for all hidden layers, specified as one of the following:

• "tanh" or 'tanh' — uses the hyperbolic tangent as activation function.

• "sigmoid" or 'sigmoid' — uses the sigmoid as activation function.

Example: "sigmoid"

WeightsInitializer — Weights initializer method

"glorot" (default) | "he" | "orthogonal" | "narrow-normal" | "zeros" | "ones"

Weights initializer method for all hidden layers, specified as one of the following:

• "glorot" or 'glorot' — uses the Glorot method.

• "he" or 'he' — uses the He method.

• "orthogonal" or 'orthogonal' — uses the orthogonal method.

• "narrow-normal" or 'narrow-normal' — uses the narrow-normal method.

• "zeros" or 'zeros' — initializes all weights to zero.

• "ones" or 'ones' — initializes all weights to one.

Example: "orthogonal"

BiasInitializer — Bias initializer method

"zeros" (default) | "ones" | "narrow-normal"

Bias initializer method for all hidden layers, specified as one of the following:

• "narrow-normal" or 'narrow-normal' — uses the narrow-normal method.

• "zeros" or 'zeros' — initializes all biases to zero.

• "ones" or 'ones' — initializes all biases to one.

Example: "narrow-normal"

Output Arguments

dlnet — Network for the state or output function

dlnetwork object

Network for the state or output function of nss, specified as a dlnetwork (Deep Learning Toolbox) object.

For continuous state-space systems the state function returns the system state derivative with respect to time, while for discrete-time state-space systems it returns the next state. The inputs of the state function are time (if IsTimeInvariant is false), the current state, and the current input (if NumInputs is positive).

The output function returns the system output as a function of time (if IsTimeInvariant is false), the current state, and the current input (if NumInputs is positive).

Note

You can use commands such as summary(dlnet), plot(dlnet), dlnet.Layers, and dlnet.Learnables to examine network details.

Version History

Introduced in R2022b