Multilayer Shallow Neural Networks and Backpropagation Training

The shallow multilayer feedforward neural network can be used for both function fitting and pattern recognition problems. With the addition of a tapped delay line, it can also be used for prediction problems, as discussed in Design Time Series Time-Delay Neural Networks. This topic shows how you can use a multilayer network. It also illustrates the basic procedures for designing any neural network.

Note

The training functions described in this topic are not limited to multilayer networks. They can be used to train arbitrary architectures (even custom networks), as long as their components are differentiable.

The work flow for the general neural network design process has seven primary steps:

1. Collect data

2. Create the network

3. Configure the network

4. Initialize the weights and biases

5. Train the network

6. Validate the network (post-training analysis)

7. Use the network

Step 1 might happen outside the framework of Deep Learning Toolbox™ software, but this step is critical to the success of the design process.

Details of this workflow are discussed in these sections:

• Multilayer Shallow Neural Network Architecture

• Prepare Data for Multilayer Shallow Neural Networks

• Create, Configure, and Initialize Multilayer Shallow Neural Networks

• Train and Apply Multilayer Shallow Neural Networks

• Analyze Shallow Neural Network Performance After Training

• Use the Network

• Limitations and Cautions

Optional workflow steps are discussed in these sections:

• Choose Neural Network Input-Output Processing Functions

• Divide Data for Optimal Neural Network Training

• Shallow Neural Networks with Parallel and GPU Computing

For time series, dynamic modeling, and prediction, see this section:

• How Dynamic Neural Networks Work