Testing and Deploying Deep Learning Models
Why Is Testing Important?
Deploying a deep learning model onto a target device requires certain performance characteristics. You want to be confident the model will work on unseen data and it is going to interact as expected with the other system components. This is where testing becomes important.
Validation and Test Accuracy
A high validation accuracy is a good start and provides you with some confidence that the model works. You don’t, however, necessarily know that it works on unseen data. Therefore, in addition to the validation data set used during training, often you will use a test data set to ensure the network accuracy is acceptable across the entire solution space.
Monte Carlo Approach
Sampling a subset of the solution space never guarantees a perfectly functioning network. You can use the Monte Carlo approach to sample the entire solution space. With synthesized test data, you can produce a dense sampling of the solution space.
System Simulation
The network is just part of the overall logic. For example, a high-five counter needs, at a minimum, a system that reads the sensor acceleration, preprocesses the sensor data, passes the data into the network, and counts labeled high fives. After you test the system in Simulink and are confident in the implementation, you can use Simulink Coder™ to build embedded C code and deploy the system.
Network Size
It doesn’t matter that your code works in simulation if it doesn’t work on the target hardware. To reduce network size or execution speed, you can start with a smaller pretrained network; for example, SqueezeNet (1 million parameters) instead of GoogLeNet (7 million parameters).
Quantization
If your network is still too large, you can reduce its size by pruning or quantizing it. Pruning is removing some of the network parameters that don’t contribute much to classifying your data. Quantizing means calculating the single- or double-precision weights and biases as 8-bit scaled integer data types. You can use the Deep Network Quantizer app for quantization and assessing the compression and its impact on accuracy.
The Takeaway
When considering deep learning for your application, think about the network architecture, access to training data, system simulation, and how to gain confidence in your network. There is no one answer for every project, but hopefully you have started seeing the benefits and possibilities of deep learning.
