Neural Network Input Layer Data Normalization ("rescale-zero-one")

Hello,

I'd like to understand how the data normalization feature of a deep learning input layer works.

If I e.g. train my network with the following input layer:

layers = [featureInputLayer(10, 'Normalization', 'rescale-zero-one')

... ];

How is the network afterwards going to normalize test data? Is it just going to normalize the test data to zero-to-one (which would falsify the results cause test data might have a different min-max range of their values) or does the network save the min-max parameters that were used to normalize the training data and applies these parameters to the test data, which means that if the test data contains some value that is bigger than any of the training data values it would be normalized to a number bigger than 1.

Thank you in advance

Answer 1

[Sai Pavan]

Hi Thomas,

I understand that you are trying to learn how the data normalization feature of a deep learning input layer works.

When using the “featureInputLayer” in MATLAB with the “rescale-zero-one” normalization option, the layer will normalize both the training and test data based on the minimum and maximum values calculated from the training data. If the test data contains values that are larger than any of the values seen during training, the normalization will still be applied using the maximum value observed during training. As a result, these test data values may be normalized to a value greater than 1. Please refer to the below documentation to know more about the “featureInputLayer”:

https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.featureinputlayer.html

The min-max normalization assumes that the distribution and range of values in the test data are similar to those in the training data. If the test data has a significantly different distribution or range, the normalization may not be appropriate, and it is advisable to consider alternative normalization strategies like standardization or retrain the model using a more representative training dataset.

Hope it helps.

Regards,

Sai Pavan