Low performance of neural network using logsig for output layer

6 views (last 30 days)
I have a binary classification problem and using “newff” for it. There is a single output in my neural network structure. If this outputs is equal or greater than 0.5 it belongs to class 1 and if smaller than 0.5 it belongs to class 0. So my targets for every sample is 0 or 1. Besides it I am normalizing data with “Mapstd” or “mapmaxmin”.
When I use “tansig” transfer function for hidden layer(s) and “purelin” for output, classification accuracy of network is good but when I change “purelin” to “logsig” the classification accuracy is really bad (0.5) and the classification accuracy is 50% in all repeats. What is the problem? (I'm using "Trainlm" without any normalization for outputs)
When I checked outputs after training, many of them are greater than 0.5 .

Accepted Answer

Greg Heath
Greg Heath on 25 Apr 2014
newpr is the version of newff that is to be used for classification; newfit is for regression.
All three are obsolete (see the help and doc documentation).
Currently, patternnet is the version of feedforwardnet that is to be used for classification; fitnet is for regression.
The default output transfer function is determined according to the default output normalization function.
Therefore, for each of the two groups of 3, it is worthwhile for you to go to the help, doc and type examples in the documentation and compare the properties of the three. For the current three
default output normalization is mapminmax [-1,1]
default output transfer functions are tansig for patternnet and purelin for feedforwardnet and fitnet
Therefore, changing the output function of feedforwardnet from purelin to logsig will not
yield the correct results because the range of logsig is not {-1,1}.
If you want to use logsig, normalized targets must be in {0,1}; The best way to do this is just to use mapminmax BEFORE train to convert targets to { 0,1 }. Then disable mapminmax during training.
This may also be a way of using softmax. However, I'm not familiar with MATLAB's version.
Hope this helps.
Thank you for formally accepting my answer
Greg Heath
Greg Heath on 26 Apr 2014
Edited: Greg Heath on 26 Apr 2014
> Thank you for answer Greg.
In general, it helps when including
1. Version numbers of MATLAB and NNTBX
2. Selected code to clarify explanations
3. Results on a MATLAB dataset
help patternnet
doc patternnet
> I disabled normalization and other pre-processing in network with this > functions (For one hidden layer network) : > > net.inputs{1}.processFcns={}; > net.outputs{2}.processFcns={}; > > Are these codes true?
Why would you ask me instead of just using them and see what happens?
net = patternnet; % Using all defaults
processFcns1 = net.inputs{1}.processFcns
processFcns2 = net.outputs{2}.processFcns
net.inputs{1}.processFcns = {};
net.outputs{2}.processFcns= {};
processFcns1 = net.inputs{1}.processFcns
processFcns2 = net.outputs{2}.processFcns
> (I normalized input data before inset it to neural network with Mapstd or Mapminmax).
Pretraining Mapstd is useful for detecting outliers. However,if you are going to use Mapminmax why not stay with the default?
> I disabled outputs normalization because my targets are 0 and 1 so I > think i dot need output normalization. (This is true?)
Yes, provided you use logsig or softmax.
> (If this is true, why default of output normalization in "patternnet" is "mapminmax" !? All outputs are binary or vectorized binary!)
Probably because they didn't want to introduce an additional [0,1]
normalization for logsig (softmax was not available then).
I would prefer a [0,1] and logsig or softmax option for patternnet.
> After that I set hidden layer transfer function to "Tansig" and output > transfer function to "logsig", But when I set output transfer function to > "purelin" the classification rate is 2~6% higher than "logsig" . Why?
Dunno. You didn't show your code.
Perhaps optimal parameter settings are different.
> PS. I don't know why but default options of patternnet doesn't have good > performance in my case. I think "trainlm" is better than "trainscg" in my > binary classification case. (why?)
Dunno. I assume they made comparisons. However, I don't know the details.
Good Questions.
To make my life easy I use zscore pretraining for outlier detection and
then just accept the maxmin/tansig combination.
I haven't used MATLAB's softmax yet ... Having trouble getting 14a to work.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!