Finding best parameters of SVM
39 views (last 30 days)
I’m designing a system that I can optimize parameters of a support vector machine (SVM) with genetic algorithm, harmony search and another optimization algorithms to find the best structure of SVM for a specific data. My problem is binary classification with 0 and 1 output and I normalize data (mapmaxmin o mapstd) before insert it to system. Besides it in some cases I use dimension reduction (for example FDA) to reduce my features. For this normalized data I must set the boundary of searching space in optimization algorithm. This is my SVM function:
I optimize only 'boxconstraint' and ‘rbf sigma’. For boxconstraint, my algorithm is searching in [0.001 400] and for sigma the searching space is same [0.001 400]. IS this searching boundaries is suitable for my problem or I must change these boundaries? Otherwise, I set ‘autoscale’ to ‘false’. Which one is better in my problem? ‘false’ or ‘true’ ?
I set kernel function to rbf. is this a good approach for this problem?
Ilya on 6 Apr 2014
Plot the classification error (or whatever it is you are optimizing) against the two optimized parameters. Is the minimum well inside the box defined by your bounds? If yes, you likely found the global minimum.
If you set 'autoscale' to false, the SVM model is more sensitive to variables with large variance and less sensitive to variables with small variance. Is this what you want? Or would you rather treat your variables on equal footing, no matter how their variance values compare? If the latter, set 'autoscale' to true.
In my experience, when variables are standardized, the optimal kernel width is often close to 1.
To optimize the box constraint, I would normally use a uniformly spaced logarithmic scale, say from 1e-6 to 1e+6.