Clear Filters
Clear Filters

Finding best parameters of SVM

2 views (last 30 days)
Jack
Jack on 5 Apr 2014
Answered: chit paing on 29 Dec 2017
Hi.
I’m designing a system that I can optimize parameters of a support vector machine (SVM) with genetic algorithm, harmony search and another optimization algorithms to find the best structure of SVM for a specific data. My problem is binary classification with 0 and 1 output and I normalize data (mapmaxmin o mapstd) before insert it to system. Besides it in some cases I use dimension reduction (for example FDA) to reduce my features. For this normalized data I must set the boundary of searching space in optimization algorithm. This is my SVM function:
svmstruct=svmtrain(TrainInputs,TrainTargets,...
'boxconstraint',Penalty,...
'kernel_function','rbf','method','QP',...
'rbf_sigma',Sigma,...
'autoscale','false');
I optimize only 'boxconstraint' and ‘rbf sigma’. For boxconstraint, my algorithm is searching in [0.001 400] and for sigma the searching space is same [0.001 400]. IS this searching boundaries is suitable for my problem or I must change these boundaries? Otherwise, I set ‘autoscale’ to ‘false’. Which one is better in my problem? ‘false’ or ‘true’ ?
I set kernel function to rbf. is this a good approach for this problem?
Thanks.

Accepted Answer

Ilya
Ilya on 6 Apr 2014
Plot the classification error (or whatever it is you are optimizing) against the two optimized parameters. Is the minimum well inside the box defined by your bounds? If yes, you likely found the global minimum.
If you set 'autoscale' to false, the SVM model is more sensitive to variables with large variance and less sensitive to variables with small variance. Is this what you want? Or would you rather treat your variables on equal footing, no matter how their variance values compare? If the latter, set 'autoscale' to true.
In my experience, when variables are standardized, the optimal kernel width is often close to 1.
To optimize the box constraint, I would normally use a uniformly spaced logarithmic scale, say from 1e-6 to 1e+6.
  4 Comments
Jack
Jack on 7 Apr 2014
Edited: Jack on 7 Apr 2014
Thank you again.
My system select best inputs (besides parameters of SVM) so I don’t have specific number of variables and it is changing in every iteration. But my initial inputs variables are 21. (So my input variables are between 1 and 21).
So I have to set this boundary in my system. According to what was explained what is your opinion in this case?
Thanks.
Ilya
Ilya on 7 Apr 2014
OK, let me be more specific. The optimal value would also depend on the kind of predictors you have (categorical, discrete, continuous), their correlations with the class label, and their mutual correlations.
If I were in your shoes, I would just play with as much data pertinent to my specific analysis as I can to see what works best for me. Asking such a specific question to people who never looked at your data won't take you far.

Sign in to comment.

More Answers (1)

chit paing
chit paing on 29 Dec 2017
rather good solution

Categories

Find more on Statistics and Machine Learning Toolbox in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!