Support-vector machine, using kernel trick with quadprog solver
Show older comments
I am posting below my own implementation of SVM optimization model for classification on saheart data set. To run the code, Optimization Toolbox and saheart data set are needed (data set is available here https://web.stanford.edu/~hastie/ElemStatLearn/datasets/SAheart.data). Most tutorials about kernel trick use as a example small mock data sets with two attributes. They are, for me, hard to reproduce to a real data set, understand. Could you introduce changes to the code, so that it will use for example polynomial kernel? I notice that there is for example fitcsvm etc. but I would like to not use it.
% read data
filename = 'saheart.txt';
delimiterIn = ',';
headerlinesIn = 0;
dataSet = importdata(filename,delimiterIn,headerlinesIn);
% organise labels
Y = dataSet(:,10);
dataSet(:,10) = [];
for i = 1:462
if Y(i,1) == 0
Y(i,1) = -1;
end
end
% hessian, introduction of decision variables, f
decc = sym ('zeta', [462 1]);
syms omega1 omega2 omega3 omega4 omega5 omega6 omega7 omega8 omega9 b
% tradeoff
C = 5;
f = 1/462*(sum(decc)) + C*power(omega1, 2) + C*power(omega2, 2) + C*power(omega3, 2)...
+ C*power(omega4, 2)+ C*power(omega5, 2)+ C*power(omega6, 2)+ C*power(omega7, 2)...
+ C*power(omega8, 2)+ C*power(omega9, 2);
x = [omega1, omega2, omega3, omega4, omega5, omega6, omega7, ...
omega8, omega9, b];
slackVariables = decc.';
x = [x slackVariables];
H = hessian(f,x);
H = double(H);
x = x.';
f = zeros(472, 1);
for i = 11:472
f(i,1) = 1/462;
end
% subjects related to hyperplane parameters
AA = zeros(462, 10);
for i = 1:462
AA(i,1) = (-1)*Y(i,1)*dataSet(i,1);
AA(i,2) = (-1)*Y(i,1)*dataSet(i,2);
AA(i,3) = (-1)*Y(i,1)*dataSet(i,3);
AA(i,4) = (-1)*Y(i,1)*dataSet(i,4);
AA(i,5) = (-1)*Y(i,1)*dataSet(i,5);
AA(i,6) = (-1)*Y(i,1)*dataSet(i,6);
AA(i,7) = (-1)*Y(i,1)*dataSet(i,7);
AA(i,8) = (-1)*Y(i,1)*dataSet(i,8);
AA(i,9) = (-1)*Y(i,1)*dataSet(i,9);
AA(i,10) = Y(i,1);
end
% subjects related to slack variables
AB = zeros(462, 462);
for i = 1:462
AB(i,i) = -1;
end
A = [AA AB];
% contstants vector
b = (-1)*ones(462,1);
% lower bounds
lb = zeros(472,1);
lb(1,1) = -Inf;
lb(2,1) = -Inf;
lb(3,1) = -Inf;
lb(4,1) = -Inf;
lb(5,1) = -Inf;
lb(6,1) = -Inf;
lb(7,1) = -Inf;
lb(8,1) = -Inf;
lb(9,1) = -Inf;
lb(10,1) = -Inf;
% launching the optimization
options = optimoptions('quadprog',...
'Algorithm','interior-point-convex','Display','off');
[x,fval,exitflag,output,lambda] = ...
quadprog(H,f,A,b,[],[],lb,[],[],options);
% inserting results to the hyperplane equation
HYPERPLANE_EQUATION = zeros(462,1);
for i = 1:462
HYPERPLANE_EQUATION(i,1) = x(1,1)*dataSet(i,1) + x(2,1)*dataSet(i,2) + x(3,1)*dataSet(i,3) + x(4,1) * dataSet(i,4) + ...
x(5,1)* dataSet(i,5) + x(6,1)* dataSet(i,6) + x(7,1)* dataSet(i,7) + x(8,1) * dataSet(i,8) + x(9,1)* dataSet(i,9) - x(10,1);
end
% computing some kind of accuracy
comparsion = [HYPERPLANE_EQUATION Y];
accuracy = 0;
for i = 1:462
if sign(comparsion(i,1)) == sign(comparsion(i,2))
accuracy = accuracy + 1;
end
end
accuracy = accuracy / 462
Answers (0)
Categories
Find more on Nonlinear Optimization in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!