Define optimization objective independently of the dimension

How do i define this objective function in matlab .the x are all vectors and i would like to solve this using the genetic algorithm.can matlab identify the function without expanding the function .what if i have 20 variables to deal deal with?

Minimze ∑_(i=1:n ) (A0i + A1*x_i1 + A2*x_i2 + A3*x_i3 - y )^2 + 1⁄6(B0 + B1*x_i1 + B2*x_i2 + B3*x_i3- z )^2

A AND B are my paramters.

Answer 1

[Matt J]

I don't know why you would want to use the genetic algorithm for such a simple quadratic function. However, the general answer to your question is that you should be using vectorized commands to write these expressions. I.e., you would put your x_ij in a matrix X and write the function something like below. Notice that this works regardless of the size of X.

   X=_____;
   y=_____;
   z=_____;

p_opt = ga(@(p) myfitness(p,X,y,z),______);

 function out=myfitness(params, X,y,z)

   n=size(X,2);
   A=params(1:n);
   B=params(n+1:end);

   out= norm(X*A-y)^2 +1/6*norm(X*B-z)^2 ;

 end

[Honey Adams]

Thank you for your answer matt. the quaprog couldn't solve this due to the problem being non_covex.

[Matt J]

I seem to remember, though, that we discussed doing it with fmincon. fmincon would be much better than ga for a problem like this, especially if you provide the gradient and Hessian, which ga doesn't allow you to do.

[Honey Adams]

is there a way to define the hessian and the linear function from a function without having to expand the whole function, finding then finding the second derivative for the quadprog/fmincon. when the variables are few it's faster to deal with, but with say twenty variables it takes time. I would be glad if you could help. Thanks in advance Matt.

[Matt J]

Didn't catch that. What is "the linear function" and why does it matter how many variables you have?

[Honey Adams]

Since I am not familiar with all the optimizations in the toolbox, I am trying different functions on each algorithm to understand how to implement them on different functions and when to use each.

[Matt J]

Th Hessian of a linear function is always zeros(N).

[Honey Adams]

I was going through the Matlab documentation on using the quadprog.I need to define the hessian and the linear part of the quadratic function and my linear constraints in order to use the qadrapog.I expanded the quadratic function and found the linear part and my hessian.H,f,A,b for the quadprog.

[Matt J]

What does "expanded" mean? What does the function look like when it is not expanded? How can you write down a quadratic function without knowing H and f from the very beginning?

[Honey Adams]

on why it matters with the number of variables my code becomes cumbersome as the number of variables increases.Because i exapnd and find the partial derivatives for each variable as the increases.assuming i have fifty variables,ids the a way to find the hessian without manually expanding and finding the partial derivtaives as well as the linear part?

[Honey Adams]

'expand' by doing away with brackets (A0i + A1*x_i1 + A2*x_i2 + A3*x_i3 - y )^2 + 1⁄6(B0 + B1*x_i1 + B2*x_i2 + B3*x_i3- z )^2

[Matt J]

A quadratic function written in terms of individual scalar variables x(i) looks like this

sum_i sum_j  x(i)*H(i,j) *x(j)/2    + sum_k f(k)*x(k)

From this form, you can read the entries H(i,j) and f(k) immediately.

In the specific case where you have a linear least squares expression

1/2*sum_i ( sum_j A(i,j)*x(j)-b(i) )^2

then in matrix vector form, the function is

1/2*|| A*x-b ||^2

and the H,f data is H=A.'*A and f=A.'*b

[Honey Adams]

Thank you so much, Matt. Did you see the code attached? That was the complexity I was referring to.tHIS I was manually doing it.

[Matt J]

Thank you for your answer matt. the quaprog couldn't solve this due to the problem being non_covex.

Incidentally, you should be using lsqlin(), not quadprog(), to solve this problem as it is more directly tailored to linear least squares optimization.

Also, The problem is convex, but depending on how small the eigenvalues of your H-matrix are, quadprog can have difficulty recognizing that.

Answer 2

[Torsten]

Write your objective as

(x*A-y).'*(x*A-y) + 1/6*(x*B-z).'*(x*B-z)

and expand.

I get

[A.', B.']*[x.'*x, 0; 0, 1/6*x.'*x]*[A;B] + [-2*y.'*x,-1/3*z.'*x]*[A;B] + constant term

This can directly used for quadprog.

Best wishes

Torsten.

[Honey Adams]

thank you very much

[Matt J]

It is inadvisable to use quadprog for linear least squares problems, particularly if the Hessian, in this case

H=2*[x.'*x, 0; 0, 1/6*x.'*x]

is close to singular. Use lsqlin instead.

[Torsten]

Why do you think H is close to singular ? Usually x has full column rank.

[Matt J]

Because of what the OP said here.

[Honey Adams]

Matt, I figured out the hessian. Good, you clarified it. I have to make effort to distinguish when to use which optimizer.i used quadprog because the original problem has linear constraints and the objective function, quadratic in nature as described in the Matlab documentation. But like you said, in some situations, my hessian is non-definite function non_convex and cant be solved with quadprog. Thank you, Thorsten. You teachers are making my study of Matlab great.Thank you all.