Unexpected behavior from FMINSEARCH when using zero or very small initial guess
5 views (last 30 days)
Show older comments
I am optimizing a function across a range of conditions; sometimes the optimum occurs at an "x" value very close to zero (e.g. 1e-4). I am using an automatic method to generate a guess that is close, to avoid local minima. However, if this guess is very small (e.g. on the order of 1e-4), FMINSEARCH terminates immediately, even though the function is obviously not at a local minimum. As far as I can tell, that's because the inital simplex is generated by taking a step that is proportional to the initial guess, which means the initial step is on the order 1e-6. Then both the tolX and tolFun are trivially met and the algorithm terminates.
Here's a simple example: using the example given in the documentation (Rosenbrock's function), it returns the correct minimum if given initial guess around [1 1]. But if you give it x0 = [1e-4 1e-4], it will terminate immediately, returing an "optimal" function value of 0.99 (should be zero).
This seems like bad behavior to me. Having read the code for FMINSEARCH, I understand why the user should never give a small guess like this, but that's not obvious from the documentation. Even worse, I'm finding occasionally I get bad behavior when using initial guesses of exactly zero, despite the efforts of FMINSEARCH to handle this case properly (but not scaling the simplex by the initial guess). It doesn't happen in the example below, but I've encountered it in more complex functions.
Example code:
fun = @(x)100*(x(2) - x(1)^2)^2 + (1 - x(1))^2;
x = fminsearch(fun,[0 0],optimset('Display','iter')), % terminates in 79 iterations, correct answer
x = fminsearch(fun,[1 1],optimset('Display','iter')), % terminates in 24 iterations, correct answer
x = fminsearch(fun,[1 1]*1e-4,optimset('Display','iter')), % terminates in 1 iteration, wrong answer
0 Comments
Answers (2)
Steven Lord
on 10 Feb 2023
Take a closer look at the message the last fminsearch call showed you that states why it completed the optimization.
Optimization terminated:
the current x satisfies the termination criteria using OPTIONS.TolX of 1.000000e-04
and F(X) satisfies the convergence criteria using OPTIONS.TolFun of 1.000000e-04
Roughly speaking, it was searching a small enough interval for the solution (TolX) and the function value wasn't changing by very much (TolFun) so it said "close enough." You could tighten the tolerances a bit (don't go crazy and try to specify them as 1e-100 or something) to tell fminsearch that it needs to try getting closer to the solution. You set these tolerances the same way you set the Display option.
0 Comments
Joshua
on 19 Apr 2024
I had the same problem where my starting values were very close to zero (1e-15) if not zero. It appears that the Fminsearch algorithm's first tests by changing the starting values by 5%.
So if 5% of your starting values is less than your specified tolerance and the resulting change in the objective function is also less than your specified tolerance, fminsearch will stop.
The algorithm first makes a simplex around the initial guess x0 by adding 5% of each component x0(i) to x0
0 Comments
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!