levenberg-marquardt with linear constraints

Hi, I just see the the box constraints are now allowed for the levenberg-marquardt algorithm in lsqnonlin. However it seems that linear constraints are still not implemented. Is that correct? Are there any plans to implement that in future realeases?

Cheers

[Bruno Luong]

Does such method exists and implemented on other optimization SW?

[Eze Far]

Hi, sorry I don't know the meaning of SW.

[Bruno Luong]

SW = software

[Eze Far]

Ah sorry, do you mean other than matlab? I don't really know that.

I just know that the levenberg-marquardt algorithm didn't allow for constraints (box, linear or non-linear) in previous matlab versions. In 2023 version, however, it does allow for box constraints, but apparently not for linear/non-linear. What I don't know is if it is due to some fundamental feature of the algorithm itself or it was just not implemented in this release.

[Bruno Luong]

AFAIK (As Far As I Know) it is not trivial to introduce constraints in LM algorithm. There are few papers on the topic but probably not mature enough to be implemented at the presence time.

[Eze Far]

ok I see. Thanks for the answer. Cheers

Answer 1

[Torsten]

x = lsqnonlin(fun,x0,lb,ub,A,b,Aeq,beq,nonlcon)

A,b,Aeq,beq define the linear constraints.

[Eze Far]

Yes, that's what I thought, but in https://nl.mathworks.com/help/optim/ug/lsqnonlin.html one can read:

"The 'interior-point' algorithm is the only algorithm that can solve problems with linear or nonlinear constraints. If you include these constraints in your problem and do not specify an algorithm, the solver automatically switches to the 'interior-point' algorithm."

I also tried myself and it switches automatically to interior-point when the linear constraints are enabled. That doesn't happen when only box constraints are set.

Answer 2

[John D'Errico]

I'm sorry, but nobody can tell you IF there are any plans to introduce some behavior in a future release, until it is released.

If someone does know, then they will be under a non-disclosure agreement. If they don't know, well what do you expect them to be able to tell you?

At best, you can submit a feature request to MathWorks. DIRECTLY TO THEM, NOT THROUGH ANSWERS. Answers is not technical support. Ask for this feature to be included in a future release. Remember that TMW will generally not write code that nobody ever wants or hopes to use. Instead, they will direct their efforts at things for which people have shown a need.

https://servicerequest.mathworks.com/mysr/cp_case_new1?cc=us

The nice thing is, I have always found them to be responsive to reasonable requests. In fact, if they do add the idea as a feature request, they will also attach your e-mail to it, so when it does come alive, you will be notified directly.

[Eze Far]

I'm not a lawyer, so I don't know what they can and cannot tell.

I don't expect enything. I was just asking, hoping that someone in the forum (not necessarily from matlab) can give any advice, maybe suggesting any unofficial function/script.

NO NEED TO WRITE IN CAPITAL LETTERS, since anyone can read normal letters.

I doubt that "nobody" will want to have linear/non linear constraints in the levenberg-marquardt algorithm.

Please, be more polite in answering next time. People will appreciate it.

[John D'Errico]

@Eze Far

Geez, RELAX!

1. TMW employees will be under an NDA. How can you not expect that?
2. I put it in caps, since far too many people seem to think Answers is indeed official tech support, so I wanted to make that part clear, drawing explicit attention to it. I could have made it bold face or italicized it instead. Would that seriously have made you happier? Really? Did my use of caps truly upset you? relax. relax. Do those make you happier?
3. I did not say that nobody will need that option, only that they may not have decided to expend the effort to add that option to the LM codes. Mybe they did not think it was necessary. Remember that every hour spent on one option is an hour not invested in development of other tools. And since they only have so many employees, this is a tradeoff they must always make. If they already offered a capability under one sub-optimizer in lsqnonlin, I can easily understand they might have chosen not to put it into all sub-optimzers in lsqnonlin. Honestly, it has been too many years since I worked seriously with Levenberg-Marquardt and writing optimization codes, that I don't know if there may be some explicit reason why they did not put it into the LM code. That they did not is a suggestion there was some reason not to do so, if it were easy.

Please develop a thicker skin, and don't get all upset when someone gives you some information that you clearly needed. As for your advice, I don't give a tinker's damn. In fact, I felt I was being quite reasonably polite. My feelings towards you are not so much anymore, but I doubt you care anymore than I care about yours. The fact is, you obviously learned several things from my response.

[Eze Far]

The following is a good answer: https://nl.mathworks.com/matlabcentral/answers/2006092-levenberg-marquardt-with-linear-constraints#comment_2843077