Why is LASSO in MATLAB so slow in the case of highly correlated predictors?

12 views (last 30 days)
I am using LASSO based on 4-fold cross-validation in a regression problem. I observed that with an increasing number of predictors, the computation time for the MATLAB LASSO function increases dramatically, such that it becomes unfeasible for me (since I need to run the LASSO several 1000 times). E.g, for 100 predictors, LASSO needs mor than 60 sec. The same example in Python takes only few seconds. What could be the reason for such a difference in computation speed? ---added later: I observed that it is not the number of predictors that affects LASSO computation time, but the degree of colinearity in the predictors. MATLAB algorithm 'cDescentCycle' takes almost all the computation time. MATLAB help suggests using ELASTIC NET (set alpha < 1) in case of highly correlated predictors. ELASTIC NET is a bit faster,but is still unfeasible slow. I have not done further tests with LASSO implemented in python. I still don't know what to do to increase speed of LASSO in the case of highly correlated predictors (reducing the number of Lambda values or increasing the RelTol parameter does help only very little, ~few sec).

Accepted Answer

Ilya
Ilya on 1 Dec 2015
There could be many reasons. The lasso function has a lot of flexibility, so make sure you are comparing apples and apples. To make it run faster, you could
  1. Use fewer values of lambda.
  2. Increase the relative tolerance.
  3. Try standardizing or not standardizing predictors.
  4. Try running in parallel if you have a Parallel Computing Toolbox license.
The function would still be likely slower than C/C++ or Fortran code.
  2 Comments
Ilya
Ilya on 2 Dec 2015
Edited: Ilya on 2 Dec 2015
If you are willing to experiment a bit, try this. Find the cdescentCycle function inside lasso and replace line 799 (line numbers could be different in your version)
for j=find(active);
with these 3 lines:
a = find(active);
a = a(randperm(numel(a)));
for j=a
Does this help?

Sign in to comment.

More Answers (0)

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!