How can I get rid of excess "for loop"

Hi, I have a problem about possibility. I solve my problem for lower values of variables but I confused when I try solve it for higher variables. I want to select one value between 10 variables in 24 times. That means I have 10^24 mathematical possibility. First I try it for 3 variables and for 2 times. Here is my code

In my exact problem, I have 24 t values.(t1...t24). Also each t has got 10 same variables. (t=[1 2 3 4 5 6 7 8 9 10 ]). So I think that I have to use for loop in 24 times. And this will be too difficult for me. So is there any easy way to solve it?

[bio lim]

http://www.mathworks.com/help/matlab/matlab_prog/vectorization.html

[Cedric]

One giga is 10^9 (for comparison). Only for saving your results using a double precision array (default in MATLAB, stored using 8 bytes per element), you would need 8*10^24/10^9 = 8*10^15 gigabytes of RAM.

So you may want to find another approach for solving your problem, preferably which doesn't involve computing all possibilities.

[Stephen23]

@mturker: it much easier for us if you do not give code as screenshots, but as plain text (properly formatted using the {} Code button that you will find just above the textbox).

[mturker]

Thanks for all the answers. That was my first question so i couldn't see how to add my codes so I preferred to attach screenshot.

"coffee" I will examine the link. "cedric" thank you for your advise but my priority is solve the question. After that I will improve my codes. "stephen" here is my codes. I dont want to use for loops in 24 times. If there is a simple way, please let me know it. Thank you all for your attention.

   t1 = [500 1000 1500];
    t2 = [500 1000 1500];
    price = [3 5];
    k = 1; 

    for i = 1:length(t1);
         for j = 1:length(t2);
           result(k, 1) =  t1(i) * price(1);
           result(k, 2) =  t2(j) * price(2);        
           k = k + 1;
         end
    end

[Walter Roberson]

Let me amplify Cedric's answer for a moment:

at 8 bytes per double, and 10^24 doubles to be stored, you need 8*10^24 bytes of storage to hold all of the answers. log2() of that is about 82.73, so you would need 83 bits of address space in order to be able to store that array. The maximum theoretically possible with a 64 bit computer is 64 bits, but in practice there is no x64 implementation that supports more than 48 bits of address space. Therefore if you had a really high end system packed with as much memory as a single x64 processor can handle, you would need over 28 million such systems just to store one copy of the output table. The output involved is 6.4 yottabyte (YB), which is to say 6.4 trillion terabyte.

So when Cedric says you may wish to find another approach, he is really saying there is no chance that you can succeed with your current approach.

I could provide you with an alternate approach that did not require nested for loops, but it would have exactly the same problem with the amount of output storage required. You need to cut down on your output by many orders of magnitude before you have any chance of success.

[Cedric]

Thank you Walter, and sorry for the poor wording of my comment above.

Mturker, when the memory usage of a numerical computation passes the gigabyte, that usually rings a bell. It generally means that we are not tackling the problem the right way. In practice, it is very rare that, by using a system with more memory (RAM), we will solve the problem. In short, it is because the memory consumption doesn't grow linearly with the size of the problem, loop indices, etc, but exponentially or more generally as a power function). In most computers, RAM is physically present as a series of sticks like this, inserted in specific slots on the motherboard. Most motherboards have between 2 to 8 slots, and each stick is usually 1GB to 16GB. That to say that there is a physical limit to the RAM that you can put in a computer, and it is usually between 4GB and 128GB.

In my comment, I divide the amount of RAM needed for storing one copy of your results array by 1GB, to give you an idea of how many 1GB RAM sticks that would use, and the result is 8 quadrillion. This tells you immediately that you won't be able to handle your problem the way you want, even if you e.g. use a more powerful machine or a smaller variable type (stored on 1 byte instead of 8 for example).

Answer 1

[Walter Roberson]

Notice that in your code, result(k,1) is independent of the value of the "for j" loop, and result(k,2) is independent of the value of the "for i" loop. Your code can then be replaced by

for i = 1 : length(t1)
  result1(i) = t1(i) * price(1);
end
for j = 1 : length(t2)
  result2(j) = t2(j) * price(2);
end

which in turn can be shortened to

result1 = t1 * price(1);
result2 = t2 * price(2);

Notice that the space for this is linear, not exponential.

Where are the minimum values? Well

[min1, idx1] = min(result1);

but you don't even need to calculate that. Instead:

if price(1) == 0
  min1 = 0; idx1 = 1;   %everywhere will be 0
elseif price(1) > 0
  [min1, idx1] = min(t1);
  min1 = min1 * price(1);
else
  [min1, idx1] = max(t1);
  min1 = min1 * price(1);
end

This generalizes: the minimum of all possible products of lists of values occurs where each list is either minimum or maximum (which of those to choose will depend upon the signs of the terms. When you have a series of minimums and maximums of lists, you can incrementally find the minimum and maximum of their products, using only two storage locations; it does not even require 2^25 products be found.

If you have sums, then remember that the smallest sum occurs where each of the terms is smallest.

If you have an optimization problem, testing each possible combination of values is only efficient for a small number of values and variables. After that you should be using an optimization routine. For example fminmax() or perhaps ga()

My speculation at the moment is that you are attempting to solve an electrical load distribution problem.

[mturker]

Thank you very much for your answer. Actually I have an optimization problem. And ı tried to solve simple example for understanding grid search method. The next step would be finding min and max value as you mentioned. I took a look your other comments to another questions about optimization and grid search. Thanks for your help

Answer 2

[bio lim]

This is how you can start.

 t1 = [500 1000 1500];
 t2 = [500 1000 1500];
 price = [3 5];
 ii = 1:length(t1);  % Assumed that your t1 and t2 have same lengths

 output_1=sort(repmat((t1(ii) * price(1))',3,1));
 output_2 = repmat((t2(ii)*price(2))',3,1);
 result = [output_1 output_2]