If each element of the 3D matrix takes one value of 4 discrete values. How can I obtain all possible combinations of this 3D matrix?

If each element of 3D matrix of dimensions (10,10,2) takes one value from this vector [0 0.625 1.25 2.5] . How can I obtain all possible combinations of this 3D matrix ?

Answer 1

[Walter Roberson]

With 4 possible values for each entry, the number of different possibilities would be 4^10 * 4^10 * 4^2, which would be 17592186044416 matrices, which is 2^44 matrices. Each matrix would require 10*10*2 = 200 doubles, and each double would require 8 bytes. The total storage requirements would exceed 2^54 bytes.

Unfortunately for you, no released version of the x64 architecture implements more than 48 address pins. You therefore cannot find any MATLAB system anywhere in the world that is able to access enough memory to construct that table.

This hints that you are working on an optimization problem in which you have been thinking that the best way to proceed is to try every possible combination and select the best. You do not have enough memory or time to do that, no matter how well it works "mathematically". You will need to find a different approach.

For example, you can use https://www.mathworks.com/help/gads/mixed-integer-optimization.html ga with integer constraints. The constraints you would use would be that each entry would be 1 to 4. Then, inside your objective function, instead of directly using the x value, you would use the x value to index into your vector to determine the numeric value to use.

[Ahmed Ibrahim]

I optimized the 3D matrix to one vector.If each element of vector with size (1,14) takes one value from this vector [0 0.625 1.25 2.5] . How can I obtain all possible combinations of this vector ?

[Walter Roberson]

vector = [0 0.625 1.25 2.5];

T = (uint32(0):uint32(2^28-1)) .';
nT = length(T);
idx = zeros(nT, 14, 'uint8');
for K = 1 : 14
  idx(:,K) = uint8(1) + uint8(bitget(T, 2*K-1)) + uint8(2) * uint8(bitget(T, 2*K));
end

M = vector(idx);

This will require that you have approximately 32 gigabytes of available memory. (If you used a different slower approach, it would be possible to cut it down to about 28 gigabytes, by not constructing the idx array explicitly.)

If you do not have 28 or more gigabytes of available memory in your system, then you cannot create the complete list of combinations.

Note that if you use this array for computations, chances are that you will need as much memory again to store temporary copies of whatever you are computing.

Chances are strong that you should be giving up on creating a complete list of all combinations and computing all possible solutions.

[Ahmed Ibrahim]

when I use this code . I have 2 errors The first one is zeros in idx = zeros(nT, 14, 'uint8'); and the second is out of memory What can I do ?

[Stephen23]

@Ahmed Ibrahim: change your algorithm.

[Adi Nor]

I have the same error. what will I change in this code ?

[Walter Roberson]

If you do not have 28 or more gigabytes of available memory in your system, then you cannot create the complete list of combinations, and no algorithm can change the fact that you do not have enough memory to do what you insist on doing.

What you need to change is to not create the complete list of all combinations of the values ahead of time. For example, instead of calculating all of the combinations and then calling some function YourFuntion() on the list of all combinations, you might be able to do:

vector = [0 0.625 1.25 2.5];

T = (uint32(0):uint32(2^14-1)) .';
nT = length(T);
znT = uint8(zeros(nT, 14));
idx = znT;
for K = 1 : 7
  idx(:,K) = uint8(1) + uint8(bitget(T, 2*K-1)) + uint8(2) * uint8(bitget(T, 2*K));
end

suffix_M = vector(idx);
clear idx

for H = T
  T1 = uint8(dec2bin(H, 14) - '0');
  idx = znT;
  for K = 1 : 7
    idx(:,K) = uint8(1) + uint8(bitget(T1, 2*K-1)) + uint8(2) * uint8(bitget(T1, 2*K));
  end

    prefix_M = vector(idx);
    clear idx

    this_M_subset = [prefix_M, suffix_M];

    these_results = YourFunction(this_M_subset);     %calculate results on a batch of M

    At this point, store these_results as your partial results.
    The best way to do that is going to depend upon what you
    are trying to do. For example if you are doing a minimization
    then you might only need to find the _best_ of the values
    and you might only ever need to store the best you have
    encountered so far. But we do not know, as you have not
    said what you are going to do with the list of combinations
    once you have it.

end

[Walter Roberson]

YourFunction = @(V) mean(V);      %just an example

vector = [0 0.625 1.25 2.5];

T = (uint32(0):uint32(2^14-1));
nT = length(T);
znT = uint8(zeros(nT, 7));
idx = znT;
for K = 1 : 7
    idx(:,K) = uint8(1) + uint8(bitget(T, 2*K-1)) + uint8(2) * uint8(bitget(T, 2*K));
end

suffix_M = vector(idx);
clear idx

wb = waitbar(0, 'processing');

for T1 = T
    waitbar(double(T1)/nT, wb);
    idx = znT;
    for K = 1 : 7
        idx(:,K) = uint8(1) + uint8(bitget(T1, 2*K-1)) + uint8(2) * uint8(bitget(T1, 2*K));
    end

      prefix_M = vector(idx);
      clear idx

      this_M_subset = [prefix_M, suffix_M];

      these_results = YourFunction(this_M_subset);

end

delete(wb);