MATLAB array size limit error - why?

99 views (last 30 days)
Andres
Andres on 18 Oct 2015
Commented: Mishary Alfarah on 13 Apr 2019
I am receiving an unexpected error because of an array allegedly exceeding the maximum array size limit with a non-double type variable.
Here is a short example to reproduce the error:
First, to keep the example variables small, go to preferences -> MATLAB -> workspace and set the limit of the maximum array size temporarily to 1% of the RAM.
You may provoke the correct error with a double array demanding too much memory:
[~,sys] = memory;
testLen = round(sys.PhysicalMemory.Total/100/2);
N = zeros(testLen,1);
Error using zeros
Requested 62859407x1 (0.5GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.
(for 12GB of RAM)
A uint8 array of the same length is ok:
n = zeros(testLen,1,'uint8');
returns no error. You can manipulate it by
n(end) = uint8(1);
as expected.
N(end-5:end).'
gives
0 0 0 0 0 1
But why do
n(end-1:end) = uint8(1);
% or
n(end-1:end) = uint8([1;1]);
and similar manipulations error:
Requested 62859405x1 (0.5GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.
?
Is MATLAB (R2015b) simply miscalculating the array memory size for non-doubles here or am I doing something wrong?
  2 Comments
dpb
dpb on 18 Oct 2015
Looks like the former to me...like end is coded only for double and not the class of the array at hand would be the likely candidate for the problem.
I hadn't realized that was a user preference; don't see it in R2012b so gather mayhaps it is a newer introduction. If so, would seem quite possible it's not been too thoroughly test as yet; I'd suggest bug report is in order (altho you might want to see if anybody else has alternate explanations first).
Andres
Andres on 19 Oct 2015
Edited: Andres on 19 Oct 2015
Thanks for your comment, dpb. Meanwhile I'm convinced this is a bug or at least an unintended behaviour, too. I have sent a bug report.

Sign in to comment.

Answers (2)

Walter Roberson
Walter Roberson on 18 Oct 2015
n(end-1:end) = uint8(1) may require that n be duplicated if n is an array that is currently shared. For example if n is a parameter passed in, and is not being used in the restricted context
n = SomeFunction(n)
whose header is
function n = SomeFunction(n)
then MATLAB may well need to duplicate the array.

Andres
Andres on 19 Oct 2015
Edited: Andres on 19 Oct 2015
Thanks for your answer ( Link), Walter. The error first occured inside a function, but my example works in plain matlab workspace, too.
Based on your idea, I did some more analysis with different data types, trying out different test lengths. If the manipulation should temporarily duplicate the memory demand of a variable, the error occurence would depend on the data type size (8/16/32/64 bit).
clear k l m n p q
k = zeros(testLen,1,'uint8');
l = false(testLen,1);
m = zeros(testLen,1,'uint16');
n = zeros(testLen,1,'uint32');
p = zeros(testLen,1,'single');
q = zeros(testLen,1);
k(end-1:end) = uint8(1);
l(end-1:end) = true;
m(end-1:end) = uint16(1);
n(end-1:end) = uint32(1);
p(end-1:end) = single(1);
q(end-1:end) = 1;
Near the maximum length that will allow double (64bit) type creation, the allowableness of the manipulation of all smaller data types changes, too:
[~,sys] = memory;
testLen = floor(sys.PhysicalMemory.Total/100/8);
% maximum length that will work with all types (k l m n p q)
% x(end-1:end) = .. will work on all variables
testLen = floor(sys.PhysicalMemory.Total/100/8)+1;
% creates variables only up to 32bit types (k l m n p)
% (i.e. smallest length that will error on double creation)
% x(end-1:end) = .. will work on all variables created
testLen = floor(sys.PhysicalMemory.Total/100/8)+2;
% creates variables only up to 32bit (k l m n p)
% does NOT allow x(end-1:end) = .. on ANY of the variables created
(maximum array size was limited to 1% of RAM, as above)
I think this is not a reasonable or intended behaviour, so I tend to send a bug report.
Btw, is there a way to get or set the maximum array size including the percentage value found in the preferences tab programmatically? The preferences tab only shows a slider without displaying a value.
  6 Comments
Zoe
Zoe on 8 Feb 2018
I unchecked that limit check and then I could run my code. Thank you very much!
Mishary Alfarah
Mishary Alfarah on 13 Apr 2019
I uchecked that limit and on my activity monitor it showed that it took 63 GB of RAM! then it immediately quit. I think keeping it on maximum maybe is better than uncheck it.

Sign in to comment.

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!