Growing Arrays Using the matfile Function

When writing a large number of large values to a MAT file, the size of the file increases in a nonincremental way. This method of increase is expected. To minimize the number of times the file must grow and ensure optimal performance though, assign initial values to the array prior to populating it with data.

For example, suppose that you have a writable MatFile object.

fileName = "matFileOfDoubles.mat";
matObj = matfile(fileName);
matObj.Properties.Writable = true;

Define parameters of the values to write. In this case, write one million values, fifty thousand at a time. The values should have a mean of 123.4, and a standard deviation of 56.7.

size = 1000000;
chunk = 50000; 
mean = 123.4;
std = 56.7;

Assign an initial value of zero to the last element in the array prior to populating it with data.

matObj.data(1,size) = 0;

View the size of the file.

s = dir("matFileOfDoubles.mat");
s.bytes

In this case, the size of matFileOfDoubles.mat is less than 5000 bytes. Assigning an initial value to the last element of the array does not create a large file. It does, however, prepare your system for the potentially large size increase of matFileOfDoubles.mat.

Write data to the array, one chunk at a time.

nout = 0;
while(nout < size)
    fprintf("Writing %d of %d\n",nout,size);
    chunkSize = min(chunk,size-nout);
    data = mean + std*randn(1,chunkSize);
    matObj.data(1,(nout+1):(nout+chunkSize)) = data;
    nout = nout + chunkSize;
end

View the size of the file.

s = dir("matFileOfDoubles.mat");
s.bytes

The file size is now larger because the array is populated with data.