Why does the "accumarray" function take more memory and time when setting "issparse=true"?

I am using the "accumarray" function and I have noticed that it takes more memory and time when setting "issparse=true", even when the output is supposed to be sparse, like the following:

>> mat = rand(50000,5000);
>> ix = ones(numel(mat),2);
>> b = accumarray(ix,mat(:),[50000 5000],@sum,0,false); % fast and low memory usage
>> b = accumarray(ix,mat(:),[50000 5000],@sum,0,true); % slow and high memory usage

Why is this the case?

Answer 1

[MathWorks Support Team]

Working with sparse arrays is generally slower than working with dense arrays when the arrays that we are effectively creating are dense or relatively small. Indexing, reading, updating, etc., is generally much slower than doing the same operation on a dense matrix in this case.

For "accumarray" to be more efficient when the output is set to sparse, two conditions need to be satisfied

1. The output of "accumarray" will be sparse - this is satisfied by your example.
2. The input is also relatively small compared to the number of elements of the output - please see below for an example:

>> idx = randi(1e4, 100, 2);
>> x = randn(100, 1);
>> A = accumarray(idx, x, [1e4 1e4], @median, 0, true) % fast
>> A = accumarray(idx, x, [1e4 1e4], @median, 0, false) % slow

If you are trying to build a sparse matrix that sums elements with equal indices, then you should use directly "sparse(I,J,V,m,n)" instead of "accumarray(...,true)".

The advantage of "accumarray", compared to "sparse", is that you can aggregate values using a custom function (passing its handle). Passing a handle to "sum" is not recommended because it forces "accumarray" to call sum through its handle instead of using its built-in and much faster default implementation (of sum). So you can make the call faster by not passing the handle to sum (it can be replaced by an empty array []).

If you are using "accumarray" because you do not want the summation behaviour above, there is no easy workaround, except if the aggregation function can be implemented through multiple calls to "sparse". To illustrate, for aggregating by computing the mean of elements with equal indices:

>> M = sparse(I,J,V,m,n) .* spfun(@(x)1./x,sparse(I,J,1,m,n));   
    % which is  sum .* (1 ./ count)
    % implemented in suitable way for sparse matrices (i.e., that avoids computing 1/0s)
    % will sometimes be much faster than:
>> M = accumarray([I,J],V,[m,n],@mean,0,true);