what is the optimal way to build a sparse matrix from row/col index, and value data?

Hello,

To objectively frame the reason I'm asking this question, I should say that I am exploring some very large datasets that will be pushing the limits of my pc, at least I expect... So I want to find the solution to my question that provides the most efficient way to construct a sparse matrix from a matrix containing the row and column index, and contents values.

My data: From a mysql database, I am executing a fetch query which returns to matlab a Nx3 matrix, where there are N rows depending on the query.

What I need to do: I need to efficiently construct a sparse matrix from this data.

The values in the first column translate to the a unique row in the output sparse matrix. The values in the second column correspond to a unique column in the output sparse matrix. And values in the third column, represent the value to be placed at the corresponding cell withing the output sparse matrix.

So, in my Nx3 matrix there is not necessarily a value for every row and column, and thus I cannot use a simple reshape operation.

Is there a vectorized way to reconstruct the sparse matrix from row/column index and content data, the way that I've described?

Ideally the method of doing this might have matlab's code optimizer use a series of pointers. As I'm dealing with large datasets, I don't think my system will have adequate overhead resources to deal with any memory copy operations...

Any suggestions would be appreciated!

Thanks, Brian

Answer 1

[Sargondjani]

did you check 'sparse'? i think this will do what you want:

S = sparse(i,j,s) where i,j,s are your 3 column vectors...

http://www.mathworks.nl/help/techdoc/ref/sparse.html

[Brian]

Thanks, Sargondjani. That's exactly what I need.

Answer 2

[James Tursa]

Is the result of the query sorted? I.e., is it already in column major order the way MATLAB will naturally store it? (1,1) followed by (2,1) followed by (3,1) etc. Also, can you do the query so that you get three separate Nx1 matrices instead of a single Nx3 matrix? If all of that is true, then a mex routine could convert these vectors into a sparse matrix directly without copying the data at all (the sparse function will copy it).