Find first non NaN element in a column of a matrix

125 views (last 30 days)
Hi,
I have a realtively large Matrix with time series data. each column stands for one series. given a variable that defines which column I am interested in, I want to extract all non NaN element into a new Array.
Example
lets say myMatrix is 3000X100 and I am looking at column 15. In this column the first 15 elements are NaN and then 300 elements contain data and the rest of the column is NaN again.
I want to extract now exactly those 300 elements in a new Array, let's call it myVec and in Addition make sure it is sorted backwards (last element in myMatrix becomes first element in myVec).
any ideas?
thy

Accepted Answer

Sebastian Castro
Sebastian Castro on 17 Sep 2015
Sure, there's lots of functionality that can help you. Here is an example.
First, I made up some "fake" data for myself. I have a 100x100 matrix where, in the 15th column, I set elements 1 through 15 and 75 through 100 to NaN. You can try it with your data if you'd like:
x = rand(100);
x([1:15,75:100], 15) = NaN;
Next you can pick out the "valid" indices of this column by logical indexing using the "isnan" function. Note that you want the values that are NOT NaN, hence the "~".
validIndices = ~isnan(x(:,15))
You can use these to index into the 15th column of your data. Just to verify, you can display the size of the vector.
myVec = x(validIndices,15)
size(myVec)
Finally, if you want to flip the matrix, you can use the "flipud" (flip up-down) function:
myVec = flipud(myVec)
If you're one of those people who prefers borderline illegible one-liners to show off their MATLAB prowess, you could always do this:
myVec = flipud(x(~isnan(x(:,15)),15));
- Sebastian
  3 Comments
Locks
Locks on 17 Sep 2015
thanks guys, exactly what I was looking for, works perfectly!
Sebastian Castro
Sebastian Castro on 17 Sep 2015
No problem!
@ Guillaume: I was joking with that comment, and I typed it even before I saw your response! Maybe it came out weird because of that, so my apologies if it seemed offensive.
Indeed, it's nice to eliminate temporary variables especially for large matrices that eat up a lot of memory. I just wanted to explain things step-by-step first so it's easier to understand, and then provide the efficient way at the end :)

Sign in to comment.

More Answers (2)

Guillaume
Guillaume on 17 Sep 2015
This will get you all the non-nan elements of column col in a reverse order:
coldata = flipud(myMatrix(~isnan(myMatrix(:, col)), col))
This will do it for all the columns at once:
nonanmat = arrayfun(@(col) flipud(myMatrix(~isnan(myMatrix(:, col)), col)), ...
1:size(myMatrix, 2), ...
'UniformOutput', false);

Tim Jackman
Tim Jackman on 17 Sep 2015
Making an example matrix:
>> MyMatrix = zeros(3000,100);
>> MyMatrix(1:15,15) = NaN;
>> MyMatrix(16:315,15) = rand(300,1);
>> MyMatrix(316:end,15) = NaN;
Now pulling out only the 15th column:
>> ExtractVec = MyMatrix(:,15);
Using logical indexing, I can extract all values from this array that are not NaN:
>> myVec = ExtractVec(isnan(ExtractVec) == 0);
They will be sorted by their original index in the column. Use flipud to reverse the order:
>> myVec = flipud(myVec);
Hopefully this will help.
  1 Comment
Guillaume
Guillaume on 17 Sep 2015
Traditionally,
isnan(x) == 0
is written
~isnan(x)
I don't know how good matlab's JIT is at optimising the former, but theoretically it involves a conversion from double to logical before a logical comparison while the latter is just a logical inversion.

Sign in to comment.

Categories

Find more on Particle & Nuclear Physics in Help Center and File Exchange

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!