MATLAB Answers

0

How to skip NaN values in cross correlation between 2 time series datasets.

Asked by Chalita Forgotson on 1 Aug 2019
Latest activity Commented on by Chalita Forgotson on 1 Aug 2019
Hi. I need to conduct cross correlation analysis between 2 time series datasets. Both datasets contain NaN values. In matlab, xcorr wouldn't skip wherever NaN values occurred in the time series. Please help.
~Thanks
My data look like this:
Soil moisture (SM):
SM = [ay1-1, ay1-2, ay1-3, ay1-4, NaN, NaN, NaN, NaN, ay2-1, ay2-2, ay2-3, NaN, NaN, NaN,....]
Plant growth (PG):
PG = [by1-1, by1-2, by1-3, by1-4, NaN, NaN, NaN, NaN, by2-1, by2-2, by2-3, NaN, NaN, NaN....]
NaN values in both vectors only occurred outside growing season; and that wherever there is NaN in SM, there is NaN in PG.
I want to see if there is any delayed responses in PG to changes in SM within growing season (lag analysis).

  6 Comments

Does it mean that replacing NaNs with zeros wouldn't affect the correlation?
Depends by what you mean by not affecting the correlation. It would mean that the matching non-nan element in the other vector wouldn't be taken into account for that particular shift, since the result of the multiplication would be 0. That may indeed be useful to do that as long as there aren't too many NaNs, but it will definitively affect the correlation values.
Replacing the NaNs by interpolated values may be more useful though. It's probably what Joe suggests doing with fillmissing (if he meant to use the 'linear' method).
Thank you for the explanation. You're right that replacing NaNs with zeros would affect the correlation in the way you've explained. What I meant by not affecting the correlation was that the matching NaN/ non-NaN element in the other vector wouldn't be taken into account for that particular shift.
I added more details about my data and what I'm trying to do above.
~Thank you so much for your help.

Sign in to comment.

1 Answer

Answer by Joe Vinciguerra on 1 Aug 2019
Edited by Joe Vinciguerra on 1 Aug 2019

I encountered this same issue a few weeks ago. My solution was to use fillmissing() to patch up the NaN's.
I also used resample() to make sure my time vector was evenly distributed, but I suppose you could delete your NaN's all together, then resample.

  1 Comment

Thank you for your suggestion. "Fillmissing()" might not work in my case because my data are ranked.

Sign in to comment.