Trying to use retime to sum daily values, but I don't want days that only have NaN values to become 0

22 views (last 30 days)
Basically I have a timetable or rainfall values (a column with half-hourly time increments and a column with the rainfall at each time point). I used retime to get daily sums:
sum_rainfall_2018 = retime(rainfall_2018, 'daily','sum');
This mostly works, but I have a number of days that only have NaN values, and retime is summing those days as 0. I need them to stay NaN because I need to know where my gaps are. Is there a simple way to do this?

Answers (3)

Jonas
Jonas on 10 Jul 2021
weird, the normal behavior of sum is giving NaN if at least one of the summed values is NaN. try @sum instead of 'sum' and if this also does not work, use a self defined function which forces the exact output

Peter Perkins
Peter Perkins on 27 Jul 2021
In the doc for retime (admittedly, not really in flashing bold letters): "All the listed methods omit NaNs, NaTs, and other missing data indicators, except for func. To include missing data indicators, specify func as a function handle to a function that includes them when aggregating data."
Haven't tried it, but I believe that if you pass in @sum, not 'sum', you will get what you want.

Everett Snieder
Everett Snieder on 17 Mar 2022
OK, I figured it out... basically when you pass an empty array into sum(), it returns 0. So to handle big time gaps (larger than your timestep), you need to create a special sum() function and pass the handle to retime:
function y = sum2(x)
if isempty(x)
y = nan();
else
y = sum(x);
end
  1 Comment
Peter Perkins
Peter Perkins on 21 Mar 2022
Right, prod and sum return the "unitary element" when given an empty:
prod([])
ans = 1
sum([])
ans = 0
But I don't think you need to do this:
tt = timetable([1;2;NaN;4;NaN;NaN],'RowTimes',datetime(2022,3,[21;21;22;22;23;23]))
tt = 6×1 timetable
Time Var1 ___________ ____ 21-Mar-2022 1 21-Mar-2022 2 22-Mar-2022 NaN 22-Mar-2022 4 23-Mar-2022 NaN 23-Mar-2022 NaN
retime(tt,'daily','sum') % 'sum' removes NaNs
ans = 3×1 timetable
Time Var1 ___________ ____ 21-Mar-2022 3 22-Mar-2022 4 23-Mar-2022 0
retime(tt,'daily',@sum) % @sum does not
ans = 3×1 timetable
Time Var1 ___________ ____ 21-Mar-2022 3 22-Mar-2022 NaN 23-Mar-2022 NaN
retime(tt,'daily',@(x)sum(x,'omitnan')) % equivalent to 'sum'
ans = 3×1 timetable
Time Var1 ___________ ____ 21-Mar-2022 3 22-Mar-2022 4 23-Mar-2022 0
It may be that your desired behavior is, "remove NaNs before summing, unless all NaN, in which case return NaN." For that, you would need your own function, because the standard behavior of sum on empty is to return 0.

Sign in to comment.

Categories

Find more on Timetables in Help Center and File Exchange

Products


Release

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!