Timetable linear interpolation within a range

18 views (last 30 days)
I have a timetable with measurments every 16 seconds. I need to linearly interpolate the missing values, only if there are less than 10 consecutive NaN rows, otherwise it have to remain as missing values.
This is the line for interpolate my data:
TT2 = retime(TT2,'regular','linear','TimeStep',seconds(16));
But that line interpolates everything. I suppose it should be approached with some kind of for loop, and an if statement making the interpolation only if the amount of consecutive NaN's is smaller than 10
I would really appreciate your help! thanks in advance!
  3 Comments
Adam Danz
Adam Danz on 29 May 2019
Edited: Adam Danz on 29 May 2019
I see. I was expecting to see missing timestamps but clearly there are none (just missing data from some time stamps). I can update with some suggestions in a bit.
unique(diff(TT2.Fecha))
ans =
duration
00:00:16
% no missing time stamps

Sign in to comment.

Accepted Answer

Adam Danz
Adam Danz on 29 May 2019
Edited: Adam Danz on 30 May 2019
This solution has 3 steps:
  1. determine which rows are within 10 or more consecutive rows of missing data
  2. interpolate all missing data
  3. replace the rows identified in step 1 with NaN values
% Find consecutive rows of NaN that exceed threshold number allowed
hasNan = all(isnan(TT2.data),2);
dIdx = find(diff([0;hasNan;0]==1)); %rows that change 1/0
s1 = dIdx(1:2:end-1); %start indices of 1s
s2 = dIdx(2:2:end); %stop indices of 1s
keepNan = (s2-s1)>=10; %which segments have too many consecutive nans
hasNan(cell2mat(arrayfun(@(x1,x2)x1:x2, s1(~keepNan),s2(~keepNan)-1,'UniformOutput',false)')) = false;
% Interp all missing values
TT2intrp = retime(TT2,'regular','linear','TimeStep',seconds(16));
% TT2intrp = fillmissing(TT2,'linear'); % This gives you the same results as your line above
% Replace the NaN values for >10 consecutive rows of missing data
TT2intrp.data(hasNan,:) = NaN;
  2 Comments
Emanuel Valdes
Emanuel Valdes on 30 May 2019
It works perfect! you solved it in a way I wasn't expecting. Very clever! thanks!!

Sign in to comment.

More Answers (0)

Categories

Find more on Data Preprocessing in Help Center and File Exchange

Products


Release

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!