Adjusting a timeseries for clock drift with exact known start and stop times

Hello,

Apologies if this is an easy answer; I'm coming from other programming languages and can not figure out how to do this Matlab. I'm using 2017b, and I can't upgrade.

I deploy units to collect data for months at a time. Each file collected is timestamped. Some of the files are 3 hours long, some are 30 minutes. Some have duty cycles of no data collection, others record continuously. It varies based on the project.

Before each deployment we sync our recording units with a GPS clock to ensure that the time at the start of the deployment is accurate. This often happens right before the deployment, but can occur a few days before the unit is deployed depending on our preperation schedule. Therefore, there can be a few hours to a few days from that GPS time sync to the first timestamped file.

When we retrieve these units, we document the clock drift. Sometimes it's a few seconds, sometimes its a few minutes, but there's always some clock drift.

I would like to adjust the timestamps of the files to account for the clock drift, so the timestamps closer to GPS accurate.

For example:

%GPS Time stamp taken before deployment when the unit was synced with GPS time.
timeGPSreset=datetime('2019-03-19 09:47:30', 'Format','yyyy-MM-dd HH:mm:ss');
%Time unit stopped according to GPS
timeGPSstop=datetime('2019-08-12 09:19:40','Format','yyyy-MM-dd HH:mm:ss');
%Time unit stopped according to internal clock.
timeClockstop=datetime('2019-08-12 09:18:07','Format','yyyy-MM-dd HH:mm:ss');

%Timestamps of the files recorded. This unit was scheduled to start at midnight after it was deployed.

>> DateTime

DateTime = 

  2272×1 datetime array

   28-Mar-2019 00:00:00
   28-Mar-2019 01:00:00
   28-Mar-2019 02:00:00
   28-Mar-2019 03:00:00
   28-Mar-2019 04:00:00
   28-Mar-2019 05:00:00
   28-Mar-2019 06:00:00
   28-Mar-2019 07:00:00
   28-Mar-2019 08:00:00
   28-Mar-2019 09:00:00
   28-Mar-2019 10:00:00
   28-Mar-2019 11:00:00
   28-Mar-2019 12:00:00
   28-Mar-2019 13:00:00
   28-Mar-2019 14:00:00
   28-Mar-2019 15:00:00
   28-Mar-2019 16:00:00
   28-Mar-2019 17:00:00
   28-Mar-2019 18:00:00
   28-Mar-2019 19:00:00
   28-Mar-2019 20:00:00
   28-Mar-2019 21:00:00
   ...
   29-Jun-2019 08:03:27
   29-Jun-2019 09:03:27
   29-Jun-2019 10:03:27
   29-Jun-2019 11:03:27
   29-Jun-2019 12:03:27
   29-Jun-2019 13:03:27
   29-Jun-2019 14:03:27
   29-Jun-2019 15:03:27
   29-Jun-2019 16:03:27
   29-Jun-2019 17:03:27
   29-Jun-2019 18:03:27
   29-Jun-2019 19:03:27
   29-Jun-2019 20:03:27
   29-Jun-2019 21:03:27
   29-Jun-2019 22:03:27
   29-Jun-2019 23:03:27
   30-Jun-2019 00:03:27
   30-Jun-2019 01:03:27
   30-Jun-2019 02:03:27
   30-Jun-2019 03:03:27
   30-Jun-2019 04:03:27
   30-Jun-2019 05:03:27
   30-Jun-2019 06:03:27
   30-Jun-2019 07:03:27
   30-Jun-2019 08:03:27
   30-Jun-2019 09:03:27
   30-Jun-2019 10:03:27
   30-Jun-2019 11:03:27
   30-Jun-2019 12:03:27
   30-Jun-2019 13:03:27
   30-Jun-2019 14:03:27
   30-Jun-2019 15:03:27

I would have thought that I could interpolate to adjust the timeseries, but I can't seem to find a function that deals directly with datetime values. I've looked into using liner interpolation with datenum, but interp1 wants inputs of corresponding length, which I don't have. I just have the start and stop, and they don't directly line up.

Thoughts and/suggestions?

****** EDIT ********

OK, so I kept working on this. Please see solution below.

[the cyclist]

Your statement, "I would like to adjust the timestamps of the files to account for the clock drift, so the timestamps closer to GPS accurate" does not make it fully clear (at least to me) what data manipulation you want to perform. Constant shift in all the data points in a vector, by some fixed offset? Different offsets, so that you have uniform spacing in the new vector?

I think that the best thing you could do here is

• upload a very small (but representative) sample of all the input data
• describe more precisely (and using the input variable names) the manipulation you want
• tell us exactly what the output should be for the sample input

Then we can help you develop an algorithm, without any guesswork.

[Emily T. Griffiths]

Thank you for the feedback. As you can see, I think I solved my issue, but I will take care in the future to be clearer in my questions.

The data manipulation I wanted to perform was a constant shift, I believe, not a fixed offset. The clock drift gets worse as the deployment goes on. So the offset would be different per timestamp. Please take a look at how I solved the issue. I'd appreciate it if you see something I missed.

Answer 1

[Emily T. Griffiths]

OK, so I kept working on this. Honestly, I think typing it out helped me find the answer I was looking for. So I'm posting it here incase other's are having this issue, and/or folks find issues with my methods.

I knew that I wanted to do some form of interpolation, as I didn't want a fix adjustment to time, but a drifting one. I finally came across the linspace function, which allows me to create linerarly spaced variables of equal length between two timestamps (in my case, I chose a larger value, 5000, since I'm trying to correct for only ~90 seconds). After converting those values into datenums, I was then able to run an interpolation and convert it back in to a datetime vector.

Thanks to all that looked at this!

timeGPSdiff=datenum(linspace(timeGPSreset,timeGPSstop,5000));
timeClockdiff=datenum(linspace(timeGPSreset,timeClockstop,5000));
DTall_num=datenum(DateTime);
newDTall=datetime(interp1(timeClockdiff,timeGPSdiff,DTall_num),'ConvertFrom','datenum');
>> newDTall
newDTall = 
  2272×1 datetime array
   28-Mar-2019 00:00:05
   28-Mar-2019 01:00:05
   28-Mar-2019 02:00:05
   28-Mar-2019 03:00:05
   28-Mar-2019 04:00:05
   28-Mar-2019 05:00:05
   28-Mar-2019 06:00:05
   28-Mar-2019 07:00:05
   28-Mar-2019 08:00:05
   28-Mar-2019 09:00:05
   28-Mar-2019 10:00:05
   28-Mar-2019 11:00:05
   28-Mar-2019 12:00:05
   28-Mar-2019 13:00:05
   28-Mar-2019 14:00:05
   28-Mar-2019 15:00:05
   28-Mar-2019 16:00:05
   28-Mar-2019 17:00:05
   28-Mar-2019 18:00:05
   28-Mar-2019 19:00:05
   28-Mar-2019 20:00:06
   28-Mar-2019 21:00:06
   28-Mar-2019 22:00:06
   28-Mar-2019 23:00:06
   29-Mar-2019 00:00:06
   29-Mar-2019 01:00:06
   29-Mar-2019 02:00:06
   29-Mar-2019 03:00:06
   ...
   29-Jun-2019 08:04:31
   29-Jun-2019 09:04:31
   29-Jun-2019 10:04:31
   29-Jun-2019 11:04:32
   29-Jun-2019 12:04:32
   29-Jun-2019 13:04:32
   29-Jun-2019 14:04:32
   29-Jun-2019 15:04:32
   29-Jun-2019 16:04:32
   29-Jun-2019 17:04:32
   29-Jun-2019 18:04:32
   29-Jun-2019 19:04:32
   29-Jun-2019 20:04:32
   29-Jun-2019 21:04:32
   29-Jun-2019 22:04:32
   29-Jun-2019 23:04:32
   30-Jun-2019 00:04:32
   30-Jun-2019 01:04:32
   30-Jun-2019 02:04:32
   30-Jun-2019 03:04:32
   30-Jun-2019 04:04:32
   30-Jun-2019 05:04:32
   30-Jun-2019 06:04:32
   30-Jun-2019 07:04:32
   30-Jun-2019 08:04:32
   30-Jun-2019 09:04:32
   30-Jun-2019 10:04:32
   30-Jun-2019 11:04:32
   30-Jun-2019 12:04:32
   30-Jun-2019 13:04:32
   30-Jun-2019 14:04:32
   30-Jun-2019 15:04:32 

:D

Answer 2

[Cris LaPierre]

What about retime? That was introduced in R2016b.

[Emily T. Griffiths]

Thanks Cris! Unfortunately, retime isn't as flexable as I needed it to be, as it retimes based on a fixed value. Or at least, that's how I understood the documentation.