How to perform a fast fourier transform (fft) or Lomb-Scargle (plomb) on this data set?

Hi all,

I have this timetable with only two dataset: time series and temperature series. I want to calculate the frequency (per hour) spectra from the temperature signal but the temperature has some NaN values. Can anyone please suggest me on how to perform an fft or plomb on this dataset? Also, I am confused on which method is more applicable since the sampling time period is a constant = 15 minutes.

Any feedback will be much appreciated!!

Answer 1

[Jack]

Hi,

When dealing with time series data containing NaN values, there are a few different approaches you can take to calculate frequency spectra:

1. Remove the NaN values: One option is to simply remove the NaN values from the temperature signal before calculating the frequency spectra. You can do this using the "rmmissing" function in MATLAB, which removes all rows or columns containing NaN values from a table or timetable.
2. Interpolate the NaN values: Another option is to interpolate the NaN values in the temperature signal using a suitable method (such as linear interpolation). This will allow you to estimate the missing values and calculate the frequency spectra using the complete signal.

Regarding the method for calculating frequency spectra, both FFT and Lomb-Scargle methods can be used for evenly and unevenly spaced data. However, Lomb-Scargle method is preferred for unevenly spaced data since it can deal with gaps or unevenly spaced samples. For evenly spaced data, both methods can be used. The choice of method will depend on the specific characteristics of your data and your research question.

Here is an example code to calculate the frequency spectra of temperature signal using FFT method after removing NaN values:

% Load data into a timetable
data = readtimetable('data.csv');

% Remove NaN values from temperature signal
temp = rmmissing(data.temperature);

% Calculate FFT of temperature signal
Fs = 4; % Sampling frequency (per hour)
L = length(temp); % Length of signal
Y = fft(temp); % Compute FFT
P2 = abs(Y/L); % Compute two-sided spectrum
P1 = P2(1:L/2+1); % Compute single-sided spectrum
P1(2:end-1) = 2*P1(2:end-1); % Double amplitudes for all but the DC component
f = Fs*(0:(L/2))/L; % Define frequency vector

% Plot frequency spectrum
plot(f,P1)
xlabel('Frequency (per hour)')
ylabel('Amplitude')

Note that the sampling frequency Fs is set to 4 (corresponding to 15-minute intervals). You can adjust this value to reflect your specific sampling rate.