Sum, then filter, then re-separate data

Hi all,

Trying to get some help with a problem I'm having:

I have two vectors that, when summed together, create a signal that has noise (spikes) that I'm wanting to filter out.

I need to filter the summed signal and then revert it back to the two vectors such that they can be re-summed without any noise.

Not sure if this is possible or if anyone has any ideas? I've attached some images and a .mat file with the data contained.

Answer 1

[Star Strider]

It would help to have the sampling frequency.

I am not certain what you want to filter, however for now I am assuming that is the noise in the valley between the summed signals —

LD = load(websave('data','https://www.mathworks.com/matlabcentral/answers/uploaded_files/1141660/data.mat'))

LD = struct with fields:
    front: [20000×1 double]
     rear: [20000×1 double]

front = LD.front;
rear  = LD.rear;

Fs = 2000;                                                      % Default Sampling Frequency
Fn = Fs/2;                                                      % Nyquist Frequency
L = numel(front);
t = linspace(0, L-1, L)/Fs;                                     % Time Vector
NFFT = 2^nextpow2(L);                                           % For Efficiency
FTfr = fft([front rear]-mean([front rear]),NFFT)/L;             % Fourier Transform
Fv = linspace(0, 1, NFFT/2+1)*Fn;                               % Frequency Vector
Iv = 1:numel(Fv);                                               % Index Vector (One-Sided Fourier Transform)

figure
plot(Fv, abs(FTfr(Iv,:))*2)
grid
xlabel('Frequency (Units)')
ylabel('Magnitude (Units)')
title('Fourier Transform')
xlim([0 Fs/100])

lim1 = [min([front rear]); max([front rear])];                      % Signal Limits

Fco = 5;                                                            % Cutoff Frequency
fr_filt = lowpass([front rear], Fco, Fs, 'ImpulseResponse','iir');  % Filter Signals

lim2 = [min(fr_filt); max(fr_filt)];                                % Signal Limits
fr_filt = max(fr_filt-9, 0);                                        % Eliminate Baseline Transients
lim3 = [min(fr_filt); max(fr_filt)];                                % Signal Limits
fr_filt = fr_filt .* lim1(2,:)./lim3(2,:);                          % Rescale Data
lim3 = [min(fr_filt); max(fr_filt)];                                % Check Signal Limits

figure
plot(t, fr_filt)
grid
xlabel('Time')
ylabel('Amplitude')
title('Filtered Signals')

figure
plot(t, sum(fr_filt,2))
grid
xlabel('Time')
ylabel('Amplitude')
title('Summed Filtered Signals')

This lowpass filter eliminates the noise reasonably well, at the expense of eliminating some other high-frequency information. That can be fine-tuned (to an extent) by changing the cutoff frequency ‘Fco’ in the lowpass call. Since I am not certain what you want to filter out, I leave that fine-tuning to you. I will help as necessary to get the desired result.

NOTE — Since I do not have the actual sampling frequency, I have scaled everything here in terms of it so this code should work with a different sampling frequency without alteration.

EDIT — (2 Oct 2022 at 03:34)

Added known sampling frequency, adjusted ‘Fco’ and thresholded the result to attenuate the baseline transients after filtering.

.

[LukeJes]

Thank you so much for such a detailed reply!

The main target for the filter is those spikes in the valleys of the summed signal, yes. I am also using a sample rate of 2000Hz if that helps.

I should have mentioned that I had tried a lowpass filter as well. My concern, as seen here, is that using one that's strong enough will result in those wavelets at the bottom of the signal. I need it to flatline / remain ~ 0 between the rises in amplitude - sorry for not mentioning this originally.

Any ideas? Thanks again for your time!

[Star Strider]

My pleasure!

I added the sampling frequency (it is necessary in order to do the signal processing correctly), changed the value for ‘Fco’ and added the post-filtering threshold:

fr_filt = max(fr_filt-9, 0); % Eliminate Baseline Transients

to eliminate the transients at the baseline, then re-scaling them to their original amplitudes. That may introduce some distortion in the signal, however It is not possible to completely eliminate the baseline transients without this sort of approach, and have an effective filter. That is simply the physics of signal processing. I doubt that any other filtering approach (for example the Savitzky-Golay filter that I frequently use) would be able to do better than this filter.

I am not exactly certain what result you want. Experiment with ‘Fco’ to get there.

[LukeJes]

Exactly what I was looking for! Thanks again for your time.

[Star Strider]

As always, my pleasure!