Detect boundaries seperating data blocks in a time series signal

I have a signal whose plot attached below (Figure 1). The expected boundaries are as shown in the second figure. I tried identifying the boundaries using findchangepts function (see code below). But the results (Figure 3) are not matching with the expected plot. Few boundaries are identified correctly. However, others are not. The data is attached.

Any ideas as to how to solve this.

import csv.*

avg_test = csvread('signal.csv');
b = fir1(128, 0.001, 'high');
y = filtfilt(b, 1, avg_test(:, 2));
[lpt,rsd] = findchangepts(y, 'Statistic','linear','MinThreshold',0.5,'MinDistance',4000);
vl = bsxfun(@times, ones(numel(lpt), 2), [min(avg_test(:, 2)) max(avg_test(:, 2))]);  
figure(1)
plot(avg_test(:, 1), avg_test(:, 2))
hold on
% plot(avg_test(:, 1), y)
hold on
x=avg_test(:, 1);
plot([x(lpt),x(lpt)]', vl', '-r', 'LineWidth',1.5)
hold off
grid

Thanks in advance.

Figure 1:Original plot

Figure 2:Expected plot

Figure 3: Obtained result

Answer 1

[William Rose]

@Ashley,

I am actually very impressed that findchangepts() did so well.

I recommend adjusting the optional arguments of findchangepts(). Maybe you can do better by varying those. If that still doesn;t work, then you will have to wrie your own code to implement the detection of the kinds of changes that matter to you.

The "Expected plot" above suggests that, in most but maybe not all cases, you are more interested in identifying when the activity increases abruptly than when it decreases abruptly.

In the cases where the "Expected" and "Obtained" plots differ, are you only concerned about the cases where the difference is "red line" versus "no red line"? Or are you also concerned abut the cases where the exact timing of the lines is slightly different? The answer will influe3nce what course you take.

There have been studies of the best algorithms for detecting when a muscle "turns on" and "turns off", using the surface electromyogram. It is possible that those algorithms could be useful to you. For example, a modified likelihood ratio method, Staude et al., 2001. Another approach is the Teager-Kaiser energy (TKE) operator, Li et al., 2007; Solnik et al., 2010. I have not kept up on that literature, so I would search for who has cited those papers, to find more recent developments.

[Ashley]

@William Rose

Thanks for the response.

I tried tuning the arguments of findchangepts() and the parameters in the code I provided is the one working best with my signal.

With respect to your question about how accurate should the red line be, For my application, the timing of the red line matters a lot. I am striving to attain an accuracy of atleast 95% however, the currect results gives only 80% accuracy (calculated as [1-( no. of error in boundary detection/total no. of boundaries)] *100).

[William Rose]

@Ashley,

How did you obtain the "Expected" plot? Your answer may point the way toward building a better change detector.

Your measure of accuracy evaluates each change point as an "error" or not an error. That make sense, since it provides a way to penalize false positive and false negative detections. I assume you have some small time window, and if the obtained and expected times are separated by less than that, the detection is graded "correct".