How to avoid the ending drop of the movmean curve?

I'm playing around with the built-in movmean function, which creates the moving average series of given data.

I noticed that in some cases the movmean curve badly fits the data at the last values, consider the following example

data = [386 379 295 238 159 190 384 402 552 347 463 259 412 481 523 589 611 479 320 403 642];
plot(movmean(data,7)) % 7 since data are collected daily and show cycles of length 7
hold on
plot(data)
legend('movmean','data','Location','northwest')

Since data are increasing, the movmean is supposed to increase too, while it drops at the end.

Is there a way to avoid this behaviour?

[christina widyaningtyas]

Hi, I also need more clear explanation related this post

I run the script to calculate "Average data for period xx seconds"

P is pressure data,

P1mean = movmean(P_1(130:5000),120); %average for 120secs

my question, how to determine 130 and 5000? my prof told me that because at the beginning of the measurement, the data was discontinuous (a.png file), so He just deleted that part.

Does this mean, the red part (b.png file) is not included in the movmean calculation?

Any one can explained it more details how to determined the value 130 and 5000, please?

Answer 1

[Stephan]

Use the Endpoints option:

data = [386 379 295 238 159 190 384 402 552 347 463 259 412 481 523 589 611 479 320 403 642];
window = 7;
avgMove = nan(1,numel(data));
avgMove(window+1:end) = movmean(data,[window 0],'Endpoints','discard');

plot(avgMove) % 7 since data are collected daily and show cycles of length 7
hold on
plot(data)
hold off
legend('movmean','data','Location','northwest')

Also if you give only 1 value for the window the data is centered, which is usually not what is expected. See documentation for details.

[giannit]

Thanks for your answer, but why did you remove the first 7 points of the moving average series? Moreover, by plotting all we can see that your curve is far from the default moving average, seems like it does not smooth data very well, does it?

[Rik]

He didn't remove points. He padded the result, because the discard option will remove points where there isn't a complete window. The apparent mismatch is because of the different window. One is centered, the other looks back.

Also, you should try to be polite to people who are helping you. Your last line sounds like you think Stephan is 5, or you're 5 yourself.

[giannit]

@Rik Thanks, so if the moving average is centered then the curve appears shifted to the right? Moreover, you said that the discard option will remove points where there isn't a complete window, but also at the end of the data there is no a complete windows (3 points on the left and 3 on the right), so it shouldn't be that first 2 and last 2 points of the centered moving average curve were missing?

Sorry it was not my intention to be rude, english is not my native language and so I don't understand very well if a sentence sounds kind or not. Is a sentence ending with "isn't it?" or "does it?" rude?

[Steven Lord]

On the documentation page for the movmean function, look at the description of the k input argument. Imagine that the window of length 3 was centered over the 0 that is the first element of A.

If the value of the 'Endpoints' name-value pair argument was 'shrink' (the default) for that window movmean would take the average of the first two elements of A (0 and 2) and ignore the section of the window that's outside the vector A.

If the value was 'discard', movmean realizes the window extends beyond the end of the vector A, and so the result of that movmean call is two elements shorter than A. The first window gets thrown out, as does the last.

If the value was 'fill' then the elements in the window that are outside the vector A are treated as NaN, and if the value was a constant the elements would be treated as that constant.

A = [0 2 4 1 3 5 7];
y1 = movmean(A, 3) % 'shrink' is the default
y2 = movmean(A, 3, 'Endpoints', 'discard') % Matches y1(2:6)
y3 = movmean(A, 3, 'Endpoints', 'fill') % First and last are NaN, otherwise matches y1
y4 = movmean(A, 3, 'Endpoints', 7) % First element is (7+0+2)/3 = 3

Also note that Stephan used a window of [window 0] not a window of 7. Compare the pictures in the descriptions of the [kb kf] and k inputs on that documentation page for an illustration of the difference. That's why the first seven points were discarded.

[giannit]

@StevenLord thanks for explanation! I plotted all y1 to y4 and to me the best smoothing of data is y1, what do you think? However, it has the problem of the drop at the end which i'm trying to solve, and none of the proposed methods seems to work for this problem