Fitting of raw data
6 views (last 30 days)
Show older comments
Mohamed Ibrahim
on 11 Sep 2019
Commented: John D'Errico
on 2 Oct 2019
Hi All,
I would like to fit the data shown in the attached photo. The two plots are from two temperature sensors (resistance sensor) against time. From the figure, at one point, you can see the lines start to broaden. This happens due to high variation of the temperature (resistance of the sensor) at low temperatures. This broadening is actually a sin function variation with very small frequency 1.15Hz (0.0145 min). I would like to find the best fit or someway to find the average of these points to elimenate the broadening.
Thanks![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/237912/image.jpeg)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/237912/image.jpeg)
2 Comments
Accepted Answer
John D'Errico
on 11 Sep 2019
Edited: John D'Errico
on 11 Sep 2019
You don't have any simple model that you can use, at least, you have not suggested any. And, even if somebody suggests using polyfit, DON'T BELIEVE THEM! A polynomial model will be poor here.
Therefore, you need to ue a spline model. A problem is the heteroscedasticity. (]Yes, it hurts even to try to type that word.) But I would suggest a simple set of weights that decrease linearly with time. So small weight means that you don't trust that point much. High weight means you think the point is likely to be accurate.
You have not included your actual data, only a picture. Else, I would show how to fit it using my SLM toolbox. But a wild guess as to the call would be...
W = 1.25*max(Time) - Time;
slm = slmengine(Time,SR,'knots',12,'increasing','on','weights',W,'plot','on');
If you don't want to see the curve fit plot, then don't turn it on. Note that you should ALWAYS plot your data, and the curve fit through it. Decide for yourself if the fit is adequate.
More knots will give you a better fit, but less smooth, chasing the bumps in the curve.
If you want it to follow those bumps in the top end, where the curve seems to jump down a little, then you can leave increasing to be 'off' (the default), or you can set a specific interval where it should only be on.
Find SLM for download (free of course) on the file exchange here:
Note that SLM does use the optimization toolbox.
A simpler alternative to SLM would be to use a moving window average or moving median filter. Without the data, it is difficult to know how well it would work. And of course, you would need to choose the window size. A Savitsky-Golay filter might also be reasonable. The problem with any of the latter tools is the window length might need to vary with Time, so where it is more noisy, you need a wider window to smooth things out.
2 Comments
John D'Errico
on 2 Oct 2019
I TOLD you that polyfit won't do anything of value. Sometimes people actually believe me.
If you usually get a poor fit using SLM, then you did not use it right.
As well, I would claim that your data has problems in it, that need to be corrected before you use ANY fitting tool.
For example, I ran your script, then I plotted the data.
- ALWAYS PLOT THE DATA.
- THINK ABOUT WHAT YOU SEE.
- Only then should you bother to use any fitting tool.
So here is your data.
whos
Name Size Bytes Class Attributes
SROhm1BCal 178898x1 1431184 double
SROhm2AAfter 178898x1 1431184 double
SROhm2ABefore 178898x1 1431184 double
SROhm2AMean 178898x1 1431184 double
Timems 178898x1 1431184 double
assigned 178898x3 4293552 double
Now plot it. I've just picked one curve.
plot(Timems,SROhm2AMean,'.')
![untitled.jpg](https://www.mathworks.com/matlabcentral/answers/uploaded_files/240808/untitled.jpeg)
What I see is a moderately nice curve, but one that then drops off due to somethign at the end. That drop-off will NOT be meaningfully fit. So if you do try to fit that with ANY tool, expect COMPLETE CRAPOLA. That is stated without any need to try to fit it.
Lets zoom in on the plot.
axis([156 158 800 1100])
grid on
![untitled.jpg](https://www.mathworks.com/matlabcentral/answers/uploaded_files/240809/untitled.jpeg)
Ok, so something completely different is happening here. If you hope to fit that fall-off, then think again. As well, if your hope is that you will be able to fit the bumps and wiggles in that curve, then you will neem to hope again. For example, if we look at this interior section:
![untitled.jpg](https://www.mathworks.com/matlabcentral/answers/uploaded_files/240810/untitled.jpeg)
Then are you looking to git the extremelely rapidly varying stuff there? Do you want to fit the jogs in the burve? OR are you looking to fit the overall shape?
Only you know what you meant by the statement that somethign did not fit well.
So first, get rid of the crap.
k = Timems <= 157.1;
plot(Timems(k),SROhm2AMean(k),'.')
![untitled.jpg](https://www.mathworks.com/matlabcentral/answers/uploaded_files/240811/untitled.jpeg)
SLM = slmengine(Timems(k),SROhm2AMean(k),'increasing','on','plot','on','knots',[0:10:50, 54:2:90, 95, 100:10:160]);
![untitled.jpg](https://www.mathworks.com/matlabcentral/answers/uploaded_files/240812/untitled.jpeg)
That seeems to be an eminently reasonable fit. If you want to get the same fit, with slightly more CPU time (only a fraction of a second) needed, then this would have worked as well, with no real thought necessary:
SLM = slmengine(Timems(k),SROhm2AMean(k),'increasing','on','plot','on','knots',75);
![untitled.jpg](https://www.mathworks.com/matlabcentral/answers/uploaded_files/240813/untitled.jpeg)
So I fail to see that you would have had a problem, IF you used the tool correctly, and thought about your data.
More Answers (1)
Mohamed Ibrahim
on 2 Oct 2019
Edited: Mohamed Ibrahim
on 2 Oct 2019
1 Comment
John D'Errico
on 2 Oct 2019
Please, learn to use COMMENTS! This is not an answer, just a response to my answer.
There is a tool in the toolbox to evaluate a spline. Use slmeval. If you had read the copius help and examples, you would have seen that.
As far as fitting the bumps there, you just need to use more knots. Or, you need to put sufficient knots in places where it is important.
Note that if you have 'increasing' set to be 'on', as I did in my example, then it cannot follow a decrease as you see there.
See Also
Categories
Find more on Get Started with Curve Fitting Toolbox in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!