When with rounding and precision

1 view (last 30 days)
I am calculating the following: p15 = 0.8683*1.15
When I round to 5 decimal places round(p15,5) the answer is 0.99854 since Matlab sees full precision as 0.9985449999999999.
You can confirm this by typing sprintf('%1.30f',.8683*1.15)
ans = 0.998544999999999900000000000000
Unfortunately, I need to match the output of a standard calculator (don't ask why) and need to have the output as 0.99855. How is this possible via Matlab?

Accepted Answer

Walter Roberson
Walter Roberson on 19 Feb 2016
round(p15 * 1e6)/1e6
By the way, you cannot see the full precision on MS Windows by using sprintf(), but you can on OS-X

More Answers (1)

MHN on 18 Feb 2016
What you said is not true! for multiplication the exact number is : 0.998545000000000
p15 = 0.8683*1.15;
r = round(p15,5);
then p15 = 0.998545000000000 and r = 0.998550000000000.
MHN on 19 Feb 2016
I did not mean that there is a "problem" in sprintf. I meant even when we just assign a value like a=0.1; and then we use sprintf('%1.50f',a) the exact value is not 0.1. So, as long as one use sprintf and the floating point system, it will not get the result 0.100000000000000000000000000000000000000000.

Sign in to comment.


Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!