precision problem ? why ans for 10 ^ 5 * 0.0633 is 6.3299e+03
1 view (last 30 days)
Show older comments
anyway to fix this ?
>> 10 ^ 5 * 0.0633
ans =
6.329999999999999e+03
>> 10 ^ 6 * 0.0633
ans =
6.329999999999999e+04
0 Comments
Answers (2)
KALYAN ACHARJYA
on 26 Feb 2021
Edited: KALYAN ACHARJYA
on 26 Feb 2021
>> 10 ^ 5 * 0.0633
ans =
6.3300e+03
>> format shortG
>> 10 ^ 5 * 0.0633
ans =
6330
See the other Format options, you can set the different format the output display
1 Comment
Steven Lord
on 26 Feb 2021
Using a different display format masks the issue.
andy, can you post for me the exact decimal value of one divided by three? The only characters you're allowed to use to write that answer are the digits 0 through 9 and a single . for a decimal point. You're not allowed to say that the answer is anything repeating; you must write all the decimal places of your answer explicitly.
Now multiply the answer you gave me by three. The result will not be equal to one. If you had been able to write an infinite number of decimal places it would have been, but you didn't. You rounded off one divided by three and that roundoff error affected the answer.
James Tursa
on 26 Feb 2021
Edited: James Tursa
on 26 Feb 2021
Normal floating point arithmetic effects. See this link:
The root issue is that 0.0633 cannot be represented exactly in IEEE double floating point format, and downstream calculations using this will be affected as a result. Calculations that mathematically should result in a known value often do not. You need to write floating point code that is tolerant of these effects. E.g., this is the closest number to 0.0633 that IEEE double precision can represent exactly
>> num2strexact(0.0633,'fixed')
ans =
'0.0632999999999999951594276126343174837529659271240234375'
0 Comments
See Also
Categories
Find more on Logical in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!