I need more precision from binocdf. Output appears to be rounded.

Hello. I'm attempting to assess the probabilities of binomial data in large data sets (greater than 100K data points) using the binocfd function. However, for extreme cases, the output is rounded. For example:

1-binocdf(1000,16200,0.5)
ans = 1

Instead of .9999999 etc. For my purposes, I need more precision in the output of probabilities. How do I prevent the function from rounding? I've tried Format Long and it doesn't appear to affect the output. Thank you!

Answer 1

[John D'Errico]

1. format has NO impact on the number, just how it is displayed.

2. Computations in MATLAB are done in double precision. There is no higher precision, unless you use a tool like the symbolic toolbox, or my own HPF or VPI tools. And none of them will work with binocdf. Therefore, you would need to re-write the binomial CDF. Not in itself a terribly difficult thing.

3. Computations as far out as binocdf(1000,16200,0.5) will be incredibly small numbers. So they will underflow. The result is effectively ZERO in double precision. For example, when one tries to compute binocdf(1000,16200,.5), the number (1/2)^16200 would be employed. This is roughly 2.06e-4877. A quick computation of the CDF (by summing 3 terms from the PDF, computations done using HPF and VPI) gives me roughly 3.8e-3249. Can you say underflow?

So you should consider if various approximations will suffice. Your computer is not infinitely powerful. Just because you can write down a computation that in theory is not zero, does not mean it is computable on your computer, using brute force in double precision.

[Daemonic]

Very helpful. Thank you.