Matlab is calculating standard deviation wrongly

I want to understand why Matlab is giving me a wrong answer when I ask for the standard deviation.

For example, I was trying to get the standard deviation of this matrix: [1 2 3;4 5 6;7 8 9].

So it should give me the standard deviation of the columns of this matrix, so it gives me the matrix [3,3,3].

It doesn't make sense, because the first column [1;4;7] gives me 3 and the standard deviation is sqrt 6.

what is happening? is Matlab using another formula for standard deviation other than the classical one? or it rounding the number sqrt6 to 3?

I need help, thank you!

[Johannes Hougaard]

Ehm - no...

As the average is

The standard formula for standard deviation

This would give you:

[Stephen23]

"is Matlab using another formula for standard deviation other than the classical one?"

Errr... it uses the classical one for a population sample where the normalization factor is N-1. You assumed the normalization factor for the entire population, where the factor is just N (which is arguably less useful because usually data is sampled from a larger population).

Answer 1

[Stephen23]

It is not wrong. The difference is explained in detail here:

https://en.wikipedia.org/wiki/Standard_deviation#Corrected_sample_standard_deviation

You can easily get the answer you want by following the advice given in the std documentation, which states "Some definitions of standard deviation use a normalization factor of N instead of N-1, which you can specify by setting w to 1." e.g.:

>> std(M,1)
ans =
    2.4495    2.4495    2.4495
>> sqrt(6)
ans =
    2.4495

Answer 2

[John D'Errico]

A = [1 2 3;4 5 6;7 8 9]
A =
     1     2     3
     4     5     6
     7     8     9
>> std(A)
ans =
     3     3     3

It works just fine.

std([1 4 7])
ans =
     3
sqrt(6)
ans =
       2.4495

I think what you misunderstand is there are indeed two ways to compute the standard deviation of a set of numbers. (This gets into basic probability and statistics. Hey, you are using the standard deviation. A stats class might be useful.)

The difference is that betyween the standard deviation of a complete population, and the standard deviation of a random sample form a distribution. The two get different formulas, for a good reason. If you want to compute the standard deviation of the complete population, then used std, with the second argument as 1.

std([1 4 7],1)
ans =
       2.4495

When does the difference matter? The difference is seen when we try to compute the standard deviation of a complete population.

So if we have just the 6 numbers on the 6 faces of a fair die, the standard deviation of the population is

x = 1:6;
>> sqrt(sum((x - mean(x)).^2)/6)
ans =
       1.7078
       
>> std(1:6,1)
ans =
       1.7078       

As you can see, std does that, if we use 1 as the second argument. That causes std to use the formula that apparently you know, dividing by n inside the sqrt.

However, when you compute the standard deviation of a random sample of numbers, if you divide by n, thus the number of data points, this will give you a biased estimator. (Again, basic prob and stats.)

X = randn(10,1e7);

>> mean(std(X,1))
ans =
      0.92264
      
>> mean(std(X))
ans =
      0.97255

So above, I generated 1e7 sets of 10 numbers, sampled from a Gaussian with a unit standard deviation. If we use the formula where we divide by n (thus std(X,1)) we get a biased estimate. It tends to be a little low. However, the usual formula for a statistical sample, where the mean itself is ALSO an estimate, yields a better estimate of the standard deviation. Again, basic prob & stats.

But this is why there are TWO formulas for the standard deviation, and why you need to know which one to use and when.

Answer 3

[Tommy]

std calculates the sample standard deviation:

>> sqrt(((-3)^2 + (0)^2 + (3)^2)/(2))

ans =

     3

. sqrt(6) would be the population standard deviation:

>> sqrt(((-3)^2 + (0)^2 + (3)^2)/(3))

ans =

    2.4495