Computing the Inverse of Co-variance Matrix results in "Inf". What could be the problem?

I have Computed the Covariance Matrix in Matlab for one of my program. Then eventually i want to calculate the Mahalanobis Distance, which will need the Inverse of the same.

But the entries in my Covariance matrix are very small and many are zeros.

The message that i get when inverse is tried to be found is :

• Warning: Matrix is singular to working precision. *

what is the meaning of all this? how can i proceed further? Please suggest me.

regards

Prashanth

Answer 1

[Roger Stafford]

It is inherent in the definition of the Mahalanobis distance of a set of observations from a group of sets of observations, that as the variance and covariance of the group decreases, that distance increases. In the limit, if the group's covariance matrix is singular, meaning that one or more of its eigenvalues are zero, that distance becomes infinite. The warning you received is an indication that this might be true of your group of observations. You should examine the eigenvalues of your covariance matrix to see if this is so. In any event, try using the 'mahal' function in the Statistics Toolbox, which is specifically designed to compute this distance.

[Nadir Nibras]

I seem to be coming across the same problem because some of my varaiances are too small. Do you know if there is any other free toolbox to solve this problem and find the mahalanobis distance?

[Bruno Luong]

It is not the problem of "toolbox" but the Mahalanobis distance is ill-defined when the Co-variance Matrix is singular. In other word you system is not (well) observable.

Answer 2

[Shyam Sangaraju]

“Warning: Matrix is singular to working precision” indicates that you have encountered a round off error that demonstrates a fundamental problem with the way computers deal with numbers. Machines use a finite number of bits to represent any number (in MATLAB's case, 64 bits are used), if a matrix has very small numbers then some precision is lost because the computer has difficulty representing the numbers accurately. When you try to find the inverse of a near singular matrix, you may receive this warning.

Please use the MATLAB function “rcond” to find whether the matrix is near to singular or not. Following sample code demonstrates how to use “rcond” function:

if( rcond(A) < 1e-12 )

    % This matrix doesn't look good.
end

‘A’ is the matrix for which we are trying to find inverse.

In this example “1e-12” is used as cut off between singular and non-singular matrix, you can use a value that suits your need.