A singular value and corresponding singular vectors of a rectangular matrix A are, respectively, a scalar σ and a pair of vectors u and v that satisfy
where is the Hermitian transpose of A. The singular vectors u and v are typically scaled to have a norm of 1. Also, if u and v are singular vectors of A, then -u and -v are singular vectors of
A as well.
The singular values σ are always real and nonnegative, even if A is complex. With the singular values in a diagonal matrix Σ and the corresponding singular vectors forming the columns of two orthogonal matrices U and V, you obtain the equations
Since U and V are unitary matrices, multiplying the first equation by on the right yields the singular value decomposition equation
The full singular value decomposition of an m-by-n matrix involves:
m-by-m matrix U
m-by-n matrix Σ
n-by-n matrix V
In other words, U and V are both
square, and Σ is the same size as A. If
A has many more rows than columns (
m > n), then
m matrix U is
large. However, most of the columns in U are multiplied by zeros in
Σ. In this situation, the economy-sized
decomposition saves both time and storage by producing an
U, an n-by-n
Σ and the same V:
The eigenvalue decomposition is the appropriate tool for analyzing a matrix when it represents a mapping from a vector space into itself, as it does for an ordinary differential equation. However, the singular value decomposition is the appropriate tool for analyzing a mapping from one vector space into another vector space, possibly with a different dimension. Most systems of simultaneous linear equations fall into this second category.
If A is square, symmetric, and positive definite, then its eigenvalue and singular value decompositions are the same. But, as A departs from symmetry and positive definiteness, the difference between the two decompositions increases. In particular, the singular value decomposition of a real matrix is always real, but the eigenvalue decomposition of a real, nonsymmetric matrix might be complex.
For the example matrix
A = 9 4 6 8 2 7
the full singular value decomposition is
[U,S,V] = svd(A) U = 0.6105 -0.7174 0.3355 0.6646 0.2336 -0.7098 0.4308 0.6563 0.6194 S = 14.9359 0 0 5.1883 0 0 V = 0.6925 -0.7214 0.7214 0.6925
You can verify that
U*S*V' is equal to
A to within round-off error. For this small problem, the economy size decomposition is only slightly smaller.
[U,S,V] = svd(A,0) U = 0.6105 -0.7174 0.6646 0.2336 0.4308 0.6563 S = 14.9359 0 0 5.1883 V = 0.6925 -0.7214 0.7214 0.6925
U*S*V' is equal to
A to within round-off error.
For large sparse matrices, using
svd to calculate
all of the singular values and singular vectors is not always
practical. For example, if you need to know just a few of the largest singular values,
then calculating all of the singular values of a 5000-by-5000 sparse matrix is extra
In cases where only a subset of the singular values and singular vectors are required,
svdsketch functions are preferred
For example, consider a 1000-by-1000 random sparse matrix with a density of about 30%.
n = 1000; A = sprand(n,n,0.3);
The six largest singular values are
S = svds(A) S = 130.2184 16.4358 16.4119 16.3688 16.3242 16.2838
Also, the six smallest singular values are
S = svds(A,6,'smallest') S = 0.0740 0.0574 0.0388 0.0282 0.0131 0.0066
For smaller matrices that can fit in memory as a full matrix,
svd(full(A)) might still be
svdsketch. However, for
truly large and sparse matrices, using
svdsketch becomes necessary.