Orthogonal diagonalization of symmetric matrices

If the matrix you want to diagonalize is symmetric, you needn't worry about whether it can be diagonalized.

If A is a symmetric matrix, then

its eigenvalues are always real

the algebraic multiplicity of each eigenvalue equals its geometric multiplicity.

Proof. (omitted)

So you can always diagonalize a symmetric matrix. The interesting thing is, you can always use an orthogonal matrix to do it. Here's why: to make P into an orthogonal matrix, you have to be able to choose its columns to be orthonormal. It's easy enough to make the basis of each eigenspace into an orthonormal basis (use the Gram-Schmidt process), but how do you know that vectors from different eigenspaces will be orthogonal?

If A is a symmetric matrix, then eigenvectors corresponding to different eigenvalues are orthogonal.

Proof. Suppose Ax = λx and Ay = μy , where λ ≠ μ and x and y are not 0.

Transpose the first equation: xTAT = λxT,

and multiply on the right by y: xTATy = λxTy.

Multiply the second equation on the left by xT: xTAy = μxTy.

Since A = AT, the left-hand sides of the last two equations are equal, so λxTy = μxTy, i.e. (λ – μ)xTy = 0. But λ ≠ μ, so xTy = 0. That says that x • y = 0, so x and y are orthogonal.

To actually carry out the diagonalization, you proceed as before; you just need to do a bit more work to make the diagonalizing matrix orthogonal.

To diagonalize a symmetric matrix by an orthogonal matrix:

Given a symmetric n x n matrix A

Find all the eigenvalues of A and a basis for each eigenspace.

Use the Gram-Schmidt process and normalization to find an orthonormal basis of each eigenspace.

Stack the vectors of all the orthonormal eigenspace bases into the columns of a matrix P.

The matrix P will be orthogonal. The matrix D = P–1AP = PTAP will be diagonal, and its diagonal entries will be the eigenvalues of A in the same order that you chose the corresponding eigenvector columns of P.