What's so useful about diagonalizing a matrix?

Question

I'm told the the purpose of diagonalisation is to bring the matrix in a 'nice' form that allows one to quickly compute with it. However in writing the matrix in this nice diagonal form you have to express it w.r.t. a new eigenvector basis. But you'll probably want the answer of your matrix multiplication written w.r.t. to the original basis, so you'll have to do a not-nice matrix multiplication regardless. Example of what I mean:

I want to compute $Ax$, where $A$ and $x$ are given w.r.t. the standard basis ($\epsilon$). However $A$ is quite large and annoying to compute, so I calculate $A_E$ which is a diagonal matrix written w.r.t. the eigenvector basis. But to compute $Ax$ using this matrix, I still have to compute the following: $$ _\epsilon S_E A_E\ _E S_\epsilon$$ Where the $S$ are basis-transition matrices, and those are quite likely to be at least as ugly as our original $A$, so I don't see what we're gaining here. If anything this seems to be a lot more work.

The only thing I can imagine being easier this way is computing something like $A^{10000}x$ or something, because $A^{10000}$ has a really easy form when $A$ is a diagonal matrix, while this would take forever if $A$ is not a diagonal matrix. But is this really the only purpose of diagonalisation; to compute things like $A^{10000}x$?

Answer 1

I think, in short, the purpose is more to provide a characterization of the matrix you are interested in, in most cases. A "simple" form such as diagonal allows you to instantly determine rank, eigenvalues, invertibility, is it a projection, etc. That is, all properties which are invariant under the similarity transform, are much easier to assess.

A practical example: principal components is an orthogonal diagonalization which give you important information regarding the independent components (eigenvectors) in a system and how important each component is (eigenvalues) - so it allows you to characterize the system in a way which is not possible in the original data. http://en.wikipedia.org/wiki/Principal_component_analysis

---

I can't think of a case where diagonalization is used purely as a means to "simplify" calculation as it is computationally expensive - it is more of an end goal in itself.

---

Answer 2

I'll add that while you mention computing integer powers of matrices, diagonalization helps in computing fractional powers and exponentiation. If you wanted to compute the matrix $\exp(\mathbf{A})$ you could either go the slow route and use the Taylor series giving: \begin{align}\mathbf{I}+\mathbf{A}+\frac{\mathbf{A}^2}{2!}+\frac{\mathbf{A}^3}{3!}+\dots+\frac{\mathbf{A}^n}{n!}+\dots\end{align}

or alternatively diagonalize $\mathbf{A}$ and it's as easy as doing $e^\lambda$ for each eigenvalue in the diagonal matrix. This significantly reduces the complexity for matrix exponentiation given a required precision.

Answer 3

Here are some situations where you need to compute the diagonal form.

1) First and foremost, diagonalisation is supposed to be applied to endomorphisms, and not matrices, meaning that a basis may not be given.

Example : consider $E$ the vector space of sequences $(u_n)_n$ such that $u_{n+3}=5u_{n+2} + u_{n+1} - u_{n}$. It is well-known that such a sequence is a linear combinaison of exponentials ($\lambda^n$). This comes from the fact that the operator $(u_n)_n \mapsto (u_{n+1})_n$ is a diagonalisable endomorphism on $E$, and $(\lambda^n)_n$ is the eigen-vector for $\lambda$.

2.1) Given a symetric matrix $A$, this can be viewed as a symetric bilinear form. You may want to know if it is a scalar product, and compute an orthognal basis, so you have to compute its diagonal form (but becareful that you need ${}_{\epsilon}S_{E}$ to be orthogonal).

2.2) Given an ellipse's equation $x^2+3y^2+4xy = 1$, you may want to know its axis.

3) Given a differential equation $X'=A.X$, you may want to know if the solutions go to $0$ or $\infty$ and in which directions.