Understanding diagonalization through change of basis

Question

I'm sorry if what I am asking is unclear, but I was hoping that somebody might be able to help me understand diagonalization a bit more by explaining it through the lens of a change of basis.

I understand the material for change of basis fairly well. The way I think of it is as a way translating vectors into an alternative perspective.

Now for diagonalization I see that the form $B = S^{-1}AS$ has returned and so I believe it must be analogous to change of basis in some way. For the change of basis, $S$ consisted of the basis vectors of $B$ and changed them into the basis vectors of $E$. Now, $S$ consists of the eigenvectors of the matrix. Therefore those vectors must form some sort of basis. However, I cannot wrap my head around what it means that $B$ (from $B = S^{-1}AS$) consists of the eigenvalues. I think it must stem from me having a shaky understanding of all things eigen to begin with.

Answer 1

If $D$ is a diagonal matrix with diagonal entries $\lambda_1, \ldots, \lambda_n$, then the eigenvectors are the standard basis vectors $e_1 = (1,0,\ldots, 0)$, $e_2 = (0,1,0, \ldots, 0)$, and so on. Specifically, check that $$D e_j = \lambda_j e_j$$ for each $j = 1,\ldots, n$. Geometrically, this means that the linear transformation defined by $D$ (i.e. sending vectors $v$ to $Dv$) just "stretches/shrinks" along each standard basis direction, according to the values of $\lambda_j$.

---

Now what if you have a matrix $M = S D S^{-1}$ where $D$ is still diagonal (as above) and where $S$ is invertible? If $s_1, \ldots, s_n$ are the columns of $S$ (note that these form a basis because $S$ is invertible), then you can show that $$Ms_j = \lambda_j s_j,$$ that is, the $s_j$ are the eigenvectors of $M$. Geometrically, $M$ also stretches/shrinks along certain directions according to the values of the $\lambda_j$, but along the directions defined by $s_j$ instead of the standard basis directions.

Answer 2

The eigenvalue equation is given by

$$Av_i=\lambda_i v_i \quad \text{ for all } i=1,\ldots,n.$$

If you combine all equations for all eigenvalues you can rewrite the previous equations in a single matrix equation as

$$A[v_1,\ldots,v_n]=[v_1,\ldots,v_n]\text{diag}[\lambda_1,\ldots,\lambda_n].$$

For simplicity assume that we have $n$ distinct eigenvalue and eigenvector pairs. If we introduce $V=[v_1,\ldots,v_n]$ and $\Lambda = \text{diag}[\lambda_1,\ldots,\lambda_n]$ solve for $\Lambda$ to obtain

$$\Lambda = V^{-1}AV.$$

Hence, by multiplying in this fashion with the eigenvector matrices we obtain a diagonal matrix with the eigenvalues on the diagonal.

You can think of this as a change of basis for the system. We first used an arbitrary coordinate system. After transforming the system we are using a system that has a coordinate axis aligned with the eigenvectors.