Geometric Interpretation of Eigendecomposition

Question

If we have an orthogonal matrix $U$, then $U^Tx$ is essentially a rotation of the vector $x$. If we have a diagonal matrix $\Lambda$, then $\Lambda x$ is scaling the vector $x$ in each direction by the corresponding diagonal value.

Since any symmetric matrix $S=U\Lambda U^T$, $Sx=U\Lambda U^Tx$ which is rotation of $x$ by some angle $\vartheta$, scaling it by $\Lambda$ and then re-rotating by angle $-\vartheta$. Does this imply that multiplying by any symmetric matrix $S$ is just a scaling by its eigenvalues since the net rotation is $0$?

Answer 1

Yes, but the orthogonal matrix $U$ determines in which directions this scaling happens. Only applying the diagonal matrix

$$\Lambda=\begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$$

scales by two on the $x$-axis and by three on the $y$-axis.

But you can scale along other (orthogonal) axes if you want to by adding a rotation matrix.

You can imagine this by first transforming the coordinate system by $U^T$, so that the desired stretching axes line up with the coordinate axes (red and blue lines in the image), then you scale along the usual coordinate axes by $\Lambda$, and then you rotate back by $U$.

Answer 2

General case for a diagonizable matrix

$$S = U.\Lambda.U^{-1} = \begin{pmatrix} 3 & 1 \\ 0 & 2 \end{pmatrix} =\begin{pmatrix} 1 & -1 / \sqrt 2 \\ 0 & 1 / \sqrt 2 \end{pmatrix} \begin{pmatrix} 3 & 0 \\ 0 & 2 \end{pmatrix} \begin{pmatrix} 1 & 1 \\ 0 & \sqrt 2 \end{pmatrix}$$

Multiplying by $S$ is equivalent to multiplying successively by $U^{-1}$, $\Lambda$ and $U$.

• Matrix $U$ is made of columns which are the eigenvectors of $S$. Multiplying by $U^{-1}$ aligns the eigenvectors with the basis vectors.

• Scaling by $\Lambda$ preserves the orientation of the eigenvectors, as they are aligned with the basis.

• The scaled, but with unchanged orientation, eigenvectors are returned to their initial orientation by multiplying by $U$.

The eigenvectors: $$v1=\begin{pmatrix} 1 \\ 0 \end{pmatrix} \text { and } v2=\begin{pmatrix} -1 / \sqrt 2 \\ 1 / \sqrt 2 \end{pmatrix}$$ are shown in color.

Specific case of a symmetric matrix

In the case of a real symmetric matrix $S$, $U$ is orthogonal. $U^{-1}=U^T$ and the relationship can be written:

$$S = U.\Lambda.U^T$$

$U$ is a unitary matrix, in the general case, a rotation matrix.

$$ S = \begin{pmatrix} 3 & \sqrt 2 \\ \sqrt 2 & 2 \end{pmatrix} =\begin{pmatrix} \sqrt {2/3} & -1 / \sqrt 3 \\ 1 / \sqrt 3 & \sqrt {2/3} \end{pmatrix} \begin{pmatrix} 4 & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} \sqrt {2/3} & 1 / \sqrt 3 \\ -1 / \sqrt 3 & \sqrt {2/3} \end{pmatrix} $$

Does this imply that multiplying by any symmetric matrix $S$ is just a scaling by its eigenvalues.

Yes, but as shown, the scaling must be applied in the basis defined by the eigenvectors, or equivalently the basis can be changed prior to apply it. In the case of a real symmetric matrix, the eigenvectors are orthogonal.