Understanding Spectral Theorem

Question

I got the proof from mentioned here.

Spectral Theorem: Let $T$ be a normal operator on a finite-dimensional complex inner product space $V$ or a self-adjoint operator on a finite-dimensional real inner product space $V$. Let $c_1, \dotsc, c_k$ be the distinct characteristic values of $T$. Let $W_j$ be the characteristic space associated with $c_j$ and $E_j$ the orthogonal projection of $V$ on $W_j$. Then $W_j$ is orthogonal to $W_i$ when $i \neq j$, $V$ is the direct sum of $W_1, \dotsc, W_k$, and $$ T = c_1 E_1 + \dotsb + c_k E_k$$

Proof: Let $\alpha$ be a vector in $W_j$, $\beta$ a vector in $W_i$, and suppose $i\neq j$. Then $c_j⟨\alpha,\beta⟩=⟨T\alpha,\beta⟩=⟨\alpha,T^*\beta⟩=⟨\alpha,\overline{c_j}\beta⟩$. Hence $(c_j-c_i)⟨\alpha,\beta⟩=0$, and since $c_j-c_i\neq 0$, it follows that $⟨\alpha,\beta⟩=0$. Thus $W_j$ is orthogonal to $W_i$ when when $i\neq j$. $\color{red}{\text{From the fact that } V \text{ has an orthonormal basis consisting of characteristic vectors, it}}$ $\color{red}{\text{follows that }V=W_1+\cdots+W_k}$. If $\alpha_j$ belongs to $V_j(1\leq j\leq k)$ and $\color{red}{\alpha_1+\cdots+\alpha_k\stackrel{?}{=}0}$, then \begin{align} 0&\color{red}{\stackrel{?}{=}}(\alpha_i,\sum_j\alpha_j)=\sum_j(\alpha_i,\alpha_j)\qquad(1)\\ &=\|\alpha_i\|^2 \end{align} For every $i$, so that $V$ is the direct sum of $W_1,\cdots,W_k$. Therefore $\color{red}{E_1+\cdots+E_k=I}$ and \begin{align} T&=TE_1+\cdots+TE_k\\ &=c_1E_1+\cdots+c_kE_k \end{align}

---

I am not a follower of Hoffman and Kunze's linear algebra but for seeking my proof of Spectral Theorem I read that part. And I didn't manage to understand the red marked line. Like:
$(1)$ Why $V$ space can be written as a sum of the $W_i$ space$?$
$(2)$ Why the sum of the vectors $\alpha_i$ is zero$?$
$(3)$ Why the equality hold for $(1)?$
$(4)$ Why the sum of eigenvectors $E_i$ is $I?$
Maybe all of my question require another reference theorem but I really want to know all of those Why.
Thanks for your time and thanks in advance.

Answer 1

(1) If you assume known that there is a basis of eigenvectors, then any vector in $V$ can be written as a linear combination of eigenvectors, and since any eigenvector belongs to some $W_i$, the result follows.

(2) This is an assumption made in order to prove that the sum is direct.

(3) This holds because it's the inner product of two vectors, the second of which is zero (see (2)).

(4) $E_i$ is not a vector, it's a projection onto the eigenspace $W_i$. Since the sum of eigenspaces is direct, if you take a basis for each $W_i$ then these bases make up a basis of $V$, the matrix of $E_i$ in this basis is going to be diagonal with $1$'s at the $W_j$ entries and $0$ elsewhere, and the sum of these matrices is the identity matrix.

Answer 2

Here's an example:

$$ P = \begin{pmatrix} 0 & \frac1{\sqrt2} & \frac1{\sqrt2} \\ 0 & \frac1{\sqrt2} & -\frac1{\sqrt2} \\ 1 & 0 & 0 \end{pmatrix};\; T = P \begin{pmatrix} -1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 2 \end{pmatrix} P^T = \begin{pmatrix} \frac12 & -\frac32 & 0 \\ -\frac32 & \frac12 & 0 \\ 0 & 0 & -1 \end{pmatrix}. $$

There are two eigenvalues: $c_1 = -1$ and $c_2 = 2$. The orthogonal projections are

\begin{align} E_1 &= P \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{pmatrix}P^T = \begin{pmatrix} \frac12 & \frac12 & 0 \\ \frac12 & \frac12 & 0 \\ 0 & 0 & 1 \end{pmatrix}, \\ E_2 &= P \begin{pmatrix} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix}P^T = \begin{pmatrix} \frac12 & -\frac12 & 0 \\ -\frac12 & \frac12 & 0 \\ 0 & 0 & 0 \end{pmatrix}. \end{align}

This works in general. To obtain $E_i$, you take $T = PDP^T$ and replace all the eigenvalues in $D$ by $1$ if they equal $c_i$ and $0$ if they are a different eigenvalue.

You should convince yourself that

1. $$ E_1 + E_2 + \dots + E_k = P \begin{pmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end{pmatrix} P^T = I. $$

2. $$ c_1 E_1 + c_2 E_2 + \dots + c_k E_k = PDP^T = T. $$

3. $E_i$ projects onto the $c_i$-eigenspace. For a start: if you multiply $P$ times the modified diagonal matrix, the remaining columns of $P$ are a basis for the $c_i$-eigenspace, in particular the column space (image) of $E_i$ is contained in the $c_i$-eigenspace.

Comments:

1. This is all easy to see if you just don't write the $P$'s and $P^T$'s and just write the matrices in terms of the eigenbasis rather than the standard basis.

2. This works for any diagonalizable matrix, if you replace $P^T$ by $P^{-1}$. I'm just using a self-adjoint (i.e. symmetric) operator because that's what the Spectral Theorem applies to. For self-adjoint matrices the special facts are: 1) they are always diagonalizable 2) we can diagonalize them with an orthogonal matrix.