Why does the spectral theorem give a canonical form for a particular equivalence class of matrices?

Question

The setting

I give the spectral theorem I have been working with:

Let $A$ be a real and symmetric matrix and $T: R^n \rightarrow R^n$ the symmetric linear application to which it corresponds (working with the standard basis).

Then there exists an orthonormal basis $N$ in which $T$ has associated a diagonal matrix. This is the same as saying that there exists an orthogonal matrix $P$ such that $D = P^{-1}AP$ is diagonal.

The problem

I am told that this diagonal matrix $D$ is a canonical form for the matrices that are unitarily similar to $A$. I want to show this.

My attempt

Basically I want to show that for any $B \in \{ B \in M_{n \times n}(R) | \exists \text{ a unitary matrix } U \text{ s.t. } A = U^*BU \}$ one can always recover the canonical form $D$ ? Is this correct? I am unsure on what I would have to show.

Answer 1

The statement as written is incorrect. You need to fix the order of diagonal entries in some way, for example by requiring that all the diagonal entries are in non-decreasing order. Otherwise $D_1=P^{-1}DP$ for any permutation matrix $P$ (such matrices are all unitary) is also diagonal.

"Canonical form" of something means more or less the following: you have some equivalence (in your problem you have two cases of this: case 1: on the set of real symmetric matrices $A\sim B$ if and only if exists orthogonal $O$ such that $A=O^{-1}BO$ - you can check that this is an equivalence relation; case 2: on the set of Hermitian matrices $A\sim B$ if and only if exists unitary $U$ such that $A=U^*BU$ - you can check that this is an equivalence relation;); you furthermore have a canonical element $D$ in each equivalence class, singled out by having some special form (in your case $D$ is a diagonal matrix with non-decreasing entries). (One should remark that the usual language for this is to say that the group of unitary matrices acts on the space of Hermitian matrices by conjugation, so that $U$ acts on $B$ by sending it to $U^*BU$; in such a case an orbit -- that is the set of all matrices of the form $U^*BU$ for a fixed $B$ and varying $U$ -- is always an equivalence class; the claim is then that each such equivalence class (i.e. for each $B$) contains unique diagonal matrix $D$ with non-decreasing entries.)

Now in the formulation "diagonal matrix $D$ is a canonical form for the matrices that are unitarily similar to $A$" you are already assuming that the class of matrices that are unitarily similar to $A$ has a diagonal matrix in it. And indeed, you know this to be true by spectral theorem ($O$ being orthogonal is in particular unitary, and $O^{-1}=O^T=O^*$, and $D=O^{-1}AO$). What remains to show is that if $D_1$ and $D_2$ are two diagonal matrices with non-decreasing entries that are unitarily equivalent then $D_1=D_2$. Well, if $D_2=U^*D_1U$ then the eigenvalues of $D_2$ and $D_1$ coincide (if $v$ is an eigenvector of $D_1$ with eigenvalue $\lambda$ then $U^*v$ is an eigenvector of $D_2$ with same eigenvalue $\lambda$; and conversely to eigenvectors $w$ of $D_2$ correspond eigenvectors $Uw$ of $D_1$). Thus the diagonal entries of $D_1$ and $D_2$ coincide; since they are arranged to be non-decreasing in both matrices, we conclude $D_1=D_2$.

Answer 2

First, recall that: $$ Det(A-\lambda I) =Det(P^{-1}DP-\lambda P^{-1}P)= Det(P^{-1})Det(D-\lambda I)Det(P)=Det(D-\lambda I) $$

So $A$ and $D$ have the same characteristic polynomial, and therefore the same eigenvalues. This is the situation I know to be described as "$A$ and $D$ are similar matrices". "Unitarily Similar" would require the matrix $P$ to be unitary ($P^{-1}=P^T$). The trick here, is that $A$ is symmetric.

Noting that:

$$ <v,Av>=<vA^T,v> $$ Because $A$ is symmetric, (thus self-adjoint). If $v_i$ and $v_j$ is a normalized eigenvector of $A$ with associated eigenvalues $\lambda_i$ and $\lambda_j$: $$ <v_i,Av_j>=v_i^T (A v_j)= (v_i^T A^T) v_j= \lambda_j v_i^T v_j= \lambda_i v_i^T v_j $$

Therefore: $$ (\lambda_j-\lambda_i )v_i^T v_j= 0 $$

With $\delta_{ij}$ the Kronecker delta. So either $A$ the eigenvectors of $A$ are ortogonal or the associated eigenvalue is the same. If any eigenvalue has multiplicity higher than 1, then it is possible to find linearly independent eigenvectors for it, and make this set of vectors ortogonal by a process like Gram-Schimidt.

This means that the matrix $P$ can be assembled by the orthonormal eigenvectors $v_i$:

$$ A [v_1,v_2, \cdots,v_n]= [v_1,v_2, \cdots,v_n] D $$ And because the vectors are orthonormal:

$$ P=[v_1,v_2, \cdots,v_n] \Rightarrow P^{-1}= P^T $$ Therefore:

$$ A P= P D \Rightarrow P^{-1}A P = D $$

Then, if $U$ is an unitary matrix, $(U^*AU)^* = U^*A^*(U^*)^*=U^*AU$ so also symmetrical (or hermitian). Therefore $PU$ diagonalizes $B$ into the same $D$.