Prove there is an orthogonal matrix

Question

Exercise 6.4.15 in Shifrin and Adams' Linear Algebra: a Geometric Approach says

Suppose $A$ and $B$ are symmetric and $AB=BA$. Prove there is an orthogonal matrix $Q$ so that both $Q^{-1}AQ$ and $Q^{-1}BQ$ are diagonal. (Hint: Let $\lambda$ be an eigenvalues of $A$. Use the Spectral Theorem to show that there is an orthonormal basis for E$(\lambda)$ consisting of eigenvectors of $B$.)

We know that if those two products of matrices were diagonal then $Q$ would be made of eigenvectors. Following the first part of the hint we know that $A$x$=\lambda$x. Can someone please solve this?

Answer 1

The spectral theorem in the textbook states that

Theorem. (Spectral Theorem). Let $A$ be a symmetric $n\times n$ matrix. Then

1. The eigenvalues of $A$ are real.

2. There is an orthonormal basis $\{{\bf q}_1,{\bf q}_2,\ldots,{\bf q}_n\}$ for $\mathbb{R}^n$ consisting of eigenvectors of $A$. That is, there is an orthogonal matrix $Q$ so that $Q^{-1}AQ=\Lambda$ is diagonal.

Now we start proving the hint of the problem. Let $A$ and $B$ be $n\times n$ matrices, $\lambda$ be an eigenvalue of $A$, and ${\bf E}(\lambda)$ be the $\lambda$-eigenspace of $A$. Let $\beta=\{{\bf q}_1,{\bf q}_2,\ldots,{\bf q}_k\}\subseteq\mathbb{R}^n$ be an orthonormal basis for ${\bf E}(\lambda)$. Since $AB=BA$, given ${\bf x}\in{\bf E}(\lambda)$, we have $$AB{\bf x}=BA{\bf x}=B\lambda{\bf x}=\lambda B{\bf x}.$$ It follows that $B{\bf x}\in{\bf E}(\lambda)$, that is, $B({\bf E}(\lambda))\subseteq{\bf E}(\lambda)$ and we have a linear transformation $\mu_B:{\bf E}(\lambda)\rightarrow{\bf E}(\lambda)$ defined by $\mu_B({\bf x})=B{\bf x}$ for all ${\bf x}\in{\bf E}(\lambda)$. Also, since $$B{\bf x}\cdot{\bf y} ={\bf x}^\top B^\top{\bf y} ={\bf x}^\top B{\bf y} ={\bf x}\cdot B{\bf y},\quad\forall {\bf x},{\bf y}\in{\bf E}(\lambda).$$ By observing the following equations \begin{align} \mu_B({\bf q}_j)=B{\bf q}_j=\sum_{i=1}^k(B{\bf q}_j\cdot{\bf q}_i){\bf q}_i,\quad\forall j=1,2,\ldots,k, \end{align} if we let $B'$ be the $k\times k$ matrix for $\mu_B$ w.r.t. the basis $\beta$, that is, $$B'=\begin{bmatrix} B{\bf q}_1\cdot{\bf q}_1& B{\bf q}_2\cdot{\bf q}_1& \cdots& B{\bf q}_k\cdot{\bf q}_1\\ B{\bf q}_1\cdot{\bf q}_2& B{\bf q}_2\cdot{\bf q}_2& \cdots& B{\bf q}_k\cdot{\bf q}_2\\ \vdots&\vdots&\ddots&\vdots\\ B{\bf q}_1\cdot{\bf q}_k& B{\bf q}_2\cdot{\bf q}_k& \cdots& B{\bf q}_k\cdot{\bf q}_k\\ \end{bmatrix}.$$ Then $B'$ is symmetric, and by the spectral theorem, there exists an orthonormal basis $\{{\bf q}'_1,{\bf q}'_2,\ldots,{\bf q}'_k\}$ for $\mathbb{R}^k$ consisting of eigenvectors of $B'$ corresponding to eigenvalues $\mu_1,\mu_2,\ldots,\mu_k$. Define $\gamma=\{{\bf q}''_1,{\bf q}''_2,\ldots,{\bf q}''_k\}\subseteq\mathbb{R}^n$ by $${\bf q}''_j=\sum_{i=1}^k({\bf q}'_j)_{i}{\bf q}_i,\quad\forall j=1,2,\ldots,k,$$ where $({\bf q}'_j)_i$ denote the $i$th entry of ${\bf q}'_j$. Then $\gamma$ is clearly an orthonormal basis for ${\bf E}(\lambda)$. Moreover, observe that \begin{align} B{\bf q}''_j\cdot{\bf q}_l =\sum_{i=1}^k({\bf q}'_j)_{i}(B{\bf q}_i\cdot{\bf q}_l) =(B'{\bf q}'_j)_l =(\mu_j{\bf q}'_j)_l =\mu_j({\bf q}'_j)_l, \end{align} which implies \begin{align} B{\bf q}''_j\cdot{\bf q}''_l &=\sum_{i=1}^k({\bf q}'_j)_{i}(B{\bf q}_i\cdot{\bf q}_l'') =\sum_{i=1}^k({\bf q}'_j)_{i}(B{\bf q}''_l\cdot{\bf q}_i)\\ &=\mu_l\sum_{i=1}^k({\bf q}'_j)_{i}({\bf q}'_l)_i =\mu_l{\bf q}'_j\cdot{\bf q}'_l =\left\{\begin{array}{ll} \mu_l&\mbox{if }j=l;\\0&\mbox{if }j\ne l. \end{array}\right. \end{align} Therefore $B{\bf q}''_j =\displaystyle\sum_{i=1}^k(B{\bf q}''_j\cdot{\bf q}''_i){\bf q}''_i =\mu_j{\bf q}''_j$, that is, $\gamma$ consists of eigenvectors of $B$, and we complete the proof of the hint. The rest of the problem will follow immediately.