Simultaneously Diagonalizing Bilinear Forms

Question

Let $\theta$ and $\psi$ be symmetric bilinear forms on a finite-dimensional real vector space $V$, and assume $\theta$ is positive definite. Show that there exists a basis $\{v_1,\ldots,v_n\}$ for $V$ and $\lambda_1,\ldots,\lambda_n\in\mathbb{R}$ such that $$\theta(v_i,v_j)=\delta_{i,j}\quad\text{and}\quad\psi(v_i,v_j)=\delta_{ij}\lambda_i$$ where $\delta_{ij}$ is the Kronecker delta function.

I think it's enough to choose a basis $\{w_1,\ldots,w_n\}$ for which the matrix representations of $\theta$ and $\psi$ are both diagonal. Then $\left\{\frac{w_1}{\sqrt{\theta(w_1,w_1)}},\ldots,\frac{w_n}{\sqrt{\theta(w_n,w_n)}}\right\}$ is the required basis.

Answer 1

The catch is finding a basis which simultaneously diagonalizes $\theta$ and $\psi$.

Take $V = \mathbb{R}^n$ for simplicity, and let $\theta (x,y) = \langle x, A y \rangle$, and $\psi (x,y) = \langle x, B y \rangle$, with $A,B \in \mathbb{R}^{n\times n}$. By assumption $A=A^T >0$, and $B= B^T$.

Since $A>0$, it has a Cholesky decomposition $A = Q^T Q$ (with Q invertible, of course). Now consider the matrix $Q^{-T} B Q^{-1}$. Since this is symmetric, it can be diagonalized by an orthogonal $U$ to give $U^T Q^{-T} B Q^{-1} U = \Lambda$, where $\Lambda$ is diagonal. Also notice that $U^T Q^{-T} A Q^{-1} U = I$ (since $Q^{-T} A Q^{-1} = I$).

Now choose a basis $v_i = Q^{-1} U e_i$, where $e_i$ is the standard basis for $\mathbb{R}^n$. Then we have $$ \theta(v_i, v_j) = \langle Q^{-1} U e_i , A Q^{-1} U e_j \rangle = \langle e_i , U^T Q^{-T} A Q^{-1} U e_j \rangle = \langle e_i , e_j \rangle, $$ and similarly $$ \psi(v_i, v_j) = \langle Q^{-1} U e_i , B Q^{-1} U e_j \rangle = \langle e_i , U^T Q^{-T} B Q^{-1} U e_j \rangle = \langle e_i , \Lambda e_j \rangle, $$ from which the desired result follows.

Answer 2

Your question essentially reduces the to spectral theorem for symmetric bilinear forms. Use $\theta$, the positive definite form, as an inner-product. This makes $(V,\theta)$ a (real) inner product space, and hence spectral theorem applied to $\psi$ will give you an answer.

---

For a sketch of the proof of the spectral theorem, what we can do is to look at the set of all vectors $S := \{ v\in V| \theta(v,v) = 1\}$. Note that by positive definiteness every vector $w\in V$ can be written as a multiple of some $s\in S$. In fact, $S$ is a topological sphere and is compact. So we can let $e_1$ be a vector in $S$ such that $\psi(e_1,e_1) = \inf_S \psi(s,s)$. Let $S_1 = S \cap \{e_1\}^\perp$ where $\perp$ is defined relative to $\theta$. We can define $e_2$ as a vector in $S_1$ such that $\psi(e_2,e_2) = \inf_{S_1} \psi(s,s)$ and so on. By induction we will have arrived at a collection of vectors which are orthonormal with respect to $\theta$. That they are also $\psi$-orthogonal follows by minimization: if there exists $s\in S_1$ such that $\psi(e_1,s) \neq 0$, we have that for $a^2 + b^2 = 1$ $$ \psi(a e_1 + b s, a e_1 + b s) = a^2 \psi(e_1,e_1) + b^2 \psi(s,s) + 2ab \psi(e_1,s) = \psi(e_1,e_1) + b^2 (\psi(s,s) - \psi(e_1,e_1) + 2ab \psi(e_1,s)$$ By choosing $|b| < 1/2$ sufficiently small such that $$ \left|\frac{1}{b}\right| > \left|\frac{\psi(s,s) - \psi(e_1,e_1)}{\psi(e_1,s)}\right| $$ and with the appropriate sign, we see we can make $$ \psi(a e_1 + bs, a e_1 + bs) < \psi(e_1,e_1) $$ contradicting the minimisation assumption. By induction the same can be said of all $e_i$, and hence they are mutually orthogonal relative to $\psi$.

---

It is important to note that the assumption that $\theta$ is positive definite is essential. In the proof above we used the fact that for a positive definite form, its corresponding "unit sphere" is a topological sphere and is a compact set in $V$. For an indefinite form or a degenerate form, the corresponding "sphere" would be non-compact (imagine some sort of hyperboloid or cylinder), and hence it can happen that the infimum of a continuous function on the surface is not achieved, breaking the argument.

In fact, given two symmetric bilinear forms without the assumption that at least one of them is positive definite, it is possible that they cannot be simultaneously diagonalised. An example: let $$ \theta = \begin{pmatrix} 1 & 0 \\ 0 & -1\end{pmatrix} \qquad \psi = \begin{pmatrix} 1 & -1 \\ -1 & -1\end{pmatrix}$$ Suppose you want $(x,y)$ and $(z,w)$ to simultaneously diagonalise the matrices. This requires in particular $$ xz = wy \qquad xz - wy - xw - zy = 0 $$ for the cross terms to vanish. Hence we have $$ xw + zy = 0 $$ Assuming $x \neq 0$ (at least one of $x,y$ is non zero), we solve by substitution $z = wy / x$ which implies $w(x^2 + y^2) = 0$. Since $x^2 + y^2 \neq 0$ if $(x,y)$ is not the zero vector, this means $w = 0$. But the equation $xz = wy = 0$ implies that $xz = 0$. By assumption this implies $z = 0$ and hence $(z,w)$ is the zero vector, which contradicts our assumption.

A similar proof can be used to show that $$ \theta = \begin{pmatrix} 0 & 1 \\ 1 & 0\end{pmatrix} \qquad \psi = \begin{pmatrix} 1 & 2 \\ 2 & 0 \end{pmatrix} $$ also cannot be simultaneously diagonalised.