invariant subspace under some conditions

Question

This came out at my linear algebra exam and I was not able to solve it. Let $f\colon \mathbb{R^{3}} \rightarrow \mathbb{R^{3}}$ be a linear transformation such that $\langle f(u),f(v) \rangle = \langle u,v \rangle$ (sorry for the poor LaTeX), for all $u,v \in \mathbb{R^{3}}$.

1. Prove that there exists a subspace $W$ of $\mathbb{R^{3}}$ with dimension 2 such that $f(W) \subseteq W$.
2. Prove that $f$ is diagonalisable iff $f_{|W}$ is (restrition of $f$ to $W$).
3. Prove that f is not diagonalisable iff $W$ in the conditions of 1. is unique.

---

My progress so far:

I know that the dot product product condition implies that the matrix of $f$ is orthogonal.

1. This is wrong somewhere, because I didn't use the dot product condition.

If $f$ is diagonalisable, let $w_1,w_2$ be lin. independent and we have $\text{span}\{w_1,w_2,f(w_1),f(w_2),\dots,f^n(w_1), f^n(w_2),\dots\}$ is the desired $W$, where $f^n(w)=f(f(\dots(w)))$, composed $n$ times. This is because $cf^{n-1}(w_1)=f^n(w_1)$ always has solution $c$, so the "only" lin. indepen. vectors are $w_1,w_2$ (sorry if I am not clear, if you don't understand this part, please ask).

Else, there will be a vector $w$ which is not an eigenvector, so $\text{span}\{w,f(w),\dots,f^n(w),\dots\}$ will be the desired $W$, if I am able to prove that this set is not $\mathbb{R}³$. If everything I done until now is correct, maybe this is where I apply the orthogonality condition. Any hints?

Answer 1

It is indeed true that, because of the dot product condition, the matrix of $f$ is orthogonal. This implicitly uses the dot product condition; in fact, the entries of the matrix $[f]^T[f]$ are $\langle f(e_i),f(e_j) \rangle$ where $\{e_1,\dots,e_n\}$ is the standard orthonormal basis.

The easiest way to do this problem is to use the properties of $f$ over $\Bbb C$. In particular, note that because $f$ is orthogonal and real, the following holds:

• $f$ is diagonalizable over $\Bbb C$ (by the spectral theorem)
• If $\lambda \in \Bbb C$ is an eigenvalue of $f$, then $|\lambda| = 1$
• If $\lambda \in \Bbb C \setminus \Bbb R$ is an eigenvalue of $f$, then $\overline{\lambda}$ is also an eigenvalue of $f$ (because $f$ is real).

Suppose that $f$ has an eigenvalue $\lambda \in \Bbb C \setminus \Bbb R$. Then $\ker(f^2 + |\lambda|^2I) = \ker(f^2 + I)$ is a $2$-dimensional invariant subspace of $\Bbb R^3$. Note that in this case, this is the only $2$-dimensional invariant subspace that we can build.

Otherwise, $f$ has real eigenvalues, and so $f$ is diagonalizable, and we can make a $2$-dimensional invariant subspace out of the span of two $1$-dimensional invariant subspaces.

---

Without using properties over $\Bbb C$:

Part 1:

because $f$ is a real matrix with size $2n+1 \times 2n+1$ (that is, of odd size), we may conclude that it has at least one real eigenvalue (and corresponding eigenvector). Let $v$ be this eigenvector.

Let $W = \text{span}(v)^\perp$. Prove, using the orthogonality of $f$, that $w \in W \implies f(w) \in W$.

Part 2:

Note that $f = f|_{W} \oplus f|_{W^\perp}$. The direct sum of transformations is diagonalizable iff each one is.

Part 3:

If the $W$ in 1 is unique, then we can't select any eigenvectors from $W$, so $f$ is not diagonalizable. If the $W$ in $1$ is not unique, then $f$ has at least two linearly independent eigenvectors, $u,v$.

Let $U = \text{span}(\{u,v\})$. Verify that $U^\perp$ must be a one dimensional invariant subspace. It follows that $f$ is diagonalizable.