A problem of eigenvectors of 3×3 complex orthogonal matrix

Question

Let $v$,$w$ be two complex vectors in $C^3$ satisfying that $v^tv=w^tw=0$ and $v$,$w$ are linearly independent.

I want to know whether there always exists a 3×3 complex orthogonal matrix $g≠I_3,-I_3$ $g^tg=I_3$ such that both $v$ and $w$ are eigenvector of $g$？

Answer 1

Yes,under the conditions stated, $$ g=I_3-\frac{2}{w^tv}\big(wv^t+vw^t\big) $$ is such an "orthogonal" matrix. The stated conditions guarantee that $\ w^tv\ne0\ $. \begin{align} gv&=v-\frac{2}{w^tv}\big(v^tvw+w^tvv\big)\\ &=-v\ ,\ \text{ and}\\ gw&=w-\frac{2}{w^tv}\big(v^tww+w^twv\big)\\ &=-w\ , \end{align} so $\ v\ $ and $\ w\ $ are eigenvectors of $\ g\ $ corresponding to the eigenvalue $\ {-}1\ $.

Also, $\ g\ $ is symmetric, so \begin{align} g^tg&=g^2\\ &=I_3^2-\frac{4}{w^tv}\big(wv^t+vw^t\big)+\frac{4}{\big(w^tv\big)^2}\big(wv^t+vw^t\big)^2\\ &=I_3-\frac{4}{w^tv}\big(wv^t+vw^t\big)+\frac{4}{\big(w^tv\big)^2}\big(v^twwv^t+w^tvvw^t\big)\\ &=I_3-\frac{4}{w^tv}\big(wv^t+vw^t\big)+\frac{4w^tv}{\big(w^tv\big)^2}\big(wv^t+vw^t\big)\\ &=I_3\ . \end{align}

Answer 2

Yes, $g$ always exists. At the end of this answer we will obtain the general form of $g$.

By a result from Horn and Merino, the Jordan form of a complex orthogonal matrix must be a direct sum of matrices of the following types:

• $J_k(\lambda)\oplus J_k(\lambda^{-1})$ for any $k$ and for some $\lambda\ne0$,
• $J_k(1)$ for any odd $k$,
• $J_k(-1)$ for any odd $k$.

Since $g$ has two linearly independent eigenvectors $v$ and $w$, its Jordan form is not a single Jordan block. As $g\ne\pm I$ and $\det g=\pm1$, it follows from the above result that $g$ is diagonalisable and its Jordan form is either $\operatorname{diag}(\pm1,\pm1,\mp1)$ or $\operatorname{diag}(\lambda,\frac{1}{\lambda},\pm1)$. So, by considering $-g$ instead of $g$ if necessary, we may assume that its spectrum is $\{\lambda,\frac{1}{\lambda},1\}$ for some $\lambda\notin\{0,1\}$.

Let $\{v,w,x\}$ be an eigenbasis of $g$. Then $g=PDP^{-1}$ where $P=\pmatrix{v&w&x}$ and $D$ is a diagonal matrix. So, $g^tg=I$ if and only if $DP^tPD=P^tP$. Now there are three possibilities:

1. If $\lambda\ne-1$, then the condition $DP^tPD=P^tP$ implies that $x^tv=x^tw=0$.
2. If $\lambda=-1$ and $gx=x$, the condition $DP^tPD=P^tP$ also implies that $x^tv=x^tw=0$.
3. If $\lambda=-1$ but $gv=v,\,gx=x$ and $gw=-w$, then $D=\operatorname{diag}(-1,1,-1)$ and hence the condition $$ \pmatrix{0&-v^tw&v^tx\\ -w^tv&0&-w^tx\\ x^tv&-x^tw&x^tx} = DP^tPD=P^tP= \pmatrix{0&v^tw&v^tx\\ w^tv&0&w^tx\\ x^tv&x^tw&x^tx} $$ implies that $$ P^tP= \pmatrix{0&0&v^tx\\ 0&0&0\\ x^tv&0&x^tx}, $$ but this is impossible because $P^tP$ is nonsingular.

Hence we must have $x^tv=x^tw=0$. Note that the vector $x$ always exists: as $v$ and $w$ are linearly independent, $X=\{x:x^tv=x^tw=0\}$ is one-dimensional. Hence $v^tw$ is necessarily nonzero (or else both $v$ and $w$ will lie inside $X$) and if $x$ spans $X$, then $\{v,w,x\}$ must be linearly independent. Moreover, as $\pmatrix{v&w&x}$ is nonsingular, $x^t\pmatrix{v&w&x}= \pmatrix{0&0&x^tx} $ is nonzero. Hence $x^tx\ne0$.

We may also give an explicit expression of $x$. Since $u\cdot(v\times w)=\det\pmatrix{u&v&w}$ for every vector $u$ (where the dot and cross products are calculated using the same formulae as in the real case), $v\times w$ must be a nonzero scalar multiple $x$ (otherwise we may put $u=x$ to arrive at a contradiction). Therefore $x=v\times w$ (up to scaling).

Anyway, from our previous discussion, we conclude that the general solution is given by: $$ g=\pm P\pmatrix{\lambda\\ &\frac{1}{\lambda}\\ &&1}P^{-1} $$ where $\lambda\ne0,1$. The answer by lonza leggiera is the case where $g=P\operatorname{diag}(-1,-1,1)P^{-1}$.