Suppose $$ u^T ( (a-Au) \times (b-Bu) )=0 $$ $a,b \in \mathbb{R}^{3x1}$ $A,B\in \mathbb{R}^{3x3}$ are known constants and $u \in \mathbb{R}^{3x1}$ is the unknown.
Can I say anything on the number of solutions for $u$?
In particular I am interested in the case where $A$ and $B$ are in the form $A=I-c c^T $, $B=I-d d^T$, $c,d \in \mathbb{R}^{3x1}$ with $\Vert c \Vert = \Vert d \Vert =1$
When $A=I-cc^T$ and $B=I-dd^T$ are orthogonal projections, there are infinitely many solutions. Note that the equation can be written as \begin{align} 0&=u^T\left[(a-Au)\times(b-Bu)\right]\\ &=u^T\left[(a-u+cc^Tu)\times(b-u+dd^Tu)\right]\\ &=u^T\left[a\times b + (c^Tu)c\times b + (d^Tu)a\times d + (c^Tu)(d^Tu)c\times d\right]\\ &=(a\times b)^Tu + (c^Tu)(c\times b)^Tu + (d^Tu)(a\times d)^Tu + (c^Tu)(d^Tu)(c\times d)^Tu.\tag{1} \end{align} When $c$ and $d$ are linearly independent, $c,d$ and $e:=c\times d$ form a basis of $\mathbb R^3$. Therefore, if we put $x=c^Tu, y=d^Tu$ and $z=e^Tu$, we may further rewrite the equation in the form of \begin{align} 0&=(?c+?d+?e)^Tu + (c^Tu)(?c+?d+?e)^Tu + (d^Tu)(?c+?d+?e)^Tu + (c^Tu)(d^Tu)(e^Tu)\\ &=(?x+?y+?z) + x(?x+?y+?z) + y(?x+?y+?z) + xyz\\ &=(xy+?x+?y+?)z+q(x,y)\tag{2} \end{align} for some polynomial $q$. It follows that when $x$ and $y$ are sufficiently large and $(2)$ is viewed as a polynomial in $z$, the coefficient of $z$ is nonzero and the equation is solvable in $z$. Hence we may recover $u$: $$ u=\pmatrix{c^T\\ d^T\\ e^T}^{-1}\pmatrix{x\\ y\\ z}. $$ Since $x$ and $y$ are arbitrary (as long as the coefficient of $z$ in $(2)$ is nonzero), there are infinitely many solutions.
The case where $c$ and $d$ are linearly dependent (i.e. $c=\pm d$ because they are unit vectors) can be handled similarly, but there are quite a few corner cases to consider.