I have a problem where I wish to find multiple solutions to the equation:
$ \mathbf{a} = \mathbf{R} \mathbf{a}$
where I wish to calculate the solutions for $\mathbf{R}$ that is both symmetric ($\mathbf{R} = \mathbf{R}^T$) and special orthogonal ($\mathbf{R} \in SO(3)$) from a given $\mathbf{a}$ that is a real-valued 3-dimensional vector of non-zero magnitude.
For example, if I have the following vector:
$\mathbf{a} = \begin{bmatrix} 1.70349 && -1.67761 && 9.51419\end{bmatrix}^T$
Then two solutions are $\mathbf{R} = \mathbf{I}$ and
$\mathbf{R} = \begin{bmatrix} -0.939693 && -0.0593912 && 0.336824 \\ -0.0593912 && -0.941511 && -0.331707 \\ 0.336824 && -0.331707 && 0.881204 \end{bmatrix}$
This second solution can be generated from $\mathbf{R} = \mathbf{Q} \mathbf{R}_z \mathbf{Q}^T$ where $\mathbf{R}_z$ is a rotation about the z-axis by $\pi$ and
$\mathbf{Q} = \begin{bmatrix} 0.984808 && 0.0301537 && -0.17101 \\ 0 && 0.984808 && 0.173648 \\ 0.173648 && -0.17101 && 0.969846 \end{bmatrix}$
How would one compute the non-trivial solution? Does this approach generalize to an extended problem where the vectors are different, i.e. solving for the solution(s) for $\mathbf{R}$ in the equation $\mathbf{a} = \mathbf{R} \mathbf{b}$, given $\mathbf{a}$ and $\mathbf{b}$ ?
The answer can be found using a variation of Wahba's Problem.
Since R is symmetric, it implies that:
$$ \begin{align} a &= Rb \\ b &= Ra \end{align} $$
Following Ref [1], we may then construct a symmetric matrix:
$$ B = \frac{1}{2} \left(ab^T + ba^T \right) $$
And compute the SVD:
$$ B = U S V^T $$
Since $B$ is symmetric, $U = V$.
If $B$ is of rank 2 (i.e. $a \neq b$), then we may compute the solution as:
$$ R = U_+ V_+^T $$
where $U_+$ and $V_+$ are forced to be special orthogonal by:
$$ \begin{align} U_+ &= U \operatorname{diag} \left( \begin{bmatrix} 1 && 1 && \det(U)\end{bmatrix} \right) \\ V_+ &= V \operatorname{diag} \left( \begin{bmatrix} 1 && 1 && \det(V)\end{bmatrix} \right) \\ \end{align} $$
If $B$ is of rank 1 (i.e. $a=b$) then we may introduce a single axis rotation $W$ about the x-axis:
$$ R = U_+ W V_+^T $$
To keep R symmmetric, we have two choices of W:
$$ \begin{align} W &= I \\ W &= \operatorname{diag} \left( \begin{bmatrix} 1 && -1 && -1 \end{bmatrix} \right) \end{align} $$
Which corresponds as the two variants that we seek.
References: [1] Markley, F. L. Attitude Determination using Vector Observations and the Singular Value Decomposition, Journal of the Astronautical Sciences, 1988, 38:245-258