Find a rotation matrix with two constraints: Aligns two vectors AND a third vector is perpendicular to a fourth vector when transformed

Question

I have four vectors $ d,w,c,n \in \mathscr{R}^3$. I want to find a rotation matrix $R$ that satisfies these constraints: $w$ is aligned with $d$ after rotation AND $c$ is perpendicular to $n$ after rotation. You can imagine it in two steps: after a first rotation that aligns $w$ with $d$, make a second rotation around $d$ so that $c$ becomes perpendicular to $n$. In a particular problem I am working on $d=[1,1,1]^T$ and $n=[0,0,1]^T$.

I believe the constraints can be expressed with these two equations

$$ d \times Rw=0$$ $$ n^T Rc=0$$

In addition, we must impose a third condition for $R$ being a rotation matrix $$ RR^T=I$$

Is there a nice way to solve for R using matrix algebra? Or how to formulate the two steps above?

I tried to play with Kronecker products to form a vectorized version of $R$, but did not reach a solution. With $S=skew(d)$ being the skew vector for cross product: $$d \times Rw=SRw=0 \rightarrow (w^T \oplus S)vec(R)=0$$ $$n^T Rc=0 \rightarrow(c^T \oplus n^T)vec(R)=0$$ but could not impose the $RR^T=I$

Answer 1

I think this will not work unless the angle between $n$ and $w$ equals that between $n$ and $d$, which may not be true.

I think $R = R_1R_2$ works where:

1. $R_1$ is a rotation around $c\times n$ by an angle of $90^o + \cos^{-1} (\frac{c\cdot n}{|c||n|})$
2. $R_2$ is a rotation around $v\times u$ by an angle of $\cos^{-1} (\frac{u\cdot v}{|u||v|})$, where $v = (c' \times w) \times c'$ and $u > = (c' \times d) \times c'$, where $c' = R_1c$.

The first step maps $c$ to $c'$ which is normal to n. the second step aligns $w$ with $d$ by aligning $v$ with $u$. Intuitively, let $S$ be the sphere with $v\times u$ (which is parallel to $c'$) at the north pole, and let $P$ be the plane of its equator. $u$ is obtained by sliding $d$ along $d$'s longitude until it falls on $P$, and $v$ is similarly obtained from $w$.

Answer 2

Let the four vectors be made unit, and let $c$ and $w$ have angle $\phi$ between them. Since orthogonal matrices preserve angles, $Rc$ must lie on the cone with axis $d=Rw$ and angle $\phi$, after rotation.
On the other hand, the vectors perpendicular to $n$ form a plane. So $c$ needs to be mapped to the intersection of the cone with this plane. Whether that is possible depends on the angle between $n$ and $d$; call it $\eta$.

If $\eta<\frac{\pi}{2}-\phi$ then the intersection is $0$, which is inadmissible. There is no solution.

If $\eta=\frac{\pi}{2}-\phi$ then the intersection is a line, so there is only one point where $Rc$ can be mapped to.

If $\eta>\frac{\pi}{2}-\phi$ then the intersection is a pair of intersecting lines, so there are two points where $Rc$ can be mapped to. To find these points, we need to solve for $\tilde{c}=Rc$, $$\tilde{c}\cdot d=\cos\phi,\qquad \tilde{c}\cdot n=0,\qquad \tilde{c}\cdot \tilde{c}=1$$ Note that $\tilde{c}\times(n\times d)=(\cos\phi) n$. Solving for $$\tilde{c}=\alpha n+\beta d+\gamma n\times d$$ gives $$\alpha=-\cos\phi\cos\eta/\sin^2\eta,\\ \beta=\cos\phi/\sin^2\eta,\\\gamma=\pm\sqrt{1-\cos^2\phi/\sin^2\eta}$$

The problem is then to find a rotation that maps $w,c$ to $d,\tilde{c}$.
The rotation that maps $(1,0,0)$, $(\cos\phi,\sin\phi,0)$ to $d,\tilde{c}$ is $$R_1=[d,\frac{\tilde{c}-(\cos\phi)d}{\sin\phi},\frac{d\times\tilde{c}}{\sin\phi}]$$ and similarly, $$R_2=[w,\frac{c-(\cos\phi) w}{\sin\phi},\frac{w\times c}{\sin\phi}]$$ so the final rotation is $$R=R_1R_2^\top$$

Answer 3

I arrived at an alternative formulation which also correctly solved the problem in a few numerical trials. It uses Rodrigues' formula for the rotation matrix given a unit vector $\mathbf{v}$ along the desired axis of rotation, and the rotation angle $\theta$ : $$ R=I +\sin(\theta)\ \mathbf {K_{v}} +(1-\cos(\theta))\ \mathbf {K_{v}} ^{2} $$ where $\mathbf {K_{v}}$ is the skew symmetric matrix which represents a cross-product $a\times b=\mathbf{K_{a}}b$

We find the final rotation matrix as a combination of two rotation matrices $R_{1}$, $R_{2}$

Starting with unit vectors $ d,w,c,n $ we first obtain $R_{1}$ which rotates along the axis $w \times d$ by the angle $\theta_{1}$ between $w$ and $d$ using the Rodrigues' formula. Note that $$cos(\theta_{1})=w \cdot d \ , \ \vert sin(\theta_{1})\vert= \Vert w \times d \Vert$$ $$ R_{1}=I +\sin(\theta_{1})\ \mathbf {K_{\frac{w \times d}{\Vert w \times d \Vert}}} +(1-\cos(\theta_{1}))\ \mathbf {K^{2}_{\frac{w \times d}{\Vert w \times d \Vert}}} $$

In the second step we define $g=R_{1}c$ and seek a rotation along axis $d$ by angle $\theta_{2}$ that makes $g$ perpendicular to $n$. Because $R_{1}w=d$ this second rotation will not affect $w$ .

Writing the constraint as $n^TR_{2}g=0$ and substituting Rodrigues' formula for $R_{2}$ we find $$ A\sin(\theta_{2})\ +\ B\cos(\theta_{2})=C $$ $$ A=n^T \ \mathbf{K_{d}} \ g$$ $$ B=\ -n^T \ \mathbf{K_{d}^2} \ g$$ $$ C= \ -n^Tg \ -n^T \ \mathbf{K_{d}^2} \ g$$ I used the symbolic toolbox of Octave (which uses Sympy) to find the solution to this trigonometric equation as $$\theta_{2} = 2\ tan^{-1}(\frac{A \pm \sqrt{A^2+B^2-C^2}}{B+C}) $$ $$ R_{2}=I +\sin(\theta_{2})\ \mathbf {K_{d}} +(1-\cos(\theta_{2}))\ \mathbf {K_{d}} ^{2} $$

and the final solution is $ R = R_{2}R_{1}$.

Note that if $A^2+B^2-C^2 >0$ there are two solutions, if $A^2+B^2-C^2 =0$ there is one solution, and if $A^2+B^2-C^2 <0$ there is no (real) solution.