Find 3D rotation vector and angle to transform a rectangle into a given quadrilateral

Question

I have a given rectangle that I need to transform into a given quadrilateral shape that resulted from a rotation and translation in 3D space, and subsequent projection.

*---------*        *---------*
|         |  -->   \         /
|         |         \        / 
*---------*          *------*

I only have the coordinates of the projected rectangle (i.e. the four coordinates of the quadrilateral's corners). I need to get back to the 3D rotation and translation that resulted in that projected shape.

Is there a simple mathematical formula that would enable me to compute the values of the rotation vector, rotation angle and translation vector?

Thanks.

Answer 1

In this answer, it is described how to construct a perspective transform $M$ that maps the any quadrilateral to any other quadrilateral.

Once $M$ is computed, it can be decomposed into a standard $2\times2$ transform $$ \begin{bmatrix}a&b&0\\c&d&0\\0&0&1\end{bmatrix} $$ a translation $$ \begin{bmatrix}1&0&0\\0&1&0\\h&k&1\end{bmatrix} $$ and rotations on the $x$ and $y$ axes $$ \begin{bmatrix}1&0&0\\0&\cos(\theta)&\sin(\theta)\\0&0&1\end{bmatrix} \quad\text{and}\quad \begin{bmatrix}\cos(\phi)&0&\sin(\phi)\\0&1&0\\0&0&1\end{bmatrix} $$ to give the $3$D motions equivalent to $M$.