How to calculate the projective transformation matrix given two planes?

Question

When projecting an image from one plane $\left\{ A \right\}$ to another plane $\left\{ B \right\}$, we have
$X' = \mathbf{H} \cdot X$
Namely,
$\begin{bmatrix}x'\\ y'\\ 1\end{bmatrix} = \begin{bmatrix} h_{11} & h_{12} & h_{13}\\ h_{21} & h_{22} & h_{23}\\ h_{31} & h_{32} & h_{33}\\ \end{bmatrix}. \begin{bmatrix}x\\ y\\ 1\end{bmatrix}$
Now, if we know the equation/relationship between the two planes,
e.g. $^B_A\mathbf{T} = \begin{bmatrix} \mathbf{n} & \mathbf{o} & \mathbf{a} & \mathbf{p} \\ 0 &0 &0 & 1 \end{bmatrix}$,
where, $\mathbf{p}$ is the translate vector between from {A} origin to {B} origin. How to calculate the transformation matrix $\mathbf{H}$? Especially, for the parameters $h_{31}$ and $h_{32}$.
In affine transformation $h_{31} = 0, h_{32} = 0$. However, in projective transformation $h_{31} = ~?,~ h_{32} = ~?$.

Answer 1

With the transformation equation you can calculate four related $(x_i,y_i) \sim (u_i,v_i)$ points in both planes and with this it's easy to calculate using

$$Ah=0$$

Where

$$A=\left(\begin{matrix} -x_1 & -y_1 & -1 & 0 & 0 & 0 & x_1u_1 & y_1u_1 & u_1 \\ 0 & 0 & 0 & -x_1 & -y_1 & -1 & x_1v_1 & y_1v_1 & v_1 \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ -x_4 & -y_4 & -1 & 0 & 0 & 0 & x_4u_4 & y_4u_4 & u_4 \\ 0 & 0 & 0 & -x_4 & -y_4 & -1 & x_4v_4 & y_4v_4 & v_4 \end{matrix}\right)$$

and

$$h =\left(\begin{matrix} h_{11} & h_{12} & h_{13} & h_{21} & h_{22} & h_{23} & h_{31} & h_{32} \end{matrix}\right) $$