Camera Calibration - Calculate Rotation and Translation

Question

I have a calibration problem between two cameras, my setup contains two static cameras. Both cameras capture the same scene but from a different viewpoint.

I estimate the individual cameras poses using PnP (Perspective n Point) with the 6D position (rotation and translation (6 degress of freedom)) of objects in the image. However I need to calculate the RT (rotation and translation) between the two cameras. How can this be approached?

EDIT: if I use PnP, e.g. solvePnP from OpenCV It returns -> retval, rvec, tvec

so for Cam1, I would have the rotation and translation (R1, t1) for Cam2 as well (R2, t2)

so could I calculate it like this

R_relation = R1^-1 * R2

t_relation = R1^-1 * (t2-t1)

so: R2 = R1 * R_relation and t2 = R1 * t_relation + t1

I am pretty new in this field, so thanks in advance :)

Answer 1

In a recent post I've derived the image coordinates $p$ of a camera that has a general center position $d$ and orientation specified by a rotation matrix $R$ relative the world frame.

The result is that

$ p = \dfrac{f R^T (r_1 - d) } { (R k)^T (r_1 - d) } $

where $r_1$ is a point specified in the world coordinate system, and $k = [0, 0, 1]^T $

The image point is $p = [P_x, P_y, f ] $

Now, the columns of matrix $R$ are unit vectors that are mutually orthogonal, that is,

$R = [u_1, u_2, u_3] $

Only three parameters are needed to specify $R$, and an additional three parameters are needed to specify the location of the origin of the camera reference frame $d$, and in addition we can also assume that the focal distance f is unknown. This makes the total number of unknowns for each camera equal to $7$. The above equation can be broken down into two equations:

$P_x ( u_3^T (r_1 - d) ) = f u_1^T (r_1 - d) $

$P_y ( u_3^T (r_1 - d) ) = f u_2^T (r_1 - d) $

Hence for each $r_1$ we can write two scalar equations, and therefore, we need $4$ points to write sufficient number of equations to solve for all the unknowns.

To generate the matrix for our numerical solver, we take $u_3$ to have the form

$u_3 = [ \sin \theta \cos \phi, \sin \theta \sin \phi, \cos \theta ]^T $

Next step is to define two canonical vectors that are orthogonal to $u_3$ which I will call $w_1$ and $w_2$ and they are defined as follows

$w_1 = [\cos \theta \cos \phi, \cos \theta \sin \phi , -\sin \theta ]^T $

$w_2 = [-sin \phi, \cos \phi , 0 ]^T $

Then we can take $u_1$ and $u_2$ to be a rotation of $w_1$ and $w_2$ as follows

$ u_1 = \cos \psi w_1 + \sin \psi w_2 $

$u_2 = - \sin \psi w_1 + \cos \psi w_2 $

This completes the specification of matrix $R$ using the three parameters $\theta, \phi, \psi $.

Using a numerical method like Newton-Raphson multivariate method, we can solve for the 7 unknowns, which are $\theta, \phi, \psi, d_x, d_y, d_z$, and $f $. I have verified with an example and a computer program that this identification works.

This has to be repeated for the second camera, and we end up having two coordinate frames $(d_1, R_1)$ for the first camera and $(d_2, R_2)$ for the second camera. These two frames can be easily related as follows. Given a world point $r$ , then

$ r = R_1 p_1 + d_1 = R_2 p_2 + d_2 $

Hence,

$ p_2 = R_2^T (R_1 p_1 + d_1 - d_2 ) $

which simplifies to,

$p_2 = (R_2^T R_1) p_1 + R_2^T (d_1 - d_2) = R p_1 + d $

where $R = (R_2^T R_1) $ and $ d = R_2^T (d_1 - d_2) $