Is there a standard method for image rectification by vanish points?

915 Views Asked by At

My problem is to do a rectification of an image when I know two vanish points $v1$ and $v2$ in homogeneous coordinates. One of them is for vertical lines and another is for horizontal lines. Is there a standard method to do this?

For example $v1 = (161283179, -310809790, 405443)$ - vertical vanish point and $v2 = (213625800, 24111240, 29076)$ - horizontal vanish point.

1

There are 1 best solutions below

1
On

I assume you want to perform metric rectification (rectify the image to the point where you can recover metric properties such as angles and ratios of lengths) as there's no way you can get a full Euclidean reconstruction with just images.

The operation that performs such a rectification is called a homography as a commenter has helpfully pointed out (although all homographies are not affine transforms, but an affine transform is a special type of homography). This homography is represented as a $3\times3$ matrix for the 2D case and has 8 degrees of freedom (as it is only defined up to scale). We only need to recover 4 of these to be able to rectify the image up to a similarity (which already imposes 4 constraints on the homography).

Section 2.7 of the book Multiple View Geometry in Computer Vision, provides two useful techniques to recover this matrix.

  1. The first is a two step approach also called stratification. It first affinely rectifies the image and then performs metric rectification. Affine rectification is performed by mapping the vanishing line (cross product of any two distinct vanishing points) to its canonical position $(0, 0, 1)$ in the affine space. This recovers 2 degrees of freedom. If $(l_1, l_2, l_3)$ are the coordinates of the vanishing line then the homography that performs the affine rectification will be $$H_1 = \begin{pmatrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ l_1 & l_2 & l_3 \end{pmatrix}$$ Metric rectification is a bit more involved. It is performed by using two distinct pairs of lines (4 lines in total), that are supposed to be orthogonal in the rectified image. We find the image of the dual conic (or equivalently the images of the circular points at infinity) and then retrieve the affine transformation that maps it to its canonical form. This further recovers 2 degrees of freedom and rectifies the images so that metric properties can be recovered.
    For each pair of lines $(l'_1, l'_2, l'_3)$ and $(m'_1, m'_2, m'_3)$ that are supposed to be orthogonal after rectification, we get the following constraint. $$(l'_1m'_1, l'_1m'_2 + l'_2m'_1, l'_2m'_2)(s_{11}, s_{12}, s_{13})^T = 0$$ Where $s_{11}, s_{12}, s_{13}$ are elements of the matrix that represents the image of the dual conic $$C^{*}_{\infty}{'} = \begin{pmatrix}s_{11} & s_{12} & 0 \\ s_{12} & s_{13} & 0 \\ 0 & 0 & 1\end{pmatrix}$$ Now, we can find the affine transformation that maps the image of the dual conic to its canonical position in metric space, $\begin{pmatrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0\end{pmatrix}$, using SVD as $$C^{*}_{\infty}{'} = H_2\begin{pmatrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0\end{pmatrix}H_2^T$$ The final homography that will rectify the image is $H = H_2 * H_1$.

  2. The second approach has a single step. We need at least 5 distinct pairs of lines (10 lines in total) that are supposed to be orthogonal in the rectified image. It is hard to find so many distinct pairs in day-to-day images and hence the stratified approach is usually favored. These pairs of lines are used to find the image of the absolute conic using constraints of the form $$(l'_1m'_1, (l'_1m'_2 + l'_2m'_1)/2, l'_2m'_2, (l'_1m'_3 + l'_3m'_1)/2, (l'_2m'_3 + l'_3m'_2)/2, l'_3m'_3)(a, b, c, d, e, f)^T = 0$$ where $C^{*}_{\infty}{'} = \begin{pmatrix}a & b/2 & d/2 \\ b/2 & c & e/2 \\ d/2 & e/2 & f\end{pmatrix}$.
    To find the mapping that transforms the image of the dual conic to its canonical position, we can rewrite $C^{*}_{\infty}{'}$ as $$\begin{pmatrix} KK^T & KK^Tv \\ v^TKK^T & v^TKK^Tv \end{pmatrix}$$ Next, using matrix decompositions, we can find the values of $K$ and $v$ that correspond to the values of the conic parameters $(a, b, c, d, e, f)$. This gives us the final homography $$H = \begin{pmatrix}K & 0 \\ v^t & 1\end{pmatrix}$$

If you are not familiar with all of the concepts mentioned in the answer, I strongly recommend going through the second chapter of the Multiple View Geometry book linked above. I would also recommend reading up on the basics of projective geometry for a more solid understanding.