How to derive the formula for the $R^2 \rightarrow R^2$ transform of a sheet of paper lying on the ground and above it is a camera, which may be at an angle.
A more sophisticated example: a sheet of paper is placed on a paraboloid surface or cylinder. Best if is possible derive parametrized formulas in both directions (obvious parameters do not have to correspond to each other but will be searched independently).
I add example such photo.
I found:
We'll denote the 3D world coordinates as (X, Y, Z) and the corresponding 2D image coordinates as (u, v).
- World to Camera Coordinates Transformation: Let's define a transformation matrix that takes a 3D point in world coordinates to camera coordinates. This matrix is typically composed of translation, rotation, and scaling components.
$\begin{bmatrix} X_\text{camera} \\ Y_\text{camera} \\ Z_\text{camera} \\ 1 \end{bmatrix} = \begin{bmatrix} R_{11} && R_{12} && R_{13} && t_x \\ R_{21} && R_{22} && R_{23} && t_y \\ R_{31}&&R_{32}&&R_{33}&&t_z\\0 && 0 && 0 && 1 \end{bmatrix} \begin{bmatrix} X_\text{world} \\ Y_\text{world} \\ Z_\text{world} \\ 1 \end{bmatrix} $
Here, R is the rotation matrix, and t is the translation vector. This transformation brings a point from the world coordinate system to the camera coordinate system. Also see Rotation_matrix in Wikipedia
- Perspective Projection: The perspective projection matrix projects a point in camera coordinates onto the image plane. This can be represented as:
$\begin{bmatrix} u \\v\\w \end{bmatrix} = \begin{bmatrix} f_x && 0 && 0 && c_x \\ 0 && f_y && 0 && c_y \\ 0 && 0 && 1 && 0 \end{bmatrix} \begin{bmatrix} X_\text{camera} \\ Y_\text{camera} \\ Z_\text{camera} \\ 1 \end{bmatrix}$
Here, $(f_x, f_y)$ are the focal lengths, and (c_x, c_y) are the coordinates of the principal point. w is extra coordinate to simplify representation
- Combining Transformations: Combining the world-to-camera transformation with the perspective projection, we get the full projection matrix (P):
$\begin{bmatrix} u \\ v \\ w \end{bmatrix} = \begin{bmatrix} f_x && 0 && 0 && c_x \\ 0 && f_y && 0 && c_y \\ 0 && 0 && 1 && 0 \end{bmatrix} \begin{bmatrix} R_{11} && R_{12} && R_{13} && t_x \\ R_{21} && R_{22} && R_{23} && t_y \\ R_{31}&&R_{32}&&R_{33}&&t_z\\0 && 0 && 0 && 1 \end{bmatrix} \begin{bmatrix} X_\text{world} \\ Y_\text{world} \\ Z_\text{world} \\ 1 \end{bmatrix} $
which is
$\begin{bmatrix} u \\ v \\ w \end{bmatrix} = \begin{bmatrix} f_x R_{11} && f_x R_{12} && f_x R_{13} && f_x t_x + c_x \\ f_y R_{21} && f_y R_{22} && f_y R_{23} && f_y t_y + c_y \\ R_{31} && R_{32} && R_{33} && t_z \end{bmatrix} \begin{bmatrix} X_\text{world} \\ Y_\text{world} \\ Z_\text{world} \\ 1 \end{bmatrix} $