Transformation matrix from quadrilateral to rectangle

Question

There exists a rectangle somewhere in space with some orientation. A camera from the coordinate center point is looking along the z axis and is seeing the rectangle as a quadrilateral (due to perspective transformation). How do I get the position and rotation of the rectangle given the points of the quadrilateral?

Answer 1

You can't recover the original rectangle from information of one perspective view alone. What the camera sees is the projection of the rectangle in the $XY$ plane. The projection does not contain depth information.

Answer 2

I think this is part of camera calibration problem. So, if we have full control of the camera, the easiest way would be to compute the camera matrix and find the 3D points corresponding to the image points, we will not have full information on the depth coordinates, but, I guess, normalizing the depth of one of the points, to say 1, the rotation of the rectangle should be obtainable. But position is not obtainable - you can get a line for each vertex.

Search terms for more information: camera matrix, pin hole camera model.