I’ve got a picture of a rectangular object, and I’m trying to figure out the relative lengths of its sides. If I trace out the sides of the object, I get two line segments about 70 degrees apart. I know that the photograph is a result of a projection of the object onto the camera sensor, so there should be some unique reversed projection where the angle is 90, which would give me an overhead view, from which calculating relative side lengths is trivial.
Now I DON’T want a complete answer. I rather like the linear algebra involved, and I’d like to solve it myself, but I’m having trouble getting a foothold in the idea of calculating the angle between these line segments at arbitrary projections. Also, when I’m done with this, I’d love to generalize it to rectangular prisms and even higher-dimensional equivalents for fun.
Thanks in advance for your hints
You don’t have enough information.
Let’s take a fairly simple example. Under the canonical camera matrix $\mathtt P=\left[\mathtt I\mid \mathbf 0\right]$, the image points $\mathbf x_0=(1,1)$, $\mathbf x_1=(1,0)$ and $\mathbf x_2=(0,2)$ back-project to the rays $\mathbf X_0(a)=a(1,1,1)$, $\mathbf X_1(b)=b(1,0,1)$ and $\mathbf X_2(c)=c(0,2,1)$, respectively. The condition that $\angle{\mathbf X_1\mathbf X_0\mathbf X_2}$ must be a right angle can be expressed as $$\left(\mathbf X_1(b)-\mathbf X_0(a)\right)\cdot\left(\mathbf X_2(c)-\mathbf X_0(a)\right) = 3a^2-2ab-3ac+bc=0.\tag1$$
Equation (1) is a simple quadratic in $a$. Solving it yields $$a = \frac16\left(2b+3c\pm\sqrt{4b^2+9c^2}\right).$$ If both $b\gt0$ and $c\gt0$, then both of these solutions are also positive. This means that for every line segment joining rays $\mathbf X_1$ and $\mathbf X_2$, there are two points on the ray $\mathbf X_0$ such that $\triangle{\mathbf X_1\mathbf X_0\mathbf X_2}$ is a right triangle. That is, there’s an infinite number of rectangles in front of the camera that are preimages of the (partial) rectangle in the image. Moreover, the ratio $\lVert\mathbf X_1(b)-\mathbf X_0(a)\rVert : \lVert\mathbf X_2(c)-\mathbf X_0(a)\rVert$ is not a constant, so you can’t even recover the rectangle’s aspect ratio from the two line segments in the image without more information.