Let's assume that we have an array of $n$ 3D points, we don't know their coordinates (thus we have $3n$ indeterminate scalar values). We also have $m$ 2D projections with known coordinates (thus we have $2nm$ determinate values). Both 2D and 3D points are labelled, i.e. we know which 2D point in a certain projection corresponds to which 3D point. Projection matrices for each of $m$ projections are unknown, but if I understand 3D-to-2D projection correctly, the projection in this case could be defined by 3 Euler angles + 1 FOV (Field-of-View) angle. FOV is the same across all projections, which yields us another $3m + 1$ indeterminate values. Both 3D points and 2D projections are normalized (to eliminate translation and scaling) to some range, for example, 0 to 1.
Overall tally so far:
- $3n + 3m + 1$ scalar indeterminates
- $2nm$ determinate values
- Indeterminates are bound to determinates with $nm$ projection matrix equations
So, my questions are:
- Is it solvable at all, i.e is it possible to get 3D coordinates (+projection parameters) that would match all given 2D projections with a given amount of tolerance $\varepsilon$ (I strongly suspect it is, and, basically, it's the task that our brain is doing constantly analyzing two 2D images from both eyes, visualizing 3D image in mind)?
- Under which conditions it is solvable? (I suspect it's something like $3n + 3m + 1 < 2nm$, but I'm not sure)
- What's the approximate calculation complexity and best way to do it (I'm really lost here - would any generic multidimensional root finding algorithm do, or are there any complications / better way?)