We use Neural Network tracking to attempt to track a complex and chaotically changing edge among 6 camera perspectives. This chaos is what we want to capture, but we run into a few issues. The tracking points stay on the line very well, but tend to move around on the line. This means that we cannot use dlt or other reconstruction methods depending on the correlation of points, since "the third point from the right of the line" might have significantly moved along the line between one perspective and another, meaning we don't have much correlation of points between perspectives. There are a few points for which we do have correlation within the frame, but most of them are not part of the chaotic edge, they are another part of the body we want to track.
So what we have are six dense sets of points, each of which estimates their own projection of a 3D curve along their known axis. Are there techniques for constructing an estimation of the original 3D curve from this data. We can assume that the original 3D curve has a fixed length, but obviously the projections will have differing lengths.
EXAMPLE:
Let's say we have three perspectives which are orthogonal projections of the unknown curve. The xy-plane projection is a dense set of points which we have already determined to be best described as a line from the origin to $(1,1)$. The projection on the xz-plane is a set of points which approximates a semicircle, and the yz-plane approximates with a different set of points another semicircle with same radius $r$. From this we can intuitively conclude that the original curve was a roundish curve starting in the origin, having it's top be $(1/2, 1/2, r)$ and ending at $(1, 1, 0)$.
This was a trivially easy case, where I can intuit the answer. But I am looking for a rigerous, method for finding the 3D curve from multiple 2D projections without any correlation between the points on one projection and another projection.