I am looking for suggestions for solving the following geometrical problem algebraically.
I have three rays in the 3D space emanating from an origin $O$, say $\alpha$, $\beta$, $\gamma$. I wanted to find a plane $\pi$ intersecting these three lines in the points $A$, $B$, $C$ respectively, so that the triangle $ABC$ is right, say in $B$, and that its sides have known lengths $\overline{AB}$ and $\overline{BC}$. Obviously, $\overline{AC}^2$ = $\overline{AB}^2$ + $\overline{BC}^2$.
Basically, given three rays out of an origin and a right triangle, the plane $\pi$ draws a pyramid having those rays as the edges and that right triangle as the base. It does not matter whether the pyramid is acute or obtuse.
I also understand that the problem may have no solution in particular circumstances. When a solution exists, though, I would like to determine the position of the intersection points $A$, $B$ and $C$ between thosee rays $\alpha, \beta, \gamma$ and that plane $\pi$.
I cast the problem using these symbols: the pyramid edges are the vectors $\vec{a}=OA$, $\vec{b}=OB$, $\vec{c}=OC$; and their unknown lengths are $a = |\vec{a}|, b = |\vec{b}|, c = |\vec{c}|$.
I use the law of the cosines to express the fact that the faces of the pyramid are generic triangles:
$ a^2 - 2 \cos \hat{\alpha\beta}\, ab + b^2 = \overline{AB}^2$
$ b^2 - 2 \cos \hat{\beta\gamma}\, bc + c^2 = \overline{BC}^2$
$ c^2 - 2 \cos \hat{\gamma\alpha}\, ca + a^2 = \overline{CA}^2$
and the Pythagorean theorem to state that the base of the pyramid is a right triangle:
$\overline{AB}^2 + \overline{BC}^2 = \overline{AC}^2 $
I have three nonlinear equations in the three variables $a, b, c$ and one condition. I lack a strategy to solve this system and obtain closed-form formulas for the solution.
My questions are
As a sanity check, does the formulation express the problem well in the first place?
How can I then solve for $a,b,c$ in the system above?
The nonlinear system of equations can be solved successfully and easily with the Newton-Raphson multivariate method. The vector function is
$ f(a,b,c) = \begin{bmatrix} a^2 + b^2 - 2 a b \cos t_1 - X^2 \\ a^2 + c^2 - 2 a c \cos t_2 - Y^2 \\ b^2 + c^2 - 2 b c \cos t_3 - (X^2 + Y^2) \end{bmatrix}$
Where $t_1$ is the angle between $v_1$ and $v_2$ and $t_2$ is the angle between $v_1$ and $v_3$ and $t_3$ is the angle between $v_2$ and $v_3$. $X$ is the length of the first leg of the right triangle and $Y$ is the length of the second leg.
Newton-Raphson method uses the Jacobian matrix of the vector function $f$. The ${ij}$-th entry of this matrix is the partial derivative of $i$-th function with respect to the $j$-th variable. Thus the Jacobian is given by
$J(a,b,c) = \begin{bmatrix} 2 a - 2 b \cos t_1 && 2 b - 2 a \cos t_1 && 0 \\ 2a - 2 c \cos t_2 && 0 && 2 c - 2 a \cos t_2 \\ 0 && 2 b - 2 c \cos t_3 && 2 c - 2 b \cos t_3 \end{bmatrix}$
Start with a good guess of the solution $x_0 = (a,b,c)$, then update your guess with the Newton-Raphson recursive update formula
$ x_{k+1} = x_k - J^{-1} f(x_k) $ , $k = 0, 1, 2, ... $
This method is very robust and will lead to a solution if it exists within $4-5$ iterations for this small number of variables (3 variables in this problem).
The only catch is you need to have a routine to find the inverse of a matrix (because you need to calculate $J^{-1}$).