Find the missing triangle vertex given 2 vertices, 2 side lengths, a right angle, and plane normal

Question

Trying to solve this problem so I can write a function in code. This is all in a 3D, left handed coordinate space if it matters.

1. Vertices $\mathbf{A}$, $\mathbf{B}$, and $\mathbf{C}$ form a triangle and each has $\mathbf{x}$, $\mathbf{y}$, and $\mathbf{z}$ coordinates.
2. Point $\mathbf{A}$ and $\mathbf{B}$ are known.
3. Distance between $\mathbf{A}$ and $\mathbf{B}$ is $\mathbf{\gamma}$ and is known.
4. Distance between $\mathbf{A}$ and $\mathbf{C}$ is $\mathbf{\beta}$ and is known.
5. Angle between $\mathbf{\overline{CA}}$ and $\mathbf{\overline{CB}}$ is a right angle.
6. Vector $\mathbf{N}$ is a known plane normal for a plane that contains $\mathbf{A}$ and $\mathbf{C}$, but not necessarily $\mathbf{B}$.
7. Find point $\mathbf{C}$.

Nothing else is known. I recognize there are 1 or 2 solutions depending on the direction of the plane normal. I recognize Pythagorean can be used to find missing triangle side length. I understand dot and cross products (if they're relevant). I get lost soon after that.

Ultimately, I'm looking for one or more equations I can translate into code to produce values for $\mathbf{C}$.

Answer 1

Let $\mathbf{A}=(a_1,a_2,a_3)$, and let $\mathbf{B}=(b_1,b_2,b_3)$. Consider the following spheres:

1. Sphere $S_1$ centered and $\frac12(a_1+b_1,a_2+b_2,a_3+b_3)$ with radius $\frac12\sqrt{(a_1-b_1)^2+(a_2-b_2)^2+(c_1-c_2)^2}$; this is the only sphere which has $\mathbf A$ and $\mathbf B$ as antipodal points;
2. Sphere $S_2$ centered at $\mathbf A$ with radius $\beta$.

The intersection of $S_1$ with $S_2$ is a circle $c$. By Thales' theorem and by definition of $S_1$, every point $\mathbf{P}\in c$ is such that the angle between $\overline{\mathbf{PA}}$ and $\overline{\mathbf{PB}}$ is a right angle and, by the definition of $S_2$, the distance between any point $\mathbf{P}\in c$ and $\mathbf A$ is $\beta$. So, all that is left is to compute the intersection between $c$ and the plane orthogonal to $\mathbf N$ passing through $\mathbf A$. Any $\mathbf C$ in that intersection will do.