Let $A(1,-3,4)$ and $B(3,-2,-1)$ and find the set of all $C(x,y,z)$ such that $ABC$ is a right triangle with hypotenuse $AB$
What I did
$$AB=(2,1,5)$$ $$BC=(x-3,y+2,z+1)$$ $$AC=(x-1,y+3,z-4)$$
If $AB$ is the hypotenuse, then $AB$ and $BC$ are orthogonal and so: $$BC \cdot AC = 0 \implies z^2-3z+5+x^2-4x+y^2+5y = 0$$
I tried to substitute $c= 5+x^2-4x+y^2+5y$ to get $$z^2-3z+c=0 \implies z = \frac {3 \pm \sqrt{-4x^2+16x-4y^2-20y-11}}{2}$$
(note: the Pithagorean identity could also be used and yields the same result, because it assumes orthogonality)
I'm not sure if this is right, but I cannot see how this set gets the form it has...
I mean, if $AC$ was the hypotenuse, then the set would be a plane: $C$ can go any further from $AB$ and can rotate around the "axis" $AB$:
The same would hold if $BC$ was the hypotenuse, and the planes in both cases would be parallel. Here are the forms of the graphs (pink is with the "+" sign and yellow with the "-" sign; in blue the set obtained when $AC$ is the hypotenuse and in green, when $BC$ is the hypotenuse)
Why does the set I got has a form of two pieces of a broken spherical shell?
Let $d=\sqrt{30}$ be the length $|AB|$. Imagine $d$ horizontal in the $xy$-plane (blue below), with its midpoint at the origin. Then, by Thales' theorem, the points $C$ that form a right angle with $d$ lie on a circle (red below) of diameter $d$, centered on the origin.
Now spin the circle about $d$, and you form a sphere. What you are seeing in 3D is this sphere, perhaps with programming or numerical computation issues near the equator.
If you re-factor your equation: $z^2-3z+c=0$, $c=5+x^2-4x+y^2+5y$, you can see this more clearly before plotting, as this can be rearranged to:
$(x-2)^2+(y+\frac{5}{2})^2+(z-\frac{3}{2})^2=\frac{15}{2}$
It should be clear that this is the equation for a sphere centered at $(2,-\frac{5}{2},\frac{3}{2})$ with radius $\sqrt{\frac{15}{2}}$ (i.e., diameter $\sqrt{30}$); note also that the center is exactly the midpoint of your two starting points.