I have a geodesic (connecting points A and B) on a sphere. There's a another point C on the same sphere. So, I know distance AB. If necessary, AC and BC can also be computed.
I need to find the shortest distance between the point C and the geodesic line AB. My initial attempt was to use an idea like Lagrange multipliers to find the shortest arc length, however, it is difficult to solve the differential equation.
How can I calculate the shortest distance between C and AB? Thanks.
I will assume that it is a sphere of radius r. Think of $A,B$ and $C$ as vectors of length $r$ in $\mathbb{R}^3$. The geodesic $AB$ is a great circle on the sphere, it divides the sphere into lower and upper hemisphere. The unit normal vector of this great circle is \begin{equation} N = \frac{A\times B}{|A\times B|}. \end{equation} The angle between $C$ and $N$ is $\theta = \cos^{-1}(\frac{C\cdot N}{r})$. Then the angle between $C$ and $AB$ is $\pi/2 - \theta$. Therefore the shortest distance is $(\pi/2 - \theta) r = r(\pi/2 - \cos^{-1}(\frac{C\cdot N}{r}))$.