How can I locate the poles of a great circle given two non-opposing points?

Question

Given two non-antipodal points on the surface of a sphere (in -lat,lon or any similar coordinate system) how do I calculate the positions of the poles (one will do, of course) of the great circle that runs through them? I can work out trigonometric formulas and know the basics of spherical trig already. I simply have not found a formula for this. It seems like it should be possible to work it out by constructing and solving a triangle with two right angles in it, but relating the third vertex back to the original reference system doesn't seem to me a straightforward problem. Perhaps there is a conversion formula for such a rotation of coordinate systems.

MOTIVATION: Google Earth should, but doesn't have a feature for drawing the great circle including two such points. I plan on using this calculation to center the provided small circle drawing tool, and just create one that has 90 degrees of arc in its 'radius'.

Answer 1

Let $A, B$ be the two given points, and $N$ be the north pole. WLOG, I pick $B$ to be in the hemisphere east of $A$.

Using the polar angles of $A$ and $B$, and the difference of their longitudes, find the absolute bearing from $A$ to $B$. This is possible by considering the spherical $\triangle NAB$ and applying a formula to solve the spherical triangle given SAS:

$$\angle NAB = \arctan \frac{\sin (NB) \sin (\angle ANB)}{\sin(NA) \cos (NB) - \cos(NA)\sin (NB)\cos(\angle ANB)}$$

Add $\pi/2$ to, or subtract $\pi/2$ from, the above absolute bearing to obtain the absolute bearing from $A$ to a pole of the great circle through $A$ and $B$. I would choose to add or subtract $\pi/2$, to obtain a new bearing between $0$ and $\pi$.

Let $X$ be this pole. Spherical $\triangle NAX$ is a quadrantal triangle with side $AX=\pi/2$, so it's possible to apply Napier's rules for quadrantal triangles and find both

1. the length of side $NX$, i.e. the polar angle of $X$: $$\cos (NX) = \sin (NA) \cos (\angle NAX);$$

2. and the angle $\angle ANX$, i.e. the difference of longitudes of $X$ and $A$: $$\tan (\angle ANX) = - \frac{\tan(\angle NAX)}{\cos(NA)}.$$

Also handle the edge cases (sometimes trivial):

• If the two given points have the same longitude, or their longitudes are $\pi$ apart;
• If one given point is the north pole or the south pole;
• If $A$ is on the equator ($NA = \pi/2$);
• If the absolute bearing from $A$ to $B$ ($\angle NAB$) is exactly $\pi/2$;
• And maybe more.