Obtain the shortest distance between two points on a sphere

Question

Let person A be at latitude : 40N and longitude 0.4W

Let person B be a the latitude: 40N and longitude 74E

Find the shortest distance between them.

We use Haversine formula:

$$a = \sin^2(\Delta\phi/2) + \cos \phi_1 \cdot \cos \phi_2 \cdot \sin^2(\Delta\Lambda/2)$$

Phi=latitude, Lambda=Longitude.

Convert to radians: Person A: $\phi=40\cdot \frac{\pi}{180}=0.7$, $\lambda=0.4\cdot \frac{\pi}{180}=0.007$

Person B: $\phi=40\cdot \frac{\pi}{180}=0.7$, $\lambda=74\cdot \frac{\pi}{180}=1.3$

$\Delta\phi=1.3-0.007=1.293$, $\Delta\lambda=0$

$$a = \cos0.7 \cdot \cos0.7 \cdot sin^2(0.6465)=0.212$$

Then we find

$c = 2 \cdot \arctan(2( \sqrt{a}, \sqrt{(1−a)} )$

We insert for $a$:

$c = 2 \cdot \arctan(2( \sqrt{0.212}, \sqrt{(1−0.212)} )=(1.48845, 2.11566)$

Then we calculate the distance, with the Earth radius at $R=6371km$

$d = 6371 \cdot c$

But how does that distant result when $c$ is a coordinate?

Any help appreciated.

Thanks

Answer 1

The reason for your confusion is a semantical misunderstanding about the formula $\arctan(x, y)$.

The formula $\arctan(x, y)$ has 2 inputs, $x$ and $y$, and has 1 output, $\theta$.

$\theta$ corresponds to the angle formed by the right triangle in which $a$ is opposite to the angle, and $b$ is adjacent to the angle.

In conclusion, you performed element-wise calculation on a vector, when instead you should have computed $\arctan$ like I explained above.

Answer 2

The angular distance as observed from the Earth center is according to the standard formula https://en.wikipedia.org/wiki/Great-circle_distance $$ \arccos( \sin 40^\circ\sin 40^\circ + \cos 40^\circ \cos 40^\circ\cos 74.4^\circ) = \arccos( 0.41317+ 0.5868\times 0.2689) $$ $$ = 0.96309 rad $$ so the distance is $6371\times 0.96309$ km $\approx 6135.8$ km.