Methods of Calculating Great Circle Distances

Question

I wish to calculate the shortest distance between two points on the Earth. What are the methods of doing this?

So far, the ones I know are:

1. Haversine formula, which assumes a spherical Earth - great circle.
2. Vincenty's formulae, which assumes an oblate spheroid and is therefore more accurate than the Haversine

I was wondering how does calculus of variations and geodesics come in. Specifically, I read this paper. Is the method of calculus of variations any more accurate than Vincenty's or the Haversine? Or is this a totally different method altogether?

I would appreciate it if someone could explain so that a 15 year old would understand, thanks.

Answer 1

"Geodesic" is just the name we give to the shortest path on a surface between two points.

• For planes or 3-dimensional Euclidean space, geodesics are just straight lines.
• On a sphere, the geodesics turn out to be "great circles". That is, a circle resulting from intersecting the sphere with a plane that passes through the sphere's center.
• For an oblate spheroid, the geodesics are tougher to identify. Any plane passing through both poles of the spheroid will produce a geodesic. The equator is also a geodesic. But if you consider two points that not directly north and south of each other, and do not both lie on the equator, the geodesic between them is not easily described.

The "calculus of variations" is a technique for finding geodesics, but it only works indirectly. (Actually the calculus of variations is useful for finding many things, but here we are only concerned with geodesics.) The basic idea behind the calculus of variations is that since a geodesic is the shortest path between two points, if we modify it anywhere, no matter how small the change, the length of the path should increase. If we use a parameter to control the size of the change, then the length of the path is a function of the parameter which has its minimum value when size of the modification is $0$.

If you don't know anything about calculus, then how this works requires a considerable amount of new concepts, far more than could be reasonably included here. Consider it voodoo magic which produces a weird equation that tells us what geodesics should look like.

If you know any calculus, then you may recall that at its minimum, a well-behaved function has a derivative of $0$. By setting the derivative of the length with respect to the variation parameter to $0$, we get a "differential equation" that the geodesic must solve. How to solve differential equations is a major concern (they pop up everywhere when math is applied), so we have developed a lot of tools for that purpose.

So the calculus of variations gives us the differential equation, but it is solving the differential equation, given the two endpoints, that provides the geodesic between them. And it is the geodesic that allows us to determine the distance along the surface between the endpoints.

• For planes, this is all simple to work out, and just leads back to the Euclidean distance formula $$s = \sqrt{(x_1 - x_2)^2 + (y_1 - y_2)^2+ (z_1 - z_2)^2}$$
• For spheres, the math requires trigonometric functions, but still results in the relatively simple haversine formula.
• For oblate spheroids, there is no nice general formula. Instead we use numerical techniques for estimating the distance to any desired precision. This is what Vincenty's formulae are doing. This is actually less of a problem than people suppose. We cannot calculate square roots or trig functions to perfect accuracy either and use similar numerical methods when we calculate with them.

The more complex the surface is, the harder it is to calculate distance along it.

---

Which brings us to the real issue in your question. The problem with using the haversine formula to calculate distances on Earth is not that the formula is inaccurate. The haversine formula is perfectly accurate for calculating distances on spheres. The inaccuracy is in treating the surface of the Earth as a perfect sphere. It is closer in shape to an oblate spheroid. So using Vincenty's formulae for oblate spheroids to calculate distance provides a better accuracy.

But Earth is also not exactly an oblate spheroid. To improve on Vincenty's formulae, we need a more accurate model of the Earth. That is not a mathematical problem. It is a Geoscience problem. You have to measure the Earth and develop of model of its surface that meets the standard of accuracy you desire. You are not going to find a simple description for it such as "oblate spheroid".

• It takes only 3 numbers to completely describe a plane: 2 angles to indicate the direction perpendicular to the plane, and the distance from the plane to the origin.
• It takes 4 numbers to completely describe a sphere: the 3 coordinates of the its center, and its radius.
• It takes 7 numbers to completely describe an oblate spheroid: 3 coordinates for its center, 2 angles to determine the direction of its axis, and the major and minor radii.
• Depending on how closely you want to model the Earth, it will likely take many thousands of numbers to describe the surface. Most likely you would want a triangulation of the surface using spherical triangles of varying radii for faces. Accurately calculating distances on such a surface would be a major undertaking, and best done by starting with the great circle route and tweaking it on each face.

And this begs the question "What exactly is the surface of the Earth anyway?" Do we include every mountain? Every hill? Every blade of grass? The more detailed your surface is, the greater the distance between any two points on it will be. If you were to look at the "blades of grass" level, you would discover that the "circumference of the Earth" is many times greater than 40,000 kilometers/25,000 miles, and constantly changing. How you answer that question depends very much on what your distance calculations will be used for. A hiker, a car traveller, and an airplane passenger have very different ideas about the distance from New York to Los Angeles.