This is embarrassing - please forgive me :)
I am going through an old high school textbook and the second chapter is in square roots. Page two has a figure a that demonstrates how the formula:
d = 8 x sqrt (2h)
At least I think that's how I am to interpret the formula or implement in code?..
Any who...where do the constants come from? I assume the 8 has something to do with the circumference of the earth? If you were to calculate visible horizon on a tennis ball, would the same formula work, do u use different constants?
Knowing the earth is oblate - how does that factor into the equation??
The formula assumes the earth is a sphere; the constants do in fact come from the size of the earth; a different formula would be needed for a tennis ball (i.e., a similar formula, but with different constants).
You can sort of see why the formula works by looking at this diagram.
The blue line is from the center of the earth to your feet; the darker blue line is from your feet to your head. The green line is from your eye to the horizon, and the orange line is from the center of the earth to the horizon point. At that horizon point, the green and orange lines are perpendicular, so this is a right triangle, and Pythagoras's theorem applies: \begin{align} R^2 + d^2 &= (R+h)^2 = R^2 + 2Rh + h^2\\ d^2 &= 2Rh + h^2 \\ d &= \sqrt{2Rh + h^2} \end{align} Now let's measure everything in miles. So $R = 4000$, and $h = H/5280$, where $H$ is your height in feet. So the formula becomes: \begin{align} d &= \sqrt{8000 \frac{H}{5280} + \frac{H^2}{5280^2}} \\ d &\approx \sqrt{1.5H + \frac{H^2}{27878400}} \end{align} Now since $H$ is a number around $6$, that second term is less than one millionth; the first term is about $9$. So we can approximate further by ignoring the second term, and write \begin{align} d &\approx \sqrt{1.5 H} \end{align} and conclude that the distance to the horizon (in miles) is about the square root of 1.5 times your height (in feet). Of course, if you're measuring height in meters and distance in km, you have to make adjustments.
And the whole argument depends on the second term under the square root being small, so when $H$ is large -- say $20\%$ of the earth's radius -- the formula starts to break down.