Why does this approximation of square roots using derivatives work this way?

Question

I came up with this way to estimate square roots by hand, but part of it doesn't seem to make sense.

Consider how $f(n) = \sqrt{n^2+\varepsilon} \approx n$ when $\varepsilon$ is small. Therefore, using the tangent line with slope $f'(n) = \frac{n}{\sqrt{n^2+\varepsilon}}$ to approximate $f$ gives

$f(n) \approx n +\varepsilon\cdot\frac{n}{\sqrt{n^2+\varepsilon}}$

However, if the original approximation is substituted in the denominator, this gives

$f(n) \approx n + \frac{\varepsilon n}{n} = n + \varepsilon$

Which obviously makes no sense. However, if the chain rule is skipped while taking the derivative, the approximation becomes

$f(n) \approx n + \frac{\varepsilon}{2n}$

Which is a good approximation. For example, when $\varepsilon = 1, \sqrt{n^2+1} \approx n+\frac{1}{2n},$ and when $\varepsilon = n, \sqrt{n^2+n} \approx n + \frac{1}{2}$

Why does this work only when the derivative is done incorrectly? I feel like I am missing something obvious, but I can't see why this works the way it does.

Answer 1

This seems like an improper application of the tangent line approximation. The usual approximation is $$f(x+\varepsilon)\approx f(x)+\varepsilon f'(x)\tag{*} $$ for $\varepsilon$ small. Your choice of $f$ doesn't match up with $(*)$. But using $f(x)=\sqrt x$ gives $$ \sqrt{x+\varepsilon}=f(x+\varepsilon)\approx\sqrt x+{\varepsilon\over{2\sqrt x}}$$ and plugging $x=n^2$ yields $$ \sqrt{n^2+\varepsilon}\approx n+\frac\varepsilon{2n},$$ the approximation you're seeking.

Answer 2

$\newcommand{\eps}{\varepsilon}$In order to make a first-order approximation, you must fix three pieces of data: A differentiable function $f$, a point $a$ of the domain where the approximation is taken, and a point $x$ at which $f(x)$ is approximated. In this situation, $$ f(x) \approx f(a) + f'(a) (x - a),\quad\text{or}\quad f(a + \eps) \approx f(a) + \eps \cdot f'(a). \tag{1} $$

In your set-up, you write "$f(n) = \sqrt{n^{2} + \eps}$". This is already unusual, in that $\eps$ is built into the definition of $f$.

Next, by writing "$f'(n) = n/\sqrt{n^{2} + \eps}$", you're taking $a = n$. Normally, first-order approximation is done in a way that $f'(a)$ is easy to evaluate exactly; here, $f'(a)$ involves a denominator to be approximated. That said, logically there's no problem.

In the third line, $$ f(n) \approx n + \eps \cdot \frac{n}{\sqrt{n^{2} + \eps}}, \tag{2} $$ each summand is problematic:

• First, the left-hand side is $f(a)$, not $f(x)$.

• Second, the "constant term" $n$ is not $f(a)$, i.e., $f(n) = \sqrt{n^{2} + \eps} \neq n$. Line (2) is not of the form (1).

• Third, "$\eps$" plays two distinct roles: A number "baked into" the definition of $f$, and a small number representing $x - a$ in (1).

Here's what I suspect you were aiming for: define $g(x) = \sqrt{x^{2}}$ and $h(x) = \sqrt{x}$. You want the linear approximation of $g$ at $a = n$, and the approximation of $h$ at $a = n^{2}$: $$ g(n + \eps) \approx n + \eps \cdot \frac{n}{\sqrt{n^{2}}};\qquad h(n^{2} + \eps) \approx n + \eps \cdot \frac{1}{2\sqrt{n^{2}}}. $$ The first is exact (so long as $n$ and $n + \eps$ are positive) since $g(x) = x$ for $x > 0$. The second is the "chain rule skipped" approximation you were expecting.