Prove that the Taylor series is the solution to this minimization problem?

Question

The answer to this question says that a MacLaurin polynomial of a function $f$ is the unique $N^\text{th}$ order polynomial that minimizes the following functional:

$$ L[g] = \lim_{x\rightarrow 0}\left(\frac{f(x)-g(x)}{x^N}\right)^2, $$ and that the minimum value attained is $0$, $$ \min_{g} L[g] = 0 $$

I'm wondering if anyone has a reference for a proof of this fact in a forward sense, i.e., starting from the above, show that $g$ must be equal to the formula for the MacLaurin series of $f$, without directly invoking some form of Taylor's theorem to expand $f$. Basically, assuming we didn't know anything about Taylor/MacLaurin series or polynomials, can we derive them from the above minimization problem?

Possible path

We know $g$ is an $N^\text{th}$ order polynomial: $$ g(x) = \sum_{n=0}^N a_n x^n = \vec{X}\cdot\vec{a}, $$ where $$ \vec{X} = \left[1\;\; x\;\; x^2\;\;\ldots\;\;x^N\right]^\text{T}, $$ and $$ \vec{a} = \left[a_0\;\; a_1\;\; a_2\;\;\ldots\;\;a_N\right]^\text{T}. $$ Now I know I'll need to know the sensitivities of that functional to the parameters: $$ \nabla_\vec{a} L[g] = \left[ \frac{\partial L[g]}{\partial a_0}\;\; \frac{\partial L[g]}{\partial a_1}\;\; \frac{\partial L[g]}{\partial a_2}\;\;\ldots\;\; \frac{\partial L[g]}{\partial a_N} \right]^\text{T} $$

And that through application of the chain rule, I'll also need that $$ \nabla_\vec{a} g = \vec{X}. $$ Setting the gradient of the functional w.r.t. $\vec{a}$ to zero for each component should give me a way to solve for $g$, I just don't see a clear path forward that results in the formula for a Taylor polynomial. I hope someone can illuminate the way.

Answer 1

Suppose $f(0) \ne g(0)$. Then the numerator of the limit is finite while the denominator goes to zero, so the limit diverges. Therefore, $L[g]$ can only be finite if $f(0) = g(0)$.

Apply L'Hôpital's rule and we have $$L[g] = \lim_{x\to0}\left(\frac{f'(x)-g'(x)}{Nx^{N-1}}\right)^2.$$ By the same logic, for $L[g]$ to be finite we must have $f'(0) = g'(0)$.

Repeat $N$ times, obtaining the conditions $f(0) = g(0)$, $f'(0) = g'(0)$, $f''(0) = g''(0)$, ..., $f^{(N-1)}(0) = g^{(N-1)}(0)$, until finally $$\begin{align} L[g] &= \lim_{x\to0}\left(\frac{f^{(N)}(x)-g^{(N)}(x)}{N!}\right)^2 \\ &= \frac1{N!}\bigl(f^{(N)}(0) - g^{(N)}(0)\bigr)^2, \end{align}$$ which is clearly minimized at $f^{(N)}(0) = g^{(N)}(0)$. Therefore, $f$ and $g$ must have the same $0$th to $N$th derivatives.

Answer 2

Let us split the problem in two parts: the existence of the minimum, and its uniqueness. For simplicity, notice that we may significantly simplify the problem by replacing the square in $L$ with a modulus, because the functions $x \mapsto x^2$ and $x \mapsto |x|$ have the same minimum, attained in the same point.

If $f$ admits derivatives up to (and including) order $n$ in $0$, let

$$T_{n,f} (x) = \sum _{k=0} ^n \frac {f^{(k)} (0)} {k!} x^k$$

be the $n$-th degree MacLaurin polynomial function of $f$.

Let $L_{n,f}$ be the (obviously positive) functional

$$L_{n,f} (g) = \lim _{x \to 0} \left| \frac {f(x) - g(x)} {x^n} \right| $$

We shall show that $T_{n,f}$ is a minimum of the functional $L_{n,f}$ by induction on $n$.

For $n=0$ this is clear for every $f$:

$$L_{0,f} (T_{0,f}) = \lim _{x \to 0} \left| \frac {f(x) - T_{0,f} (x)} {1} \right| = \lim _{x \to 0} \left| \frac {f(x) - f(0)} {1} \right| = 0$$

so $T_{0,f}$ is a point of minimum for $L_{0,f}$.

For $n>0$, the induction hypothesis is that for every $f$ having derivatives up to (and including) order $n$ in $0$, $T_{n,f}$ is a point of minimum for $L_{n,f}$; we shall prove the same statement for $n+1$:

$$L_{n+1, f} (T_{n+1, f}) = \lim _{x \to 0} \left| \frac {f(x) - T_{n+1, f} (x)} {x^{n+1}} \right| = \left| \lim _{x \to 0} \frac {f(x) - T_{n+1, f} (x)} {x^{n+1}} \right| = \left| \lim _{x \to 0} \frac {f'(x) - T'_{n+1, f} (x)} {(n+1) x^n} \right| = \frac 1 {n+1} \lim _{x \to 0} \left| \frac {f'(x) - T_{n, f'} (x)} {x^n} \right| = \frac 1 {n+1} L_{n, f'} (T_{n, f'}) = 0 .$$

In words, the explanation of the above equalities:

• the second "$=$" is because the modulus function $x \mapsto |x|$ is continuous, so we may slip the limit inside and outside of it;

• the third "$=$" is an application of L'Hospital's theorem;

• the fourth "$=$" is because the derivative of $T_{n+1, f}$ is $T_{n, f'}$ (check it!)

• the fifth "$=$" is again by continuity

• the last "$=$" is by the induction hypothesis (notice that if $f$ has derivatives up to (and including) order $n+1$ in $0$, then $f'$ has derivatives up to (and including) order $n$ in $0$).

We have thus shown that for every $f$ having derivatives up to (and including) order $n+1$ in $0$, $T_{n+1, f}$ is a point of minimum for $L_{n+1, f}$, and this ends the proof by induction, therefore establishing that the MacLaurin polynomial functions are points of minimum for their respective functionals.

---

Let us now prove the uniqueness part. Let $g$, a polynomial function of degree $n$, be a point of minimum for $L_{n,f}$. Then

$$0 = L_{n,f} (g) = \lim _{x \to 0} \left| \frac {f(x) - g(x)} {x^n} \right| = \lim _{x \to 0} \left| \frac {f(x) - T_{n,f}(x) + T_{n,f}(x) - g(x)} {x^n} \right| \le \\ \underbrace{ \lim _{x \to 0} \left| \frac {f(x) - T_{n,f} (x)} {x^n} \right|} _{= L_{n,f} (T_{n,f}) = 0} + \lim _{x \to 0} \left| \frac {T_{n,f} (x) - g(x)} {x^n} \right| = \lim _{x \to 0} \left| \frac {T_{n,f} (x) - g(x)} {x^n} \right| .$$

To show that this implies $g = T_{n,f}$ we shall prove a more general result: if $p$ is a polynomial function of degree $n$ such that $\lim \limits _{x \to 0} \frac {p(x)} {x^n} = 0$, then $p=0$. Again, we shall use induction.

If $n=0$, then $p$ is a constant (for this problem I consider the $0$ polynomial to have degree $0$, for a uniform formulation of the proof), and it immediately follows that the only possible constant is $0$.

Assuming the result proven for $n$, then if $p$ has degree $n+1$ we may write (by L'Hospital's theorem, again):

$$0 = \lim _{x \to 0} \frac {p(x)} {x^{n+1}} = \lim _{x \to 0} \frac {p'(x)} {(n+1)x^n} ,$$

which shows, according to the induction hypothesis, that $p' = 0$, so that $p$ must be a constant. Again, it is obvious that the only constant that works is $p=0$, and this ends our proof.

Applying now the above result to $p = T_{n,f} - g$ shows that $g = T_{n,f}$, so $T_{n,f}$ is the unique point of minimum for $L_{n,f}$.