Bound on remainder for vector valued Taylor series

Question

Consider an $n$-times differentiable map $f$ between finite dimensional real vector spaces. Specifically, let $f$ be defined on an open subset $U$.

Suppose the closed interval $[x,x+h]\subset U$ is entirely contained in $U$ and that the $(n+1)^\text{th}$ derivative exists on the interior $(x,x+h)$ of the interval. Write $M=\sup_{(x,x+h)}\|f^{(n+1)}(c)\|$.

I would like to prove the remainder of order $n$ satisfies the inequality $$\|R_nf(x)\|\leq\frac{M}{(n+1)!}\|h\|^{n+1}.$$

The only thing I can think of is precomposing with the straight path from $x$ to $x+h$ and then choosing coordinates on the target space to apply the mean value theorem in each coordinate. However, this gives an estimate in terms of the operator norms of the coordinates of the $(n+1)^\text{th}$ derivative which is looser than the "total" operator norm.

---

I probed a bit online and found the following "guide".

Consider the comopsite $f(x+th)$ with $t\in [0,1]$. Write $g_n(t)$ for the $n^\text{th}$ order remainder for $f(x+th)$. Prove the estimate for every $t\in (0,1)$ $$\|g_n^\prime(t)\|\leq \frac{1}{n!}M\|th\|^n\|h\|.$$ Use it to deduce $\|g_n(1)-g_n(0)\|\leq \frac{1}{(n+1)!}M\|h\|^{n+1}$ as desired.

Given the estimate on $g^\prime _n$ I know how to get the second inequality. However, I don't see how to prove this first estimate. Induction does not seem to help.

Answer 1

In these situations, the Mean-value Theorem/inequality (whatever you want to call it) is your best friend. What I'm about to write, I learnt from Henri Cartan's book on Differential Calculus. The relevant results are Theorem $3.11$ (Mean value theorem) and Corollary $5.53$, and Theorem $5.62$ (Taylor's theorem with Lagrange Remainder).

Theorem $3.11$ (Mean-Value Theorem):

Let $E$ be any real Banach space, $[a,b]$ an interval in $\Bbb{R}$ with $a< b$. Let $u:[a,b] \to E$ and $g:[a,b] \to \Bbb{R}$ be continuous functions on $[a,b]$, and differentiable on the open interval $(a,b)$. Suppose that for all $t \in (a,b)$, it is true that \begin{align} \lVert u'(t)\rVert & \leq g'(t) \end{align} Then, $\lVert u(b) - u(a)\rVert \leq g(b) - g(a)$.

The proof of this theorem, while elementary in methods, is slightly technical. But, you said you know how to deduce the second inequality from the first, so I guess you must know of the Mean value theorem either in this form or something similar, so I'll omit the proof.

---

Next, we have the following result (Proposition $5.51$ and Corollary $5.53$ combined)

Theorem:

Let $E$ be a real Banach space, $I$ an open subset of $\Bbb{R}$ containing the closed interval $[0,1]$, and let $v: I \to E$ be $(n+1)$-times differentiable on $I$, and suppose that $M:= \sup\limits_{t \in [0,1]} \lVert v^{(n+1)}(t)\rVert < \infty$. (Note that boundedness of derivative is obviously a weaker assumption than the $(n+1)^{th}$ derivative being continuous on $I$). Then the $n^{th}$ order remainder of $v$ at $0$ satisfies: \begin{align} \left \lVert v(t) - \sum_{k=0}^n \dfrac{v^{(k)}(0)}{k!} t^k\right \rVert &\leq \dfrac{M}{(n+1)!}. \tag{$\ddot{\smile}$} \end{align}

This will be the crucial link to help you deduce the theorem you want, because, we'll apply it to $v(t) = f(x + th)$, for $t$ in an open interval containing $[0,1]$. Here's the proof of this theorem:

Define the functions $u: I \to E$, and $g: I \to \Bbb{R}$ by \begin{align} u(t) &= \sum_{k=0}^n \dfrac{(1-t)^k}{k!} v^{(k)}(t) \quad \text{and} \quad g(t) = -M \dfrac{(1-t)^{n+1}}{(n+1)!} \end{align}

Then a simple calculation using the product rule (there will be a telescoping sum) shows that \begin{align} u'(t) &= \dfrac{(1-t)^n}{n!} v^{(n+1)}(t) \end{align} Hence, for all $t \in [0,1]$, we have \begin{align} \lVert u'(t)\rVert & \leq M \dfrac{(1-t)^n}{n!} = g'(t) \end{align} This is exactly the situation of the Mean-value theorem, so, we have $\lVert u(1) - u(0)\rVert \leq g(1) - g(0)$. If you plug in what $u$ and $g$ are, you'll find exactly $(\ddot{\smile})$. (The $u$ was constructed to ensure exactly this).

---

Lastly, we let $v(t) = f(x+th)$, where the $f,x,h$ are as in your question. Then, the $(n+1)^{th}$ derivative of $v$ at $t$ is given by (chain rule, and induction) \begin{align} v^{n+1}(t) &= (D^{n+1}f)_{x+th} [h]^{n+1} \in E \end{align}

Btw here, the RHS means the $(n+1)^{th}$ Frechet derivative of $f$ at $x+th$ evaluated on the $n+1$ tuple $(h, \dots, h) \in E^{n+1}$. Now, we can bound this using the operator norm, and properties of continuous multi-linear maps: \begin{align} \lVert v^{n+1}(t)\rVert & \leq \lVert D^{n+1}f{(x+th)}\rVert \cdot \lVert h\rVert^{n+1} \\ & \leq M \cdot \lVert h \rVert^{n+1} \end{align}

So, if in $(\ddot{\smile})$ you now substitute what $v$ is, and replace $M$ by $M \cdot \lVert h\rVert^{n+1}$, then you find that \begin{align} \left \lVert f(x+th) - \sum_{k=0}^n \dfrac{D^kf_x(th)^k}{k!} \right \rVert & \leq \dfrac{M}{(n+1)!}\lVert h \rVert^{n+1} \end{align} which is exactly the bound on the remainder term.

Answer 2

Let $j^n_a(x)$ be the $n^\text{th}$ order Taylor expansion of $f$ about $a$. Let $R_a^nf(x)=f(x)-j_a^n(x)$ be the remainder. The answer below makes crucial use of the mean value inequality of peek-a-boo's answer.

Theorem. Let $V$ be a finite-dimensional real vector space. Let $I\supset(a,a+h)$ be an open subset of $\mathbb{R}$. Let $f:I\to V $ be $(n+1)$-times differentiable in $(a,a+h)$. Suppose for any $t\in(a,a+h)$ we have $$ m\leq\|D^{n+1}f(t)\|\leq M. $$ Then for any $t\in(a,a+h)$ we have $$ \tfrac{m}{(n+1)!}(t-a)^{n+1}\leq\|R_{a}^{n}f(t)\|\leq\tfrac{M}{(n+1)!}(t-a)^{n+1}. $$

Proof. By induction on $n$. The case $n=-1$ is trivial. Assume the assertion holds for $n-1$. We shall apply this to the function $Df$ which is by assumption $n$-times differentiable on $(a,b)$. The function $Df$ satisfies the hypothesis because $D^{n}Df=D^{n+1}f$ whence $$ m\leq\|D^{n}Df(t)\|\leq M. $$ By the induction hypothesis $$ \tfrac{m}{n!}(t-a)^{n}\leq\|R_{a}^{n-1}Df(t)\|\leq\tfrac{M}{n!}(t-a)^{n}. $$ Observe the LHS and RHS are respectively the derivatives at $t$ of the functions $$ \tfrac{m}{(n+1)!}(t-a)^{n+1},\tfrac{M}{(n+1)!}(t-a)^{n+1}. $$ Moreover, $R_{a}^{n-1}Df=DR_{a}^{n}f$. The mean value inequality along with the fact $R_{a}^{n}f(a)=0$ therefore gives the desired inequality.

Theorem. (Vector-valued Taylor theorem.) Let $V,W$ be finite-dimensional real vector spaces and $U\subset V$ open. Suppose $[a,a+h]\subset U$ and let $f:U\to W $ be $(n+1)$-times differentiable at the points of the open interval $(a,a+h)$. If for every $x\in(a,a+h)$ we have $$ \|D^{n+1}f(x)\|\leq M. $$ Then for any $x\in(a,a+h)$ we have $$ \|R_{a}^{n}f(x)\|\leq\tfrac{M}{(n+1)!}\|x-a\|^{n+1}. $$

Proof. Define $\gamma(t)=a+th$ and consider the composite $f\circ\gamma$ on the open unit interval $(0,1)$. By the chain rule and induction $$ (f\circ\gamma)^{(k)}(t)=D^{k}f(a+th)(h^{\otimes k})\in{\bf V}. $$ This has two consequences. First, $$ \sup_{(0,1)}\|D^{n+1}(f\circ\gamma)(t)\|\leq\|D^{n+1}f(a+th)\|\|h\|^{n+1}\leq M\|h\|^{n+1} $$ whence $$ \|R_{0}^{n}(f\circ\gamma)(t)\|\leq\tfrac{M\|h\|^{n+1}}{(n+1)!}t^{n+1}=\tfrac{M}{(n+1)!}\|th\|^{n+1} $$ Second, taking $t=0$ shows $$ (f\circ\gamma)^{(k)}(0)=D^{k}f(a)(h^{\otimes k}) $$ whence $$ (f\circ\gamma)^{(k)}(0)t^{k}=D^{k}f(a)(h^{\otimes k})t^{k}=D^{k}f(a)(th)^{\otimes k}. $$ Combining these equalities for all $0\leq k\leq n$ gives by definition $$ R_{0}^{n}(f\circ\gamma)(t)=R_{a}^{n}f(\gamma t). $$ Now, the above theorem and the above estimate combine to give $$ \|R_{a}^{n}f(\gamma t)\|=\|R_{0}^{n}(f\circ\gamma)(t)\|\leq\tfrac{M}{(n+1)!}\|th\|^{n+1}. $$ Finally, $x\in(a,a+h)$ means $x=a+th$ so that $th=x-a$. Plugging this in gives the desired inequality.