Multivariate Mean Value Theorem Reference

Question

There is this result that I'm almost sure I've proven it a few times, but every once in a while I forget wheter it is true or false. Also, everytime I forget it, I try to look it up and never find a reference of it again (wasn't in one of the analysis books on my hard drive?). The result is:

Let $f\colon U \subseteq \mathbb{R}^n \to \mathbb{R}^m$ a differentiable function on a connected subset of $\mathbb{R}^n$. Then, for any $x,y \in U$ there is a $z \in U$ satisfying: $$|f(x)-f(y)| \leq |Df(z)(x-y)|$$

I find it on wikipedia for the case $m = 1$ or $n = 1$, but I can't seem to find the general proof anywhere. Am I mad? Could someone please point me to a reference of this theorem?

Answer 1

These slides give the description of the multivariate mean value theorem with a proof. The statement they provide is, for $x,y \in \mathbb{R}^{n}$:

\begin{equation} ||f(x) - f(y)||_q \leq \sup_{z\in[x,y]}||f'(z)||_{(q,p)}||x-y||_p \end{equation}

Where $z\in[x,y]$ denotes a vector $z$ contained in the set of points between $x,y\in\mathbb{R}^n$, and $||f'(z)||_{(q,p)}$ is the $L_{(p,q)}$ norm of the derivative matrix of $f:\mathbb{R}^n \rightarrow \mathbb{R}^m$ evaluated at $z$.

Consequences of the Multivariate Mean Value Theorem (MMVT)

An interesting result emerges when $q=p=k$. In this setting, the MMVT has the following form:

\begin{equation} ||f(x) - f(y)||_k \leq \sup_{z\in[x,y]}||f'(z)||_{(k,k)}||x-y||_k \end{equation}

With $||f'(z)||_{(k,k)} = ||[\nabla f_1(z),...,\nabla f_n(z)]||_k$, where $[\nabla f_1(z),...,\nabla f_n(z)]$ is the gradient of each of the output variables with respect to the domain variables concatenated together. Thus the properties of the matrix derivative evaluated at points between $x$ and $y$ gives an indication of whether the function $f(x)$ is a contractive map for points $x$ and $y$.

I believe that if the map is contractive for the $L_k$ norm for some specific $k\in \mathbb{Z}^+$ then it is contractive for all $k\in \mathbb{Z}^+$, but it might be something interesting to show or find a counterexample.

Another interesting consequence is, specifying $q=1$, then the expression for the MMVT is given by:

\begin{equation} ||f(x) - f(y)||_1 \leq \sup_{z\in[x,y]}||f'(z)||_{(1,p)}||x-y||_p \end{equation}

Where $||f'(z)||_{(1,p)} = \sum_{i=1}^n||\nabla f_i(z)||_p$

Indicating that the $L_1$ norm of $||f(x) - f(y)||_1$ is bounded above by the sum of the gradients of the output variables with respect to the domain variables.

That is, if the gradients of $f_i(z)$ for $i\in\{1,...,m\}$ are sufficiently small, and the points $x$ and $y$ are sufficiently close together, then the $m$ elements of $f(x)$ and $f(y)$ have a tendency to be shrunken close to each other. This is a consequence of the $L_1$ norm on the difference.

Finally, if $q = \infty$, we get the following result for the MMVT:

\begin{equation} ||f(x) - f(y)||_\infty \leq \sup_{z\in[x,y]}||f'(z)||_{(\infty,p)}||x-y||_p \end{equation}

With $||f'(z)||_{(\infty,p)} = \max_{i \in \{1,...,m\}}||\nabla f_i(z)||_p$.

This expression indicates that the output variable $j = \max_{j} |f_j(x) - f_j(y)|$, which is the largest difference between $f(x)$ and $f(y)$ along any individual dimension, is bounded above by a function involving the maximum $L_p$ norm of the $m$ gradients. Note that the norm of the gradient involves all dimensions, and $L_\infty$ norm on $f(x) - f(y)$ involves only the maximal difference along an individual dimension.

Interesting Point

The three previous results are interesting when you consider unit vectors around the origin. That is $x = 0$ and $||y||_p = 1$.

Utilizing those restrictions on $x$ and $y$ gives us that the the projection of the unit ball under $f$ is contained in the $L_q$-ball with radius provided by some functions of the gradient evaluated along $\overleftarrow{y}$. Which seems obvious, but this setting gives some mathematical tractability for it.

Answer 2

I cannot point you to a reference but I have a strong feeling I saw a similar result. However, I think that since as soon as you have connectedness in $\mathbb{R}^n$ you also have path-connectedness and connectedness by polygonal paths you can basically mimic the proof of the convex argument for each "segment", thus obtaining a finite set of points $z_i$ that verify the MVT in the given segment and then choose $z$ to be the point where the matrix norm of the differential is the highest among the $z_i$.

Answer 3

You can find (almost) the result you ask for in Mathematical Analysis by T.M. Apostol, Sec. 12.11, Example 2 (see also Principles of mathematical analysis by Walter Rudin, Theorem 5.19). The result is derived from an interesting generalization of the classical Mean Value Theorem (MVT) for real-valued functions to multivariable functions. Namely,

$f:U\subseteq\mathbb{R}^n \rightarrow \mathbb{R}^m $ differentiable in $U$ (open set). $x, y\in U$ such that the line segment $[x,y]\subseteq U$. Then for every vector $a\in \mathbb{R}^m $ there is a point $z \in [x,y]$ such that: $$ a \cdot (f(x)-f(y)) = a \cdot Df(z)(x-y) $$

This multivariable MVT result can be derived quite straight forward applying the standard one-dimensional MVT to the real-valued function $\varphi:[0,1] \rightarrow \mathbb{R}$, $\varphi (t)= a \cdot f(tx + (1-t)y) $. In fact, one only needs $f$ continuous in $[x,y]$ and differentiable in $(x,y)$.

Using this result and taking $ a = \frac{f(x)-f(y)}{|f(x)-f(y)|}$, i.e., $|a|=1$ and applying the Cauchy-Schwarz inequality to the right-hand side one readily gets: $$ |f(x)-f(y)| \leq |Df(z)(x-y)| $$