Let $U \in \mathbb{R}^{n}$ be open, and let $f: U \mapsto \mathbb{R}$ be a $C^{1}$ function. We obviously have $\nabla f$ = $(Df)^{T}$ where $T$ denotes the transpose of the matrix.
When are the norms of the gradient of $f$ and the derivative of $f$ equivalent? In other words, is the norm of the gradient is $||\nabla f||$ $\textbf{always}$ equal to the norm of the derivative $||Df$|| ?
My question is motivated by the Mean Value Inequality, i.e.
$$||f(b) - f(a)|| \leq M ||b - a||$$
where $a, b \in U$ with the segment $\{a+t(b-a): 0 \leq t \leq 1\} \subset U$ and $M = \mbox{max}_{0 \leq t \leq 1} ||Df(a+t(b-a)||$.
Can we replace $Df$ with $\nabla f$ in $M$ and write $M = \mbox{max}_{0 \leq t \leq 1} ||\nabla f(a+t(b-a)||$ ?
Another question. In the $||\nabla f||$ formulation, the norm is just the standard Euclidean one. For the Mean Value Inequality, what would be the standard norm used for the computation of $||Df||$ ?
Thanks.
Yeah, if you measure things correctly. You have made the observation that the differential is often written as a row vector and then the gradient is just the transpose of this vector. If you measure the norm of both row/columns vectors in the same way (say, using the standard Euclidean norm) which is a reasonable thing to do then clearly the result will be the same.
A more abstract and coordinate free approach would be to treat the differential $Df(p)$ as a linear functional. That is, let us fix an inner product on $\mathbb{R}^n$ (say the standard Euclidean inner product) and denote by $\| \cdot \|$ the induced norm on $\mathbb{R}^n$. Now it makes more sense to measure the norm of $Df(p)$ using the operator norm
$$ \| Df(p) \| := \sup_{||x|| = 1} | Df(p)x |. $$
In turns out that also in this case, if you define the gradient of $f$ correctly as vector (by the formula $Df(p)x = \left< (\nabla f)(p), x \right>$), you will also have $\| Df(p) \| = \| (\nabla f)(p) \|$ where on the left hand side we use the operator norm.