Dot product of the gradient of a function

Question

In matrix calculus, I keep on seeing things like $\langle \nabla f(x), v\rangle$, which is the dot product of the gradient of a function with a vector.

I was wondering if there is any intuitive understanding of what this means.

For example, we have the Mean Value Theorem:

Let $\mathcal{O}$ be an open subset of $\mathbb{R}^{n}$ and suppose the mapping $F : \mathcal{O} \rightarrow \mathbb{R}^{m}$ is continuously differentiable. Suppose that the points $x$ and $x + h$ are in $\mathcal{O}$ and that the segment joining these points are also in $\mathcal{O}$. Then, there exist numbers $\theta_1, \theta_2, \ldots, \theta_m$ in the open interval $(0, 1)$ such that $$F_{i}(x + h) - F_{i}(x) = \langle \nabla F_{i}(x + \theta_{i}h), h\rangle $$

I was wondering if there is any good way to interpret $\langle \nabla F_{i}(x + \theta_{i}h), h\rangle$ in this context.

Thanks

Answer 1

Note that the derivative $d_pf$ of $f\colon\mathbb R^n\to\mathbb R$ in a point $p$ is not a vector, but a linear form instead.

In presence of an inner product $\langle.,.\rangle$ the gradient $\nabla^{\langle .,.\rangle}f(p)$ in respect to the inner product $\langle .,.\rangle$ is the unique vector which represents this linear form in presence of the specified inner product, that is $$d_pf(v)=\langle\nabla^{\langle .,.\rangle}f(p),v\rangle.$$

Answer 2

It is enough to consider a function $f:\>{\cal O}\to{\mathbb R}$. Assume that the segment $[x,x+h]=:\sigma\subset{\cal O}$. We then are told that there is a $\theta\in(0,1)$ such that $$f(x+h)-f(x)=\langle\nabla f(x+\theta h),h\rangle\ .\tag{1}$$ Now the definition of "differentiable" says that $$f(x+h)-f(x)=\langle\nabla f(x),h\rangle+o\bigl(|h|\bigr)\qquad(h\to0)\ ,$$ so that we have the approximation $f(x+h)-f(x)\approx \langle\nabla f(x),h\rangle$ for small $|h|$. The relation $(1)$ gives an "exact equation", but involves an unknown $\theta\in(0,1)$. It says that the true increment $f(x+h)-f(x)$ is equal to the true derivative of $f$ at some point on $\sigma$, applied to the increment vector $h$ of the independent variable. This is the multivariate version of the MVT $$f(x+h)-f(x)=f'(x+\theta h)\,h,\qquad0<\theta<1\ ,$$ from calculus 101.