Variational characterization of gradient?

164 Views Asked by At

Let $f: \mathbb{R}^n \to \mathbb{R}$ be a differentiable function. One way to define the gradient of $f$ is as the vector whose inner product with any other vector gives the directional derivative in that direction:

$$ u \cdot \nabla f = D_u f.$$

However, this definition may be a bit abstract for students who are being introduced to the concept for the very first time. Another, perhaps more intuitive description of the gradient is that it is the "direction of steepest ascent."

Question: Is there a nice way to make this description more formal?

For instance, a student might ask "what does steepest really mean?" One possible description is

$$ \nabla f(x) = \lim_{h \to 0}\ \mathrm{arg}\!\max_{u \in S^n} f(x+hu),$$

where $S^n$ denotes the unit sphere in $\mathbb{R}^n$. In other words: among all possible directions, which one increases the value of the function most if we walk for a distance $h$? Now take the limit as we take smaller and smaller steps.

I don't particularly like this definition, though, because it's complicated, involving both maximization of a function over a set and limits—either of which might be alien to a student early in her career.

Still, I have to think there's a nice, simple way to convey the idea of "steepest ascent" in precise mathematical terms...

1

There are 1 best solutions below

0
On

Here is a different way to go about it, although perhaps different than what you had in mind.

Consider a function $f = f(x,y)$. The rate of change of $f$ is $f_x$ in the $x$ direction and $f_y$ in the $y$ direction. These are rates of change for perpendicular directions, so the maximum possible rate of change is given by \begin{align} \sqrt{f_x^2 + f_y^2} \end{align} This is also equal to the magnitude of a vector given by \begin{align} f_x\hat{\mathbf{x}} + f_y\hat{\mathbf{y}}. \end{align} This vector is called the gradient vector, and is denoted $\nabla f$. If $\hat{\mathbf{u}}$ is a unit vector such that \begin{align} \hat{\mathbf{u}}\cdot\nabla f=\left|\nabla f\right| \end{align} then $\hat{\mathbf{u}}$ points in the direction of $\nabla f$. Therefore the direction of $\hat{\mathbf{u}}$ is the direction of maximal increase of $f$.