Gradients, Directional Derivatives and Change in Scalar Functions

Question

In single variable scalar function $\ f(x)\ $the sign of the derivative can tell you whether the function is increasing or decreasing at the point. I was trying to find an analogous concept in multi-variable scalar function $\varphi(\vec r)\ $since its output is a scalar quantity just like in the single variable function. Now in these functions we have the gradient vectors $\nabla\varphi(\vec a)\ $which gives us the direction of maximum rate of change of the scalar field at $\vec a $. But since it is a vector I assume its magnitude is always +ve and does not tell us anything about increase or decrease of the scalar quantity. Then there is the directional derivative $\varphi'(\vec a,\hat n) $ which is the dot product of the gradient vector $\nabla\varphi(\vec a)\ $and the the unit vector $\hat n $. In these dot products we get get scalar quantity with a sign. I used to believe that the sign indicated whether the function increased or decreased in that particular direction. But by definition of directional derivative $$\varphi'(\vec a,\hat n)=\nabla\varphi(\vec a)\cdot\hat n=|\nabla\varphi(\vec a)|\cos\theta $$ where $\theta$ is the angle betweenn $\hat n$ and $\nabla\varphi $. So as we can see the sign only arises from $\cos\theta $. So I assume the +ve sign indicates that the change along $\hat n $ is in the same direction as $\nabla\varphi $ i.e it is increasing if $\nabla\varphi $ is increasing or vice versa and the -ve sign indicates it is in the opposite direction of $\nabla \varphi $ i.e it increases when $\nabla\varphi\ $ decrease or vice versa.

So am I wrong or is there such an analogue ?

Answer 1

Your initial guess was correct.

Note that $$\varphi'(\vec a,\hat n)=\nabla\varphi(\vec a)\cdot\hat n=|\nabla\varphi(\vec a)|\cos\theta$$

Note that if $$\theta =0$$ you get $$\cos \theta =1 $$and as a result your directional derivative is positive and maximized.

That is if your directin matches with your gradiant vector your function increases the most.

On the other hand if $$\theta = \pi $$ you get $$\cos \theta = -1$$ your directinal derivative is negative and your function decreases the most.

Answer 2

As I noted in a comment, $\nabla\varphi\cdot\hat n$ isn’t really the definition of the directional derivative in the direction of $\hat n$ but a consequence of the differentiability of $\varphi$ at a point. The fundamental definition is a difference quotient similar to the one used to define the derivative of a single-variable function: $$\lim_{h\to0}{\varphi(\vec a + h\hat n)-\varphi(\vec a)\over h}.$$ If you visualize the graph of $\phi:\mathbb R^n\to\mathbb R$ as a hypersurface in $\mathbb R^{n+1}$ (it’s easiest to do this in $\mathbb R^3$, but the ideas hold in higher dimension), this amounts to taking a “vertical” slice through the hypersurface and computing the derivative of the resulting curve. Just as in the single-variable case, the sign of this derivative tells you whether the function is increasing, stationary, or decreasing in the direction of $\hat n$. Note that this “vertical” cutting hyperplane is the same for both $\hat n$ and $-\hat n$. The difference between the two is which “side” you’re looking at: the intersection curve for $-\hat n$ is the reflection of the curve for $\hat n$.

If $\varphi$ is differentiable at $\vec a$, then an equivalent interpretation of the directional derivative is that it’s the slope of the tangent hyperplane in the direction of $\hat n$. This is the geometric content of the expression $\nabla\varphi(\vec a)\cdot\hat n$. As you know, this slope is greatest in the direction of the gradient of $\varphi$. If we take $\hat n = {\nabla\varphi(\vec a)\over\lVert\nabla\varphi(\vec a)\rVert}$, so that the angle $\theta$ between the gradient and $\hat n$ is zero, then using the above formula the directional derivative in the direction of the gradient is equal to $\lVert\nabla\varphi(\vec a)\rVert$. This is always nonnegative, so $\varphi$ is always nondecreasing in the direction of its gradient. That this must be the case is fairly obvious if you think about it in terms of the tangent plane to a surface in $\mathbb R^3$: in any direction that the slope is negative, you can simply go in the opposite direction to get a positive slope, so the maximum slope cannot be negative. So, if you go in the opposite direction from the gradient, i.e., $\theta=\pi$, the directional derivative is equal to $-\lVert\nabla\varphi(\vec a)\rVert$, and $\varphi$ must be nonincreasing. For other values of $\theta$, the directional derivative will be some value in between these extremes. Going back to our surface in $\mathbb R^3$ and assuming that the gradient doesn’t vanish (in which case $\vec a$ is a stationary/critical point of $\varphi$), this means that $\varphi$ is increasing for $0\le\theta\lt\pi/2$ and decreasing for $\pi/2\lt\theta\le\pi$.

Answer 3

You're right, and this can be generalized: $df=\nabla f \cdot \vec{ds}$. This is the total derivative or total differential.

As usual, $\nabla f$ is the direction of greatest change of the scalar $f$.

$\vec{ds}$ is incremental displacement having both magnitude and direction. This is also referred to as the line element. If you normalize $\vec{ds}$, you get the familiar directional derivative.

Further, this principle is independent of coordinate system.

In cartesian coordinates, $\vec{ds}=dx\hat{i}+dy\hat{j}+dz\hat{k}$

In Cylindrical coordinates, $\vec{ds}=dr\hat{r}+rd\theta\hat{\theta}+dz\hat{k}$

We know $df=\frac{\partial f}{\partial x}dx+\frac{\partial f}{\partial y}dy+\frac{\partial f}{\partial z}dz$. Given this, we must have $\nabla f=\frac{\partial f}{\partial x}\hat{i}+\frac{\partial f}{\partial y}\hat{j}+\frac{\partial f}{\partial z}\hat{k}$ to satisfy differential/displacement relationship.

We also have that $df=\frac{\partial f}{\partial r}dr+\frac{\partial f}{\partial \theta} d\theta+\frac{\partial f}{\partial z}dz$. For a dot product to give the expected differential, we must have that the gradient is $\nabla f=\frac{\partial f}{\partial r}\hat{r}+\frac{1}{r}\frac{\partial f}{\partial \theta}\hat{\theta}+\frac{\partial f}{\partial z}\hat{k}$. Note the appearance of of the factor $\frac{1}{r}$.

The representation of the line element in a given coordinate system dictates the representation of the gradient.

The total derivative, closely related to the directional derivative, establishes the coordinate invariant definition of the gradient.

Answer 4

So my question was how to tell whether a multi-variable scalar function is increasing or decreasing along a given direction. And the answer I needed was

"gradient vector of a scalar field always points toward the increasing value of scalar"

At first I found it hard to believe. Lets consider a scalar field $ f(x,y,z)$. Let the partial derivtives of the scalr field at apoint $P$ be $ f_x=-2, f_y=-2, f_z=-2$ . So the gradient vector $ \nabla f=-2\hat i-2\hat j-2\hat k$. It is obvious that the scalar dicrease along x, y & z axis. So how can the scalar function increase along $-2\hat i-2\hat j-2\hat k $. Apparently it can since $-2\hat i$ is pointing opposite to x axis and so are the other partial deerivatives, and by the value of the partial derivatives the scalar is supposed to increase along the direction opposite to the axis. So in conclusion

"gradient vector of a scalar field always points toward the increasing value of scalar.

And to answer the last part the +ve or -ve sign of directional derivative at a point along a given vector indicates the increase or decrease in the value of the scalar along that particular direction. Since if the orientation of the vector with respect to the gradient vector $0\leq\theta<\pi$ then it increases and if $\pi\leq\theta<2\pi$ then it decreases