In what sense does a gradient point in the direction of maximum increase?

Question

Maybe an example will help. $F(x,y)= x^2 + y^2$. Now the gradient would look like $<2x , 2y > or 2xi + 2yj$ where $i$ and $j$ are unit vectors of the $x$ and $y$ axis. This vector field spans every quadrant. The gradient is sort of a place holder to build a vector field.

Certainly it does not point in one direction so in what sense is it pointing in the maximum direction? The maximum direction of what?

If you don't mind a little extra question on the side, but relevant, since it was the source of my original confusion. Is directional derivative is a dot product with the gradient and a unit vector pointing in any direction you choose but if the angle between the gradient and the unit vector is 0 , the cosine is 1 so that means that unit vector was pointing in the direction of the gradient ?

Since the directional derivative is a number only of what utility is it other than this observation ? I may be missing the boat on directional derivative.

Answer 1

Suppose you are climbing a mountain. At any point you have different choices of directions. If you are tired you probably choose a not so steep direction to climb. If you have lots of energy and you want to get to the top you choose a direction of maximum ascent. That is the direction of the gradient vector at that point. The opposite direction would be the steepest decent direction which is negative gradient at that point.

Answer 2

When we say that the gradient points in the direction of maximum increase, we don't mean the whole vector field, we mean one vector, at one particular point $(x,y)$, points in the direction you should move from that point to achieve fastest increase of the function.

By definition the gradient is a vector that represents a linear map, and it is this linear map only that is called the "differential" of the function (a multivariate word for "derivative"). The directional derivative is just the evaluation of that map on a given vector. Since the gradient represents the map, by definition the evaluation is the dot product with the gradient.

In a way, a directional derivative is the derivative of your initial function, if you artificially take down to $1$ the degree of freedom of its variable.