Why is the maximum increase given by $||\nabla f(x, y)||$?

Question

I understand the steps of the proof in the book, but I don't see intuitively the of maximum increase at a point $P$ must be given by the $||\nabla f(x, y)||$. A graph has infinite directional derivatives at point $P$, I just don't see what is special about the sum of the directional derivatives in the $x$ direction and in the $y$ direction. If at a point $P$ the absolute value of the gradient is equal to the maximum increase, can't we just rotate the graph any amount around point $P$, the same maximum increase but a different gradient?

P.S. Please try to avoid using many advanced logic symbols.

Answer 1

The directional derivative is the dot product between the gradient and the unit vector along the given direction. That dot product attains its maximum value when the two vectors are parallel. Draw yourself the conclusion.

Answer 2

Consider the derivative of f(x, y), at $(x_0, y_0)$, in the direction of vector making angle $\theta$ with the x-axis, that is, in the direction of the unit vector $cos(\theta)\vec{i}+ sin(\theta)\vec{j}$. Writing $x= x_0+ cos(\theta)$, $y= y_0+ sin(\theta)$, by the chain rule, the derivative in that direction is $\frac{\partial f}{\partial x} cos(\theta)+ \frac{\partial f}{\partial y} sin(\theta)$.

To find the maximum (as well as minimum) of that, differentiate with respect to $\theta$ and set the derivative equal to 0: $-\frac{\partial f}{\partial x} sin(\theta)+ \frac{\partial f}{\partial y}cos(\theta)= 0$

$\frac{\partial f}{\partial y}cos(\theta)= \frac{\partial f}{\partial x}sin( \theta)$

$\frac{\frac{\partial f}{\partial y}}{\frac{\partial f}{\partial x}}= \frac{sin(\theta)}{cos(\theta)}= tan(\theta)$

That is, the function has maximum increase in the direction such that the tangent of the angle is the x-component of the gradient vector over the y-compenent which is precisely the direction in which the gradient points.

The function has minimum increase, of course, in the opposite direction.

Answer 3

Due to Cauchy-Schwartz inequality and the fact that $D_u f=\langle \nabla f, u\rangle$. Hence,

$$D_u f=\langle \nabla f,u \rangle \leq \|\nabla f\| \cdot \| u\| \leq \| \nabla f \| $$

for all $u$. But for $u=\frac{\nabla f}{\| \nabla f\|}$, we get:

$$D_u f= \langle \nabla f,\frac{\nabla f}{\| \nabla f\|} \rangle=\frac{1}{\|\nabla f\|}\langle \nabla f,\nabla f \rangle=\frac{1}{\|\nabla f\|}\|\nabla f \| ^2=\| \nabla f\|.$$