The gradient is everywhere perpendicular to the contour lines of a function

Question

In this article, there's something saying:

The gradient is everywhere perpendicular to the contour lines of a function

It justifies it by saying:

Since along contour lines the change in height is zero, this means the directional derivative along the contour is zero

But what is exactly a directional derivative along a contour? I only know about directional derivatives in directions of vectors.

I'm asking this because I'm studying Lagrange multipliers, and my book justifies it geometrically only. It seeks to find the max or min of a function $f$ with the restriction on the domain $\{(x,y) | g(x,y) = 0\}$ It says that when $(x_0,y_0)$ is a critical point (either max or min), it means that if we trace the contour lines near this point, we'll observe a contour line of $f$ that is 'tangent' to the restriction $g$. Here's the image:

I can't see why this is true geometrically, could somebody give me a reason? I want a proof but I also want to understand it geometrically.

My book then ends by saying that the gradients of $f$ and $g$ must be perpendicular. But I can't see this.

Answer 1

But what is exactly a directional derivative along a contour? I only know about directional derivatives in directions of vectors.

Say you have a smooth curve $\gamma(t)$. Then the directional derivative along the contour should be $$ (f\circ \gamma)'(t) = \frac{d}{dt} f(\gamma(t)) = \lim_{h\to 0} \frac{f(\gamma(t+h)) - f(\gamma(t))}h.$$

---

Proposition. For each $t$, $\frac{d}{dt} f(\gamma(t)) $ is equal to the directional derivative of $f$ at $\gamma(t)$ in the direction $\gamma'(t)$.

---

Proof. Let $t$ be fixed. Recall that the directional derivative of $f$ at $\gamma(t)$ in the direction $\gamma'(t)$ is $$ (D_{\gamma'(t)}f)(\gamma(t)) = \lim_{h\to 0} \frac{f(\gamma(t)+ \gamma'(t)h) - f(\gamma(t))}h = \gamma'(t)\cdot (\nabla f)(\gamma(t)). $$ But by the multivariate chain rule, this is precisely $$(f\circ \gamma)'(t)= \nabla f(\gamma(t))\cdot \gamma'(t) $$ hence the result. $\square$

The claim

The gradient is everywhere perpendicular to the contour lines of a function.

then follows: on a contour line, $f\circ \gamma$ is constant. Thus for any $t,h$, $$ f(\gamma(t+h)) - f(\gamma(t)) \equiv 0.$$ Thus $$ \lim_{h\to 0} \frac{f(\gamma(t+h)) - f(\gamma(t))}h = 0.$$ By the proposition: $$ \nabla f(\gamma(t))\cdot \gamma'(t) = 0$$ which is to say that $\nabla f(\gamma(t))$ is perpendicular to the tangent vector $\gamma'(t)$ of the contour, which is the claim.

(In $n>1$ dimensions, you would do this for $n$ curves whose tangent vectors form a basis of the tangent space.)

Answer 2

Let an isocurve be described by the parametric equation $x=x(t),y=y(t)$.

We have

$$f(x(t),y(t))=C\implies \frac{df}{dt}=\frac{\partial f}{\partial x}\dot x+\frac{\partial f}{\partial y}\dot y=\nabla f\cdot\vec t=0$$where $\vec t=(\dot x,\dot y)$ is a tangent vector.

Answer 3

Suppose we have some function $F(x,y,z): \mathbb R^3\to \mathbb R$

$\nabla F$ will point in the direction of the greatest change in $F.$

Now consider the surface created by $F(x,y,z) = K$

$\nabla F$ doesn't change. But, the surface is not allowed the freedom to "move" in this direction. It can only move perpendicular to this direction. $\nabla F$ normal vector to the plane tangent to the surface at this point. Or $\nabla F$ is always normal to the surface.