Why is the gradient of a function at a local extreme value orthogonal to the line tangent to the constraint curve?

Question

Visually and geometrically, I can see how this makes sense, but is there a more formal explanation of why this is true? With only an understanding of the geometric picture, I don't understand why the Lagrange multiplier method doesn't always provide all of points at which the function has an absolute maximum or absolute minimum, specifically in the case when the constraint curve closes in on itself.