Proof of Lagrange Multipliers with Multiple Constraints

Question

I've found the following explanation for the Lagrange multipliers method with multiple constraints on Wikipedia. Attached. In $\displaystyle \nabla f(\mathbf {x} )\in A^{\perp }=S$, I do not understand why $A^{\perp }=S$. I agree that a vector in $S$ is also in $A^{\perp }$ by definition, but I cannot see why a vector in $A^{\perp }$ must necessarily be part of $S$. What is the explanation for this?

Answer 1

Intuitively speaking, you need to move in a direction perpendicular to the gradients of all the constraint functions because the directional derivative of a given constraint function is given by a vector in the direction you're moving dotted with the gradient of that constraint function. So if the direction you move is not perpendicular to one of the gradients then that constraint function will change, causing you to move off the constraint surface.

In actuality the situation is slightly more complicated for finite displacements. I can explain why this fact doesn't change the Lagrange criterion if you want, but this is getting into technicalities, not intuition anymore.

This tells you what the admissible directions are, they are the directions perpendicular to all the gradients of the constraint functions. The Lagrange criterion comes about from requiring that $\nabla f$ is perpendicular to all the admissible directions, so that the directional derivative of $f$ in any admissible direction is zero. In other words $\nabla f$ needs to be in the orthogonal complement of the admissible directions $A^\perp$. But $A$ itself was the orthogonal complement of the span of the gradients of the constraint functions, call that span $S$. So $\nabla f \in (S^\perp)^\perp=S$, i.e. the span of the gradients of the constraint functions.