Consider the example below. Here, We found that $\nabla g$ is only zero at $(0, 0)$ but we see that the point $(0,0)$ does not satisfy $g(x,y) = 0$. But I don't understand why we need to ensure that that the $\nabla g \ne 0$. What would it mean if $\nabla g = 0$ satisfies the constraint? What would I need to do in this case? How would my steps change? Also, later they mention that $\lambda$ is not equal to $0$ since it would lead to a contradiction if it did. Once again, I don't understand why we need to make sure that $\lambda \ne 0$. Why do we need to make that step? What happens if $\lambda$ can be zero? What would that mean? What measures should I take if that happens?
In brief, I'm asking what do I do if $\nabla g = 0$ satisfies the constraint or $\lambda = 0$.
Great question! Consider the constraint $g(x,y) = y^3-x^2$. Note that $\nabla g(0,0)=(0,0)$. Now consider the function $f(x,y) = y$. Your task is to minimize $f(x,y)$ subject to the constraint $g(x,y)=0$. Do you succeed with Lagrange multipliers?
Because the constraint curve is not smooth at the minimum point, the method fails.
So far as the vanishing of $\lambda$ is concerned, I would like to see exactly what your text said there. If $\nabla g$ is nonzero everywherew on the constraint curve, you certainly can have $\lambda = 0$. Here's an example: Consider the constraint $g(x,y) = x^2 + y^2 = 1$. Note that $\nabla g(x,y)\ne (0,0)$ for every point of the constraint curve. Let $f(x,y) = x^2 + y^2 - 2x$. Then $f(x,y)$ has a global minimum at $(1,0)$, which happens to lie on the constraint curve. So $\nabla f(1,0) = (0,0)$, and the only way we can have $\nabla f(1,0) = \lambda \nabla g(1,0)$ is to have $\lambda = 0$.