Is the Lagrange Multiplier typically the magnitude of the gradient of the objective function in the direction of the constraint function?
Let $f(x,y,z)$ be an objective function and $g(x,y,z)=0$ be a constraint.
$\nabla f $ is the direction of maximum growth of $f$.
$\frac{\nabla f \cdot \nabla g}{\nabla g \cdot \nabla g} \nabla g$ is the vector projection of the gradient of $f$ along the gradient of $g$.
So $\nabla f - \frac{\nabla f \cdot \nabla g}{\nabla g \cdot \nabla g} \nabla g$ is the direction of greatest change of $f$ that leaves $g$ unchanged.
Instead of $df=\nabla f \cdot \vec{ds}=0$ to get an extremum, use $\nabla f - \frac{\nabla f \cdot \nabla g}{\nabla g \cdot \nabla g} \nabla g=0$ to take the constraint into account.
But this is the same criterion for lagrange multipliers, $\nabla f = \lambda \nabla g$. So I'm wondering is it usually the case that $\lambda =\frac{\nabla f \cdot \nabla g}{\nabla g \cdot \nabla g}$?
I've always thought of $\lambda$ as a constant in the past.
Yes, indeed. Nice observation. From $\nabla f = \lambda \nabla g$, taking the scalar product with $\nabla g$ follows the equation for $\lambda$ which you wrote.