I've heard from a reuptable source that it is problematic to differentiate the Lagrangian w.r.t the lagrange multiplier.
I know that doing so is rather a waste of time since it just goves you back the constraints $h(x)=A$ that you started with, but this repitable source said that there is an additional reason why it is not only a waste of time but also problematic?
Any hints as to why it is problematic?
It is only a matter of definitions. Let us consider the problem of optimizing the function $F\colon X\to \mathbb R$ on the constraint that $G\colon X \to \mathbb R$ satisfies $G(x)=0$. (Here $X$ is a differentiable manifold, or a Banach space. The point is that one must be able to do differential calculus on $X$).
To me, a Lagrange multiplier associated to this problem is a scalar $\mu$ such that $$F'(x_0)=\mu G'(x_0)$$ at a point $x_0\in X$. Somebody else may instead consider the function ("Lagrangian") $$ L(x, \mu)=F(x)-\mu G(x)$$ and say that $\mu$ is simply the second independent variable in $L(x, \mu)$. In the first case it makes no sense to differentiate with respect to $\mu$, as it is only defined at critical points of the optimization problem. In the second case the derivative of $L$ with respect to $\mu$ is simply $-G$.