I have a question, I tried to google it, but I am not sure yet.
The question is, can we use the Lagrange multipliers in iterative update instead of stochastic gradient descent in order to optimize any function?
For example, I used to work on optimizing such function in iterative way using the SGD in order to optimize such parameters, and the optimization is working well. So I decided to extend that function to add a constraint for that function, is it possible in all ways to use Lagrange multipliers instead of SGD as long as we have constraints? Then update the function in iterative way.
if you have easy documents which explain that point, it will be appreciated to help me.
thank you so much in advance.