I have been trying to understand how Lagrange multipliers and the Lagrangian work. The Lagrangian is defined as $$\mathcal{L} = F\left(\mathbf{x}\right) + \mathbf{\lambda}^T\mathbf{g}\left(\mathbf{x}\right)$$
and can also be defined as $$\mathcal{L} = F\left(\mathbf{x}\right) - \mathbf{\lambda}^T\mathbf{g}\left(\mathbf{x}\right)$$
Why add that when $\mathbf{g}\left(\mathbf{x}\right) = \mathbf{0}$ for the equality constraints? Also, I stumbled on some source saying that the sign in front of $\lambda$ matters when it comes down to more complex optimization (i.e. the Lagrangian is defined as a sum of those 2 terms or a diference of those 2 terms). Can one explain how it is so for both questions?
Quote: "Why add that when g(x)=0 for the equality constraints? "
The full expression for Lagrangian is $$\mathfrak{L}=\mathbf{F}\left ( \mathbf{x} \right )-\lambda ^{T}\left ( \mathbf{G\left ( \mathbf{x} \right )-k} \right )$$ where $\mathbf{G\left ( x \right )}=k$ is the equality constraint. Hence, $\mathbf{g\left ( x \right )}=\mathbf{G\left ( x \right )-k}=0$
Regarding the sign in front of the Lagrange Multiplier, if you do $\mathbf{G\left ( x \right )-k}$, the sign is $-$. If you do $\mathbf{k-G\left ( x \right )}$, the sign is $+$. This is to ensure that $\frac{\partial \mathfrak{L}}{\partial \mathbf{x}}=\frac{\partial \mathbf{F}}{\partial \mathbf{x}}-\lambda \frac{\partial \mathbf{G}}{\partial \mathbf{x}}$.
If the constraint is an inequality, i.e., $\mathbf{G\left ( x \right )}< \mathbf{k}$, it is preferable that the smaller quantity $\mathbf{G\left ( x \right )}$ is subtracted from the larger quantity $\mathbf{k}$ and form the Lagrangian as $\mathfrak{L}=\mathbf{F\left ( x \right )}+\lambda ^{T}\left ( \mathbf{k}-\mathbf{G\left ( x \right )} \right )$