Inner product term in Augmented Lagrangian

87 Views Asked by At

I am confused about the inner product term in the ALM.

For example, $$ \begin{aligned} &\min_{\boldsymbol{x}, \boldsymbol{z}} \quad f_{1}(\boldsymbol{x})+f_{2}(\boldsymbol{z})\\ &\text { s.t. } \quad A x+B z=b \end{aligned} $$

is equivalent to $$ \operatorname{max}_{\lambda} \min_{x, z} f_{1}(x)+f_{2}(z)+ \langle A x+B z-b, \lambda\rangle+\frac{\rho}{2}\left\|A x+B z-b \right\|_{2}^{2}, $$

where $\lambda$ is the Lagrangian multiplier and $\rho$ is the penalty term.

I have read some papers which 'simplify' the above formula into $$ \operatorname{max}_{\lambda} \min_{x, z} f_{1}(x)+f_{2}(z)+ \frac{\rho}{2}\left\|A x+B z-b + \frac{1}{\rho} \lambda\right\|_{2}^{2}. $$

I am very confused why the inner product term can be merged into the quadratic term since this implies the $\frac{1}{\rho} \| \lambda\|_2^2$ is pushed to zero.

Can anyone tell me the equivalence of two formula? What is the initution behind it? Thanks in advance!