how can update parameter in each iteration?

116 Views Asked by At

In an augmented Lagrangian with inequality constraint

\begin{align} &\min f(\mathbf{x}) \\ &\text{subject to} \\ &c_i(\mathbf{x}) = 0, \quad \forall i \in I \end{align}

The augmented Lagrangian method uses the following unconstrained objective

$$ \min \Phi_k (\mathbf{x}) = f (\mathbf{x}) + \frac{\mu_k}{2} ~ \sum_{i\in I} ~ c_i(\mathbf{x})^2 - \sum_{i\in I} ~ \lambda_i c_i(\mathbf{x})$$

and after each iteration, in addition to updating $\mu_k$, the variable $\lambda</$ is also updated according to the rule $\lambda_i \leftarrow \lambda_i - \mu_k c_i(\hat {x}_k) $ where $\hat {x}_k$ is the solution to the unconstrained problem at the $k$th step, i.e. $\hat{x}_k=\text{argmin} \Phi_k(\mathbf{x}) $

If the method can be handled using inequality constraints, how $\lambda_i$ is updated?

In other words, for the below problem

\begin{align} &\min f(\mathbf{x}) \\ &\text{subject to} \\ &c_i(\mathbf{x}) \le 0, \quad \forall i \in I \end{align}

how can we update $\lambda_i$ in each iteration of augmented Lagrangian method?

1

There are 1 best solutions below

1
On BEST ANSWER

We can convert it to a problem with equality constraints and bound constraints by introducing slack variables $s_i$ and replacing the inequalities

$$c_i(\textbf{x}) \ge 0, \quad i \in I$$

by

$$c_i(\textbf{x}) − s_i = 0, \quad s_i \ge 0 \quad \forall i \in I$$

This transformation gives rise to a problem containing equality constraints and $\lambda = \max(\lambda_i − c_i(x)/\mu, 0) \ \ \forall i \in I.$