Does ADMM work for nonconvex optimization problems?

252 Views Asked by At

I need to solve the following nonconvex optimization problem: \begin{equation} \begin{split} \min_{x,y}\quad &f(x)+g(y)\\ \mathrm{s.t.}\quad &Ax+By=b \end{split} \end{equation} where $f$ is noncovex and $g$ is convex. A natural way is to use ADMM to solve this problem, which can be outlined as follows:

Define the augmented Lagrangian as $$\mathcal{L}_{\beta}(x,y;\omega)=f(x)+g(y)+w^{T}(Ax+By-b)+\frac{\beta}{2}||Ax+By-b||_2^2,$$ then we could use ADMM directly by solving the following subproblems:

\begin{equation} \begin{aligned} x^{k+1}:&=\arg\min_{x} \mathcal{L}_{\beta}(x,y^k;\omega^k), \\ y^{k+1}:&=\arg\min_{y} \mathcal{L}_{\beta}(x^{k+1},y;\omega^k), \\ \omega^{k+1}:&=\omega^{k}+\beta(Ax^{k+1}+By^{k+1}-b). \\ \end{aligned} \end{equation}

As we know, ADMM works for convex optimization problem with the guarantee of global convergence, but for this nonconvex problem, what's the convergence behavior?

1

There are 1 best solutions below

0
On

In general, the convergence behavior can be arbitrarily bad. But, it all depends on the structure of $f(x)$. If you can find nice convex envelopes of the $f(x)$ you can get numerical bounds on the convergence. E.g., if $f(x)$ is bilinear, like $f(x)=x_1 x_2$. McCormick's relaxations provide envelopes https://optimization.mccormick.northwestern.edu/index.php/McCormick_envelopes

I would recommend finding convex envelopes to $f(x)$. Solving the relaxations like you would solve convex problems. Then evaluating the actual objective function at feasible points close to the solution of the enveloped functions.