I need to solve the following nonconvex optimization problem: \begin{equation} \begin{split} \min_{x,y}\quad &f(x)+g(y)\\ \mathrm{s.t.}\quad &Ax+By=b \end{split} \end{equation} where $f$ is noncovex and $g$ is convex. A natural way is to use ADMM to solve this problem, which can be outlined as follows:
Define the augmented Lagrangian as $$\mathcal{L}_{\beta}(x,y;\omega)=f(x)+g(y)+w^{T}(Ax+By-b)+\frac{\beta}{2}||Ax+By-b||_2^2,$$ then we could use ADMM directly by solving the following subproblems:
\begin{equation} \begin{aligned} x^{k+1}:&=\arg\min_{x} \mathcal{L}_{\beta}(x,y^k;\omega^k), \\ y^{k+1}:&=\arg\min_{y} \mathcal{L}_{\beta}(x^{k+1},y;\omega^k), \\ \omega^{k+1}:&=\omega^{k}+\beta(Ax^{k+1}+By^{k+1}-b). \\ \end{aligned} \end{equation}
As we know, ADMM works for convex optimization problem with the guarantee of global convergence, but for this nonconvex problem, what's the convergence behavior?
In general, the convergence behavior can be arbitrarily bad. But, it all depends on the structure of $f(x)$. If you can find nice convex envelopes of the $f(x)$ you can get numerical bounds on the convergence. E.g., if $f(x)$ is bilinear, like $f(x)=x_1 x_2$. McCormick's relaxations provide envelopes https://optimization.mccormick.northwestern.edu/index.php/McCormick_envelopes
I would recommend finding convex envelopes to $f(x)$. Solving the relaxations like you would solve convex problems. Then evaluating the actual objective function at feasible points close to the solution of the enveloped functions.