Consider the following convex optimization problem: $$\text{minimize} ~ \sum_{i=1}^N f_i(x) ~ \text{ s.t. } ~ x \in X,$$ where $X$ is an $m$-dimensional convex set, and $f_i$'s are convex. We can rewrite this as: $$\text{minimize} ~ \sum_{i=1}^N f_i(x_i) ~ \text{ s.t. } ~ x_i \in X ~ \text{ and } ~ x_i = z ~\forall i.$$ This is called the global consensus problem (refer to Chapter 7 of Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers). The partial augmented Lagrangian for this problem is: $$L_\rho(x, z, y) = \sum_{i=1}^N\left( f_i(x_i) + y_i^T (x_i - z) + \rho/2 \Vert x_i - z \Vert_2^2 \right).$$ Let $(x^*, z^*, y^*)$ be the optimal solution. We know that $x_i^* = z^*$ for all $i$. Here are my questions:
- Do we have $y_i^* = 0_m$?
- If not, can we show that $\sum_i y_i^* = 0_m$? Also, what can we say about $\Vert y_i^* \Vert_2$ if $X = \{ x \in [0, 1]^m : \Vert x \Vert_1 \le 1\}$?