I have an intuitive understanding of why the simple method of coordinate descent does not work with linearly coupled constraints such as;
$$\min_x\sum_if_i(x_i)$$ $$s.t.$$ $$Ax=b$$
If we try to minimize the above problem with respect to one variable at a time, it doesn't converge to the optimal solution, even though $f$ is convex. How can we more formally explain the reason?
Also, are there any iterative methods to solve the above problem, with additional inequality constraints such as:
$$\min_x\sum_if_i(x_i)$$ $$s.t.$$ $$Ax=b$$ $$Gx\le H$$
I have read about block coordinate descent methods proposed for linearly coupled constraints, but I have a more complex model in hand with the inequality constraints. Thank you.