What do you call iteratively optimizing w.r.t. various groups of variables?

Question

Suppose $f(\vec x, \vec y)$ is a function of two vectors of variables such that there is no analytical solution for

$$\arg\min_{(\vec x, \vec y)} f(\cdot, \cdot) $$

However, suppose that

$$\arg\min_{\vec x} f(\cdot | \vec y) $$

and

$$\arg\min_{\vec y} f(\vec x | \cdot) $$

are each individually tractable and that iteratively solving these two subproblems generates a sequence of approximations that converges to the solution of the original problem.

Is there a general name for such an iterative algorithm? Batch minimization?

Answer 1

This approach is called Block Coordinate Descent. Simple coordinate descent methods work with a single variable at a time- in block coordinate descent the optimization is done with groups of variables. It is not obvious in general that block coordinate descent will converge to an optimal solution. This has been the topic of a lot of research in recent years.