Global Convergence of a simple Alternating Optimization problem

82 Views Asked by At

Let $\bf A$ be a known $M \times M$ matrix with complex-valued entries. The optimization problem is to find which matrices $\bf P$ and $\bf C$ solve

$$ \min | | {\bf A} - {\bf P C } | |^2,$$

where $\bf C$ is diagonal matrix with complex-valued diagonal entries, ${\bf P}$ is a symmetric matrix (i.e. ${\bf P} = {\bf P}^T$) with complex-valued entries, and the operator $| | \cdot | |$ denotes the Frobenious norm.

Sadly, the optimization problem is not convex over the entire parameter space $\{ {\bf P} , {\bf C } \}$, but the problem is quadratic if either $ {\bf P}$ or ${\bf C } $ is fixed. Thus, alternating optimization boils down to solving least squares problems iteratively. With that, we are sure that the iterations converge as they always decrease the target funtion's value. The problem is proving that this procedure in this setup converges to the global optimum.