Suppose we have the bivariate function $f(x,y)$. I want to solve the following problem:
\begin{equation} \min\limits_{(x,y)} \; \; f(x,y) \end{equation}
I want to prove theoretically that simultaneous optimization over $(x,y)$ outperforms the sequential optimization approach where the function is optimized over $y$ for a fixed $x$ and then $y$ is fixed at the obtained solution to solve the optimization over $x$ this time and this procedure goes iteratively until the it converges to some solution $(x_{seq}^*, y_{seq}^*)$. Suppose $(x_{sim}^*,y_{sim}^*)$ is the solution obtained from simultaneous optimization. I need to show that:
\begin{equation} f(x_{seq}^*, y_{seq}^*) \; \ge \; f(x_{sim}^*,y_{sim}^*) \end{equation}
that is, the optimum achieved by sequential approach is always suboptimal compared with the solution obtained from simultaneous approach.
I appreciate any insights on this. The general problem that I have is non-convex, however I am also interested in the proofs for biconvex case where $f$ is convex in $x$ for each $y$ and convex in $y$ for each $x$.