If $f$ is convex in $x$ and $z$, how is $\sup_x(x^Ty - \inf_z f(x,z)) = \sup_{x,z}(x^Ty - f(x,z))$?

56 Views Asked by At

From Convex Optimization by Boyd and Vanderberghe:

If $f$ is convex in $x$ and $z$, then $$\sup_x(x^Ty - \inf_z f(x,z)) = \sup_{x,z}(x^Ty - f(x,z))$$

How are these equal? It seems like they would not be equal as the smallest $z$ may be different from the largest $z$.

1

There are 1 best solutions below

0
On BEST ANSWER

I believe it is not related to convexity at all, but to the elementary property stating that $$\sup_{x, y} \phi(x, y) = \sup_x \sup_y \phi(x, y) = \sup_y \sup_x \phi(x, y).$$ In your case, using the fact that $\inf_x h(x) = -\sup{-h(x)}$, we obtain: $$ \sup_x \{ x^T y - \inf_z f(x, z) \} = \sup_x \{ x^T y + \sup_z \{ -f(x, z) \} \} = \sup_x \sup_z \{ x^T y - f(x, z) \} = \sup_{x, z} \{x^T y - f(x, z) \} $$