Optimizing a function of argmax

Question

The function $f(\mathbf{x})$ is defined as follows:

$$f(\mathbf{x}) = \arg \max_y g(\mathbf{x}, y)$$

In other words, to evaluate the function $f$ for a given $\mathbf{x}$, we need to solve an optimization problem. The function $g(\mathbf{x},y)$ is concave in $y$ for a fixed value of $\mathbf{x}$, but there is no closed-form expression available for the solution of this concave problem. Furthermore, $g(\mathbf{x},y)$ is not concave in $\mathbf{x}$ for a fixed $y$.

My question is this: I wish to optimize the function $f(\mathbf{x})$; how would I do this? For instance, is it correct to state that $$\mathrm{arg.}\underset{x}{\mathrm{max}}f(\mathbf{x})=\mathrm{arg.}\underset{x,y}{\mathrm{max}}\ g(\mathbf{x},y)$$

So in that case would maximizing $f(\mathbf{x})$ be equivalent to the following optimization problem:

$$\mathrm{\underset{\mathbf{x},y}{maximize}}\quad g(\mathbf{x},y)$$

Any help would be greatly appreciated!

Answer 1

Under the assumptions of smoothness... The equality is not always verified.

If you consider a function of two variables and you optimize wrt each variable sequencelly, then the solution may be different from the solution when considering the two variables at the same time.