Minimisation and expectation inequality

Question

Let $g(\boldsymbol{\theta},\boldsymbol{X})$ be a continuous function mapping a nonstochastic parameter $\boldsymbol{\theta} \in \mathbb{R}^K$ and a random variable $\boldsymbol{X}$ to $\mathbb{R}$. Is it the case that $$ \min_{\boldsymbol{\theta}} \mathbb{E}\left[ g(\boldsymbol{\theta},\boldsymbol{X})\right] \leq \mathbb{E}\left[\min_{\boldsymbol{\theta}} g(\boldsymbol{\theta},\boldsymbol{X})\right]?$$

Answer 1

First of all, it would probably be better to use $\inf$ rather than $\min$ here, unless you know for certain that for each $x$, $g(\cdot, x)$ achieves a minimum.

Now we note that for any $\theta'\in\mathbb{R}^K, x\in\mathbb{R},$ we naturally have $$\min_\theta g(\theta, x)\leq g(\theta', x).$$ In particular, we then have $$\mathbb{E}[\min_\theta g(\theta, X)]\leq \mathbb{E}[g(\theta', X)].$$ Since the inequality holds for all $\theta'$, we can take the minimum over $\theta'$ to obtain $$\mathbb{E}[\min_\theta g(\theta, X)]\leq \min_{\theta'}\mathbb{E}[g(\theta', X)].$$ That is, the inequality holds the other way.

(In the above it would in general be better to use $\inf$ rather than $\min$)