Suppose that $X$ is an arbitrary random variable, is the following is true for any function $f$: $$\underset{y\in \mathcal Y} \sup \mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]?$$
If $f$ is convex in $X$, then the inequality clearly holds, since the supremum of a family of convex functions is still convex. If $f$ is not convex in $X$, I think the inequality still holds for the following reason:
For any realization of $X$ and any value of $y$, we have $f(X,y) \le \underset{y\in \mathcal Y} \sup f(X,y)$. Therefore, for any $y$, $\mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$. In other words, $\mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$ is an upper bound of the set $\left\{\mathbb E\big[f(X,y)\big]: y\in \mathcal Y\right\}$, so it follows that $\underset{y\in \mathcal Y} \sup \mathbb E\big[f(X,y)\big] \le \mathbb E\big[\underset{y\in \mathcal Y} \sup f(X,y)\big]$.
So it appears that convexity of $f$is not needed at all for the inequality to hold. Am I mistaken somewhere? I'd appreciate it if someone would correct me, if I missed something. Thanks a lot!
Edit: sorry - I totally missed you already had the proof in the question - it's correct!
We can equivalently think of this as having a function $f_y$ for each $y$. Then what is always the case is that for each $y$ we have $\sup_y f_y(x)\geq f_y(x)$ for each $x$, and taking the expectation over $X$ this gives $$ \mathbb{E}\left[\sup_y f_y(X)\right]\geq \mathbb{E}\left[f_y(X)\right]$$ Now take the sup over the right side to get the inequality we wanted.