The context here is a zero-sum, finite game (i.e. $g$ is the gain to one player, bounded above and below, $\alpha$ and $\beta$ are finite dimensional probability vectors).
Say we have that $\sup_{\alpha} \inf_{\beta} g(\alpha, \beta) = g(\alpha^*, \beta^*) $ (1).
From my understanding, this means: fix $\alpha$, then find the infimum of $g$ over $\beta$ for that $\alpha$ (say, $g_{\alpha}$). Now let $\alpha$ be a free parameter again and find the supremum of that infimum over $\alpha$.
Does (1) mean that:
i) $\sup_{\alpha} \inf_{\beta} g(\alpha, \beta) = \inf_{\beta} g(\alpha^*, \beta)= g(\alpha^*, \beta^*) $
OR
ii) $\sup_{\alpha} \inf_{\beta} g(\alpha, \beta) = \sup_{\alpha} g(\alpha, \beta^*)= g(\alpha^*, \beta^*) $
(Or both? Or neither?)
I'm sorry - I know this is probably an easy question. I'm just struggling to get my head around the concept of taking multiple sups/infs because in my game theory classes the above statements (i and ii) seem to be chucked around interchangeably and I'm not sure whether that's only because we're usually working in a framework where we use the Minimax Theorem, i.e. we also have that $ \inf_{\beta} \sup_{\alpha} g(\alpha, \beta) = g(\alpha^*, \beta^*) $ as $\alpha^*, \beta^*$ are maximin and minimax strategies (and then in which case, if one of the above is true then both will be true by an analogous argument). This is of course always the case in a finite game, so I guess I'm asking about the above in a more general framework.
My thinking is that, after fixing $\alpha$ and taking the infimum over $\beta$ you get $g_{\alpha}$, a quantity independent of $\beta$ given $\alpha$. So then the 'last step' you do is one where $\alpha$ is the free parameter and $\beta$ is fixed, and so ii makes more sense. But at the same time, the $\beta$ which satisfies the infimum condition is only fixed given $\alpha$, and so I don't really feel entirely comfortable with saying 'at the last step $\beta$ is fixed'.
I think I have a misunderstanding about what it means to take the sup of an inf (or vice versa).
It's neither.
Assume 2 dimensions: $\alpha=(\alpha_1, \alpha_2), \beta=(\beta_1, \beta_2)$ and $g(\alpha,\beta) = \alpha_1 + \beta_1$.
Then $$\inf_\beta{g(\alpha,\beta)} = \alpha_1 + 0 = \alpha_1,$$ hence $$ \sup_\alpha{\inf_\beta{g(\alpha,\beta)}} = 1 = g(\alpha^\star,\beta^\star),$$ with $\alpha^\star=\beta^\star=(\frac12,\frac12).$
OTOH, $\inf_\beta{g(\alpha^\star,\beta)} = \frac12$ and $\sup_\alpha{g(\alpha,\beta^\star)} = \frac32$.