I have some questions regarding the Minimax theorem:
the Max–min inequality says: $\sup_{z \in Z} \inf_{w \in W} f(z,w) \leq \inf_{w \in W} \sup_{z \in Z} f(z, w)$ Now, I know the term of Infimum of a set $\ A$ as the greatest element $\ m $ s.t. each element in $\ A$ is bigger or equal to $\ m$ (and a similar definition for the supremum). But when I see $\ \inf_{w \in W} f(z,w)$ I'm not sure what it means since $\ f(z,w)$ it is not a set. it is a function, and it depends on z and w. so How what the term $\ \inf_{w \in W}$ means in that case?
Thanks!
It might help to use different labels.
Pick points $a \in Z$ and $b \in W$. Then $$\inf_{w \in W} f(a,w) \le f(a,b) \le \sup_{z \in Z} f(z,b).$$
The left hand side is completely independent of $b$, so it forms a lower bound of all possible expressions on the right. It cannot exceed the greatest lower bound, so regardless of the choice of $a$ you get $$\inf_{w \in W} f(a,w) \le \inf_{b \in W} \sup_{z \in Z} f(z,b).$$ Likewise, the expression on the right is independent of $a$ and forms an upper bound for all possible expressions on the left. Thus $$ \sup_{a \in Z} \inf_{w \in W} f(a,w) \le \inf_{b \in W} \sup_{z \in Z} f(z,b).$$
$a$ and $b$ are dummy indices in this last expression and can be replaced by any other letters, including $z$ and $w$.