How $f^*(y) := \sup_x \left\{ \langle y, x \rangle - f(x) \right\} \stackrel{\equiv}{?} -\inf_x \left\{ f(x) - \langle y, x \rangle \right\}$

Question

I am sorry for asking stupid question perhaps.

The defintion of convex conjugate, for instance, in Wikipedia is, $$f^*(y) := \sup_x \left\{ \langle y, x \rangle - f(x) \right\} \stackrel{\equiv}{?} -\inf_x \left\{ f(x) - \langle y, x \rangle \right\}.$$

I am so confused with the equivalence between supremum and infimum in this definition, and will appreciate your help in the clarification.

My confusion is like this

$$\max_x g(x) \equiv \min_x -g(x).$$

If the above is true, then same should hold for supremum and infimum, that is, $$\sup_x g(x) \equiv \inf_x -g(x).$$

If this is true, then why convex conjugate definition has extra "negative"? I am sorry again for asking stupid question and apologies for that as I am learning.

Answer 1

Let $A$ be a bounded non empty set. Then $\operatorname{sup}(-A)=-\operatorname{inf}(A)$.

Indeed, let $x \in A$, then $x \geq \inf(A)$. Thus, $-x \leq -\inf(A)$. Therefore, $-\inf(A)$ is a upper bound of the set $-A$. Moreover, given $\varepsilon>0$, there exists $x_0 \in A$ such that $x_0<\inf(A)+\varepsilon$. Thus, $-x_0> -\inf(A)-\varepsilon$, which proves that $-\inf(A)$ is the supremum of $-A$.

Then, $\sup\{\langle y,x\rangle-f(x)\}=\sup\{-(f(x)-\langle y,x\rangle)\}=-\inf\{f(x)-\langle y,x\rangle\}$.