Understanding the conjugate of a function

Question

I am bad at math and am having an extremely hard time trying to understand the conjugate of a function. I am not sure why the dotted line is the conjugate function. How do I find that? This is my limited understanding of the steps.

1) draw $y^Tx$ which is equals to $x y$ in 2 dimensions. Question: is $x$ my gradient or is $y$ my gradient? Because the original equation is $f(x)$ but now is $f^* (y)$?

2) Maximize the function $x y - f(x)$ with respect to $f$ because I am to find the supremum of the expression. How do I draw this?

So I only know how to draw the line $x y$. Then I am stuck. It would be nice if someone could give a numerical example by stepping through some x or y values ( i am not sure if i am supposed to step through the x or y values). I can only understand after seeing an example with numbers.

Question: Why do I shift the line downwards (blue gap) ? The red gap looks bigger to me and we are finding the maximum.

Answer 1

The dotted line is not the conjugate itself. It is a method to compute $f^*(y)$ for a given $y$. For that given value of $y$ (say $y=3$) you want to solve $\sup_x \{ 3x - f(x) \}$. The supremum is attained where the derivative of $3x - f(x)$ is $0$, i.e., $f'(x) = 3$. By moving a dotted line with slope $3$ up or down until it touches the function, you find such $x$ (a horizontal line would find $x$ such that $f'(x)=0$).

Answer 2

If $f(x) = x^2$, then $f^*(y) = \sup_x xy-x^2$. We see that the $\sup$ over $x$ is attained when $y-2x = 0$, so we have $f^*(y) = {y^2 \over 4}$.

If $f(x) = c$, a constant, then $f^*(y) = \sup_x xy-c$. We see that this is $+\infty$ whenever $y \neq 0$, and is $-c$ for $y = 0$. Hence $f^*(y) = I_{\{0\}}(y)- c$.