Convex conjugate of a function?

Question

The conjugate of a function $f$ is $$f^*(y)=\sup_{x\in \mathop{\rm dom} f} (\left< y,x\right> - f(x)).$$

Let $f(x)=\frac{1}{2}\left< Ax,x\right>+\left< b,x\right>+c$ on open set $\Omega$ of $\mathbb R^n$, where $A$ is a definite positif symmetric matrix. I showed that $f$ is convex function.

Now, I would like compute the conjugate function $f^*(y)$ of $f$?

For this, I will calculate the derivative of the function $$g(x)=\left< y,x\right> - f(x) = \left< y,x\right> -\frac{1}{2}\left< Ax,x\right>-\left< b,x\right>-c$$ with respect to $x$, for find its maximum. So, the derivative of $\left< y,x\right>$ is $y$ and the derivative of $-\left< b,x\right>-c$ is $-b$, but what it the derivative of $\left< Ax,x\right>$ ?

Remark: In a pdf I found that: $f^*(y)= \frac{1}{2} \left< y-b, A^{-1}(y-b)\right>-c$.

Thank you in advance

Answer 1

The derivative of $$ f(x) = \frac12 \langle Ax ,x \rangle + \langle b, x\rangle + c $$ at $x$ is $$ df(x)h = \langle Ax, h\rangle + \langle b,h\rangle $$ and $df(x)$ is the zero linear map if $Ax=-b$. Plugging in, we find a minimum value of $f$ $$ f(-A^{-1}b) = \frac12\langle b,A^{-1}b\rangle - \langle b.A^{-1}b\rangle + c = c - \frac12 \langle b,A^{-1}b\rangle$$ Finding the maximum value of $\langle y,x\rangle - f(x)$ is similar.

Answer 2

Let $f(x) = \langle Ax, x \rangle$. Then \begin{align} f(x + \Delta x) &= \langle Ax + A\Delta x, x + \Delta x \rangle \\ &= \underbrace{\langle Ax, x \rangle}_{f(x)} + \langle Ax, \Delta x \rangle + \langle A \Delta x, x \rangle + \underbrace{\langle A \Delta x, \Delta x \rangle}_{\text{negligible}} \\ & \approx f(x) + \langle 2 A x, \Delta x \rangle. \end{align} Comparing this with the equation $$ f(x + \Delta x) \approx f(x) + \langle \nabla f(x), \Delta x \rangle $$ we see that $\nabla f(x) = 2 A x$.