Sufficient condition for convex conjugate to be second-order differentiable

Question

Let $f:\mathbb{R}^n\to (-\infty,\infty]$ be a convex lower-semicontinuous function, we then define its conjugate by $$ f^*(y)=\sup_{x\in \mathbb{R}^n}\{x^Ty-f(x)\}. $$ Then there exist well-known sufficient conditions such that $f^*$ is Lipschitz differentiable (e.g. $f$ is strongly convex). May I know whether there exist sufficient conditions such that $f^*$ is $C^2$ or even $C^2$ with Lipschitz 2nd-order derivatives?

---

The motivation is that I would like to approximate $f$ by a function of the form: $$ f_\epsilon(y)=\sup_{x\in \mathbb{R}^n}\{x^Ty-I_C(x)-\epsilon g(x)\}. $$ where $C$ is a convex set (say a standard simplex). I was wondering under which condition on the convex function $g$ such that $f_\epsilon$ is a $C^2$-function.

For $n=1$, $C=[-1,1]$, we know $$ \sup_{x\in [-1,1]}\{x^Ty-\epsilon|x|^2/2\}= \begin{cases} x^2/(2\epsilon) &|x|\le \epsilon,\\ |x|-\epsilon/2 &|x|> \epsilon,\\ \end{cases} $$ which is not $C^2$. But if we choose $g(x)=1-\sqrt{1-x^2}$ for $x\in[-1,1]$, we can obtain a smooth function.

Answer 1

This is not exactly an answer to your problem. But there is a related known result:

Let $f(x) = \frac{1}{p}{\Vert x \Vert}^p, p > 1$. Then $f^\star(s)=\frac{1}{q}{\Vert s \Vert}^q, q > 1, \frac{1}{p} + \frac{1}{q}=1.$

If you make $q \ge 3$, then $f^\star(s)$ is $C^2$ and 2nd order Lipschitz in $[0, M], M > 0$. Based on this, you can at least find a class of functions whose conjugates are 2nd order Lipschitz.

For a general answer, I was wondering if $\nabla f$ is "strongly convex" could be a sufficient condition.

---

I might have some observations to say about your "motivation" part:

The reason for the discontinuity of your example is due to the limitation of the domain of $\epsilon x^2/2$ from $\mathbb{R}$ to $[-1, 1]$.

Due to the restriction to a smaller domain, the set of gradients of $\epsilon x^2/2$ is reduced to $[-\epsilon, \epsilon]$ from $\mathbb{R}$. For any $y$ outside the reduced set of $[-\epsilon, \epsilon]$, but inside the domain of $f^*$, $f^*(y)$ is geometrically the minimum vertical distance between the hyperplane $x^Ty$ and the boundary points of $\epsilon g(x)$, not affected by the curvature of $g(x)$, and hence likely causing discontinuity.

The reason $f^*$ of your $g(x)=1-\sqrt{1-x^2}$ does not suffer discontinuity is exactly because domain of $g$ is NOT reduced by your choice of $C=[-1, 1]$, i.e. gradients of $g(x)=1-\sqrt{1-x^2}$ is still $\mathbb{R}$ under $[-1, 1]$. Hence no boundary points are involved to cause discontinuity. BTW, if you let $C=[-1/2, 1/2]$, my guess is discontinuity will occur for $g(x)=1-\sqrt{1-x^2}$.

So the answer for your "motivation" part is that, for a "well-behaved" (loosely termed, e.g. bounded & $C^\infty$, maybe?) $g(x)$, if $\nabla g(x)$ takes the entire $\mathbb{R}^d$ as its image under your choice of $C$, the corresponding shape $f^*$ is only affected by $g(x)$, and should be $C^2$ and continuous.

Answer 2

Theorem Let $f:\mathbb{R}^{N}\rightarrow\left( -\infty,\infty\right] $ be a proper, lower semicontinuous, convex function. Then $f^{\ast}:\mathbb{R}% ^{N}\rightarrow\left( -\infty,\infty\right] $ is differentiable in $\left( \operatorname*{dom}\nolimits_{e}f^{\ast}\right) ^{\circ}$ if and only if $f$ is strictly convex in all convex sets contained in $$ \bigcup_{y\in\left( \operatorname*{dom}\nolimits_{e}f^{\ast}\right) ^{\circ }}\partial f^{\ast}\left( y\right) . $$

In particular, this applies if $f$ is always strictly convex. The proof is in the same book.

Theorem Let $f:\mathbb{R}^{N}\rightarrow\left( -\infty,\infty\right] $ be a proper, lower semicontinuous, convex function. Assume that $\nabla f$ exists and is differentiable in a neighborhood of some point $x_{0}\in\mathbb{R}^{N}$, that $\nabla^{2}f$ is continuous at $x_{0}$, and that the $N\times N$ matrix $\nabla^{2}f\left( x_{0}\right) $ is nonsingular. Let $y_{0}:=\nabla f\left( x_{0}\right) $. Then $f^{\ast}$ is differentiable in a neighborhood of $y_{0}$, $\nabla f^{\ast}$ is differentiable at $y_{0}$, and $$ \nabla^{2}f^{\ast}\left( y_{0}\right) =\left( \nabla^{2}f\left( x_{0}\right) \right) ^{-1}. $$ Moreover, if $f$ is of class $C^{2}$ in a neighborhood of $x_{0}$, then $f^{\ast}$ is of class $C^{2}$ in a neighborhood of $y_{0}$.

You can find the proof in the book "Convex Analysis and Minimization Algorithms II Advanced Theory and Bundle Methods" by Jean-Baptiste Hiriart-Urruty and Claude Lemaréchal Convex Analysis II See Corollary 4.2.10