Function $f$ such that convex conjugate $f^*$ has certain domain?

14 Views Asked by At

Given a convex function $f$ that is finite for all $x > 0$, $f(1) = 0$, and $\lim_{t\to 0^+}f(t) = f(0)\in(-\infty,\infty]$, and two distributions $P, Q$, the $f$-divergence from $P$ to $Q$ is defined as

$$D_f(P, Q) = \mathbb{E}_Q[f(dP/dQ)].$$

The convex conjugate $f^\ast$ of a function $f$ is defined to be $f^\ast(y) = \sup_{x\in\mathbb{R}^+}xy - f(x).$

$f$-divergences have what is known as variational representations, namely

$$D_f(P, Q) = \sup_{g} \mathbb{E}_P[g(X)] - \mathbb{E}_Q[f^\ast(g(X))].$$

Here, the supremum is over some subset of all $g : \mathbb{R}\to\mathbb{R}$ depending on the effective domain of a chosen extension of $f^\ast$. See page 119 for details. I won't explain it here as it is both somewhat tedious, something I don't understand the best, and I imagine someone who knows a positive answer to this question is familiar with it already.

One particular $f$-divergence is the $\chi^2$ divergence, which is the $f$-divergence for $f(x) = (x-1)^2$

$$\chi^2(P||Q) = \mathbb{E}_Q[(dP/dQ-1)^2].$$

It has a quite nice variational representation, namely

$$\chi^2(P||Q) = \sup_g : \frac{(\mathbb{E}_P[g(x)] - \mathbb{E}_Q[g(x)])^2}{\mathsf{Var}_Q[g(x)]}.$$

In a somewhat esoteric setting, I have encountered the expression

$$\sup_{g : \mathbb{R}\to[-1,1]} \frac{(\mathbb{E}_P[g(x)]-\mathbb{E}_Q[g(x)])^2}{\mathbb{E}_Q[|g(x)|]},$$

i.e. precisely the $\chi^2$ divergence's variational representation, with the exception of

  1. one divides by the mean absolute deviation $\mathbb{E}[|g(x)|]$ rather than the variance, and

  2. the supremum is over $g : \mathbb{R}\to [-1,1]$ of limited range.

Is this equal to some $f$-divergence?

I would be interested if this is only approximately true, i.e. up to upper and lower bounds by universal constants. When comparing this to $\chi^2$ one can (clearly) obtain bounds, but they are in general $g$ dependent, and quite poor quality overall.