Question regarding Bregman Divergence of two probability density functions (p.d.f.)

91 Views Asked by At

Bregman Divergence between two probability density functions is given by the following form: $$ d_{\phi}(g,f) = \int_{\mathcal{x}} \phi(g) - \phi(f) - (g - f) \phi^{'}(f) \;\;d \mathcal{x}\; \; , $$ where, $\mathcal{x}$ : support of the 2 probability density functions, $g$ and $f$;

$\phi$ : a convex function.

How can we then prove that $$d_{\phi}(g,f) = 0, \; \; \; \textrm{if and only if} \;\; g = f \;\;?$$

1

There are 1 best solutions below

0
On

It's somewhat easier to see Bregman divergences in the more general context of inner-product spaces, if you take a smooth, strictly convex function $\phi$ then

$$ \phi(z) > \phi(x) + \langle z-x,\phi'(x)\rangle $$

for all $(x,z)$ s.t. $x\neq z$. Rearranging:

$$ \phi(z) - \phi(x) - \langle z-x,\phi'(x)\rangle > 0$$

the left hand side is the Bregman divergence associated with $\phi$ and is positive for all non-coinciding points (in your case, points are pdfs).