Triangle Inequality for Le Cam's Metric?

Question

Let $P, Q$ be probability distributions (say absolutely continuous with respect to the Lebesgue measure for simplicity). The Le Cam metric (also sometimes called the Triangular Discrimination) is defined as

$$\Delta(P, Q)^2 = \frac{1}{2}\int \frac{(dP-dQ)^2}{dP+dQ}$$

In Le Cam's book Asymptotic Methods in Statistical Decision Theory, it is stated (page 48) that the non-trivial part of verifying $\Delta(P, Q)$ is a metric reduces to the following inequality

Let $c \geq b\geq a\geq 0$. Then $$ \frac{c-a}{\sqrt{c+a}} \leq \frac{c-b}{\sqrt{c+b}}+\frac{b-a}{\sqrt{b+a}}. $$

This itself follows from convexity of $x\mapsto 1/\sqrt{x}$.

I am trying to verify this myself. I see how an inequality of the above form may be useful, in particular if the condition $c\geq b\geq a\geq 0$ may be ignored, then one can define $f(P,Q) = \frac{dP-dQ}{\sqrt{dP+dQ}}$, a signed measure with respect to the Lebesgue measure. The above inequality (ignoring the condition on $a,b,c$) can then be used to prove that $f(P,Q) \leq f(P,R)+f(R,Q)$. One can then argue that

$$\Delta(P,Q) = \sqrt{1/2}\lVert f(P,Q)\rVert_2 \leq \sqrt{1/2}\lVert f(P,R)+f(R,Q)\rVert_2 \leq \sqrt{1/2}\lVert f(P,R)\rVert_2+\sqrt{1/2}\lVert f(R,Q)\rVert_2.$$

E.g. the result from sub-additivity of $\lVert \cdot\rVert_2$.

I'm stuck when trying to extend the proof to the case the condition on $(c,b,a)$ does not hold. I've tried extending the proof of the above inequality to (for example) when $b\geq c\geq a\geq 0$, but this goes nowhere.

Can someone help me fill in this proof sketch?

Answer 1

For $p,q\geq0$ define $L(p,q)=\frac{(p-q)^2}{p+q}$ $q+p>0$ and $0$ if $p=q=0$.

• Fix $0<p<q$, define the function $R(r)=\sqrt{L(p, r)}+\sqrt{L( r, q)}$. The function $R$ has minima at $r=p$ and $r=q$. Notice that \begin{align}R'(r)&=\frac12\left(\frac{(r-p)(r+3p)}{(r+p)^2\sqrt{L(p, r)}} + \frac{(r-q)(r+3q)}{(r+q)^2\sqrt{L(q, r)}}\right)\\ &=\frac12\left(\frac{(r-p)(r+3p)}{(r+p)^{3/2}|p-r|} + \frac{(r-q)(r+3q)}{(r+q)^{3/2}|q-r|}\right) \end{align} If $0\leq r<p$, $R'(r)<0$ and if $q<r$, $R'(r)>0$. Also, $$\lim_{r\rightarrow p-}\frac{(r-p)(r+3p)}{(r+p)^{3/2}|p-r|}=-\sqrt{\frac{2}{p}},\qquad \lim_{r\rightarrow q+}\frac{(r-q)(r+3q)}{(r+q)^{3/2}|p-r|}=\sqrt{\frac{2}{q}}$$ This shows that for $0\leq r\leq p$ or $q\leq r$ $$ R(p)=R(q)\leq R(r)$$ For $p<r<q$, we have the inequality proven by the OP which reduces to $$\sqrt{L(p,q)}=R(p)=R(q)\leq R(r)=\sqrt{L(p,r)}+\sqrt{L(r,q)}$$

• To use the inequality, suppose $P_0,P_1,P_2$ are probability measures dominated by a $\sigma$-finite measure $\mu$, and let $f_j=\frac{dP_j}{d\mu}$ for $j=0,1,2$. The arguments above imply that $$\frac{|f_0-f_1|}{\sqrt{f_0+ f_1}}\leq \frac{|f_0-f_2|}{\sqrt{f_0+ f_2}}\ + \frac{|f_1-f_2|}{\sqrt{f_1+ f_2}}=h\ $$ Since $|f|\leq |h|$ implies $\|f\|_{L_2(\mu)}\leq \|h\|_{L_2(\mu)}$, the triangle inequality implies that $$\Big(\int\frac{(f_0-f_1)^2}{f_0+ f_1}\,d\mu\Big)^{1/2}\leq \Big(\int\frac{(f_0-f_2)^2}{f_0+ f_2}\,d\mu\Big)^{1/2} +\Big(\int\frac{(f_1-f_2)^2}{f_1+ f_2}\,d\mu\Big)^{1/2} $$

---

Just a little digression.

Definition (Csiszár, Ali, Silvey): Suppose $\mu$ is a $\sigma$-finite measure on $(X,\mathscr{B})$, and let $\mathcal{P}_\mu$ the collection of probability measures $P$ on $(X,\mathscr{B})$ such that $P\ll\mu$. For any convex function $g:(0,\infty)\rightarrow\mathbb{R}$ define the functional $$l_g(P_0,P_1)=\int g\Big(\frac{f_0}{f_1}\Big)\mathbb{1}_{\{f_0,f_1>0\}}f_1\,d\mu + g(0+)P_1(f_0=0)+g^*(0+)P_0(f_1=0) $$ where $f_j=\frac{dP_j}{d\mu}$ and $g^*(x)=x g(x^{-1})$ for $x>0$ and $g^*(0+)=\lim_{x\rightarrow0+}g^*(x) \in\overline{\mathbb{R}}$.

The functional $I_g(P_0,P_1)$ is called the $g$-divergent of $P_0$ with respect to $P_1$.

• If $\mu\ll \nu$ and $f=\frac{d\mu}{d\nu}$, then $\frac{dP_j}{d\nu}=f_jf$ and so, the definition of $I_g$ is independent of the dominating measure $\mu$.

• Notice that $g^*$ is also convex: Let $c=(1-t)a+tb$, where $a,b>0$ and $0<t<1$. Then \begin{align} g^*(c)=cg\Big(\frac{1}{c}\Big)=cg\big(\frac{(1-t)a}{c}\frac{1}{a}+\frac{tb}{c}\frac{1}{b}\big)\leq(1-t)a\,g(1/a)+tb\,g(1/b)=(1-t)g^*(a)+tg^*(b) \end{align}

• The convexity assumption implies that $$g(x)\geq g(1)+D^+g(1)\,(x-1),\qquad x>0$$ and so, $g(0+),g^*(0)>-\infty$. Also $$g(f_0/f_1)\mathbb{1}_{\{f_0>0\}}f_1\geq g(1)f_1+D^+g(1)(f_0-f_1) $$ Thus $I_g$ is well defined and may take the value $+\infty$.

---

The functional described by the OP is based on $v(x)=\frac{(1-x)^2}{x+1}$; then $v^*(x)=v(x)$, $v(0+)=1=v^*(0+)$.

\begin{align} l_v(P_0,P_1)&=\int\frac{(f_0-f_1)^2}{f_0+f_1}\mathbb{1}_{\{f_0,f_1>0\}}\,d\mu +\int\mathbb{1}_{\{f_0=0\}}f_1\,d\mu+ \int\mathbb{1}_{\{f_1=0\}}f_0\,d\mu\\ &=\int\frac{(f_0-f_1)^2}{f_0+f_1}\,d\mu \end{align}