Some converse of Jensen inequality for some extreme random variable

58 Views Asked by At

This problem arises in my research in a more general form. It concerns information theory and statistical bounds using information theory. I tagged this question with Jensen inequality since the statement can be seen as a converse for Jensen inequality for the following extreme random variable $X$.

Suppose that $X$ is a random variable on $[0,1]$ that averages to $1/2$ such that \begin{align*} X=\begin{cases} 0&\text{with probability } 1/2-\varepsilon (1-p)\\ p&\text{with probability } \varepsilon\\ 1&\text{with probability } 1/2-\varepsilon p\\ \end{cases} \end{align*} for some $p\in(0,1)$ and $\varepsilon \leq \frac{1}{2\max(p,1-p)}$. Let for any $q\in (0,1)$, $h_{q}(x)=\min\left( \frac{x}{q}, \frac{1-x}{1-q} \right)$.

Show (or disprove) that for any random variable $Y$ on $[0,1]$ that averages to $1/2$, if we have $\mathbb E[h_p(X)]\leq \mathbb E[h_p(Y)]$ then for any $q\in (0,1)$, $\mathbb E[h_{q}(X)]\leq \mathbb E[h_{q}(Y)]$.


Due to the construction of $X$, for any $q$, $\mathbb E[h_{q}(X)]=\varepsilon \cdot h_{q}(p)$ and so $\mathbb E[h_p(X)]\leq \mathbb E[h_p(Y)]$ implies that $\varepsilon\leq \frac{\mathbb E[h_{p}(Y)]}{h_p(p)}=\mathbb E[h_{p}(Y)]$, now this means that \begin{align*} \mathbb E[h_{q}(X)]&=\varepsilon \cdot h_{q}(p)\\ &\leq \mathbb E[h_{p}(Y)]\cdot h_{q}(p) \end{align*} Now observe that $h_{q}$ is linear on $[0,q]$ and on $[q,1]$ so we can partition $[0,1]$ in three intervals, suppose without loss of generality that $p < q$, then denote the event $A=\{ Y\in [0,p]\}$, $B=\{ Y\in (p,q) \}$ and $C=\{ Y\in[q,1] \}$ \begin{align*} \mathbb E[h_p(Y)] &= \mathbb P[A] \frac{\mathbb E[Y|A]}{p}+\mathbb P[B]\frac{1-\mathbb E[Y|B]}{1-p}+\mathbb P[C]\frac{1-\mathbb E[Y|C]}{1-p} \end{align*} and \begin{align*} \mathbb E[h_{q}(Y)] &= \mathbb P[A] \frac{\mathbb E[Y|A]}{q}+\mathbb P[B]\frac{\mathbb E[Y|B]}{q}+\mathbb P[C]\frac{1-\mathbb E[Y|C]}{1-q} \end{align*} We can now compare those three expressions one by one when we multiply the first one by $h_{q}(p)=\frac{p}{q}\leq \frac{1-p}{1-q}$. \begin{align*} \mathbb P[A] \frac{\mathbb E[Y|A]}{p}\cdot h_{q}(p) &= \mathbb P[A] \frac{\mathbb E[Y|A]}{q}\\ \mathbb P[C]\frac{1-\mathbb E[Y|C]}{1-p}\cdot h_{q}(p)&\leq \mathbb P[C]\frac{1-\mathbb E[Y|C]}{1-q} \end{align*} for the middle one, set $a=\mathbb E[Y|B]$, then $p\leq a \leq q$ so that \begin{align*} p\leq a &\Leftrightarrow (1-a)p\leq a (1-p)\\ &\Leftrightarrow \frac{1-a}{1-p} \frac{p}{q} \leq \frac{a}{q} \end{align*} This finishes the proof that $\mathbb E[h_{p}(Y)]\cdot h_{q}(p)\leq \mathbb E[h_{q}(Y)]$ and so finishes the proof of the statement.

It would be nice to have some feedback on that proof and since I want to generalize to more dimensions I would appreciate to have a much simpler proof if someone have an idea.

1

There are 1 best solutions below

0
On BEST ANSWER

I just have found a much more satisfying proof of the fact that $\mathbb E[h_p(Y)]\cdot h_q(p)\leq \mathbb E[h_q(Y)]$ :

\begin{align*} h_p(x)\cdot h_q(p) &= \min\left(\frac{x}{p},\frac{1-x}{1-p}\right)\cdot \min\left(\frac{p}{q},\frac{1-p}{1-q}\right)\\ &= \min\left( \frac{x}{p}\cdot\frac{p}{q},\frac{1-x}{1-p}\cdot\frac{p}{q},\frac{x}{p}\cdot\frac{1-p}{1-q},\frac{1-x}{1-p}\cdot\frac{1-p}{1-q}\right)\\ &\leq \min\left( \frac{x}{p}\cdot\frac{p}{q},\frac{1-x}{1-p}\cdot\frac{1-p}{1-q}\right)\\ &=\min\left(\frac{x}{q},\frac{1-x}{1-q}\right)\\ &=h_q(x) \end{align*}