Finding the differential entropy for random variable

93 Views Asked by At

I'm having trouble understanding how to approach this problem, and would like to get some help and explanations :)

Let $X$ be a random variable with density $f_X(x)$. It is known that $X$ is non-negative ($f_X(x)=0$ for $x<0$). It is also known that $h(X)=−10$.

For a variable Y with $f_Y(y)=\frac 1 2 f_X(y)+\frac 1 2 f_X(-y)$, what's the differential entropy?

-- So my idea was to write the definition of differential entropy and then substitute with the definition of $f_Y(y)$, while noticing that the intergral can be in the range $[0,\infty]$ since $f_X(x)=0$ for $x<0$. From here I can't figure out what to do.

Thanks for your help :)

Edit: This is where I got:

$h(Y)=\int \frac 1 2 (f_X(y)+f_X(-y))log \frac 1 {0.5(f_X(y)+f_X(-y))}$

Also, since $f_X(x)=0$ for negative x:

$h(X)=\int _0 ^\infty f_X(x)log \frac 1 {f_Y(x)}dx$

and this is were i'm stuck. I thought maybe split to cases, for y>0 i get that $f_X(-y)=0$, but it doesn't get me too far.

1

There are 1 best solutions below

0
On

$f_Y$ can be written as $f_Y(y) = \begin{cases} \frac{1}{2} f_X(y) & y \ge 0 \\ \frac{1}{2} f_X(-y) & y < 0\end{cases}$. Then,

\begin{align} h(Y) &= -\int_{-\infty}^\infty f_Y(y) \log f_Y(y) \, dy \\ &= -\int_{0}^\infty f_Y(y) \log f_Y(y) \, dy - \int_{-\infty}^0 f_Y(y) \log f_Y(y) \, dy \\ &= -\int_{0}^\infty \frac{1}{2} f_X(y) \log (f_X(y)/2) \, dy - \int_{-\infty}^0 \frac{1}{2} f_X(-y) \log (f_X(-y)/2) \, dy \\ &= -\int_{0}^\infty f_X(y) \log (f_X(y)/2) \, dy \quad \text{(I think this is the last line in your post)} \\ &= -\int_0^\infty f_X(y) \log f_X(y) \, dy + \log(2) \int_0^\infty f_X(y) \, dy \\ &= h(X) + \log(2). \end{align}


Why the answer is intuitive: Note that the distribution of $Y$ can be formulated as follows: let $Z \sim \text{Bernoulli}(1/2)$ be independent of $X$. Then the conditional distribution of $Y \mid Z$ can be specified as $Y=\begin{cases} X & Z=1 \\ -X & Z=0\end{cases}$. Then by the chain rule of entropy, $$h(Y) = h(Y \mid Z) + h(Z) = h(X) + \log(2).$$