How to solve the expectation of the function of RV whose value is also RV?

26 Views Asked by At

Let $X_1$ and $X_2$ be discrete random variables drawn according to PMFs $p_1(\cdot)$ and $p_2(\cdot)$ over the respective $\cal{X}_1=\{1,2,\ldots,m\}$ and $\cal{X}_2=\{m+1,\ldots,n\}$.

Let $$X=\begin{cases}X_1\quad \text{with probability }\alpha,\\X_2\quad \text{with probability }1-\alpha.\end{cases}$$

Let $$H(X) = E\log_2{\frac{1}{p(X)}} = \sum_{x\in\cal{X}} p(x) \log_2{\frac{1}{p(x)}},$$ where $p(x)$ is PMF of $X$ that represents $\Pr\{X=x\}$.


Find $H(X)$ in terms of $H(X_1)$, $H(X_2)$, and $\alpha$.


My answer is

\begin{align} H(X) &= -\underset{<X>}{E}\log_2{p(X)}\\ &= -\Pr\{X=X_1\}\underset{<X_1>}{E}\log_2{p(X_1)} - \Pr\{X=X_2\}\underset{<X_2>}{E}\log_2{p(X_2)}\\ &= -\alpha H(X_1) - (1-\alpha) H(X_2) \end{align}


However, the answer of the solution is $$H(X)=-\alpha\log_2\alpha-(1-\alpha)\log_2(1-\alpha)+\alpha H(X_1)+(1-\alpha)H(X_2)$$

1

There are 1 best solutions below

1
On BEST ANSWER

We have: \begin{align} H(X) =&~-\sum_{x = 1}^{n}p(x)\lg p(x) \\ =&~-\sum_{x=1}^{m} p(x)\lg p(x)-\sum_{x=m+1}^n p(x)\lg p(x) \\ =&~-\sum_{x=1}^{m} \alpha p_1(x)\cdot \lg (\alpha p_1(x)) - \sum_{x=m+1}^{n} (1 - \alpha) p_2(x) \cdot \lg ((1 - \alpha) p_2(x)) \\ =&~-\sum_{x=1}^{m}\alpha p_1(x) \cdot (\lg(\alpha) + \lg(p_1(x))) - \sum_{x = m + 1}^{n} (1 - \alpha)p_2(x) \cdot (\lg (1 - \alpha) + \lg(p_2(x))) \\ =&~-\sum_{x=1}^{m} \alpha\lg(\alpha)\cdot p_1(x) - \alpha\sum_{x=1}^{m}p_1(x)\lg(p_1(x)) - \sum_{x=m+1}^n(1 - \alpha)\lg (1 - \alpha) \cdot p_2(x) \\ &\quad\quad\quad\quad -(1 - \alpha)\sum_{x = m + 1}^n p_2(x) \lg(p_2(x)) \\ =&~-\alpha\lg(\alpha) - (1 - \alpha)\lg(1 - \alpha) + \alpha H(X_1) + (1 - \alpha) H(X_2) \end{align}