Calculation of Information(Fisher) of a function of Bernoulli random variables.

Question

If $ X_1,X_2, ...X_n $ follows iid $ Bernoulli(p) $ then What will be the Fisher's Information $ I_U(p) $ where $ U = X_1 + X_2 + 2X_3 $?

I am facing a problem to find the probability mass function(pmf) of $U$, to be more specific, I don't know what to do with the term $ 2X_3 $ of $U$ where $X_i$s are iid $Bernoulli(p)$. That's why I can not proceed for further calculation.

To calculate the Fisher's Information, I need the pmf as a function of $p$, so that I can take the derivative for $p$.

Kindly help me to find the pmf of $U$.

Answer 1

What are the possible values of $U$? Note that $X_1,X_2,X_3$ all take values in $\{0,1\}$, so
$U=X_1+X_2+2X_3$ can take values $0,1,2,3,4$.

• $U=0\implies X_1=X_2=X_3=0$ and the probability of this happening is $(1-p)(1-p)(1-p)=(1-p)^3$ since $X_i$-s are independent and $P(X_i=0)=1-p$
• $U=1 \implies X_1=1, X_2=X_3=0$ which has probability $p(1-p)^2$
  or $X_1=X_3=0, X_2=1$ which also has probability $p(1-p)^2$,
  so $U=1$ with probability $2p(1-p)^2$
• $U=2\implies X_1=X_2=1,X_3=0$ which has probability $p^2(1-p)$
  or $X_1=X_2=0, X_3=1$ which has probability $(1-p)^2p$
  so $U=2$ with probability $p^2(1-p)+(1-p)^2p=p(1-p)(p+1-p)=p(1-p)$
• $U=3\implies X_1=X_3=1,X_2=0$ which has probability $p^2(1-p)$
  or $X_2=X_3=1,X_1=0$ which has probability $p^2(1-p)$
  so $U=3$ with probability $2p^2(1-p)$
• $U=4 \implies X_1=X_2=X_3=1$ which has probability $p^3$.

So $U=\left\{ \begin{array}{ll} 0 & \text{ with probability } (1-p)^3\\ 1 & \text{ with probability } 2p(1-p)^2\\ 2 & \text{ with probability } p(1-p)\\ 3 & \text{ with probability } 2p^2(1-p)\\ 4 & \text{ with probability } p^3 \end{array}\right\}$

$\implies \log(f(U;p)) = \left\{ \begin{array}{ll} \log((1-p)^3) & \text{ if } U=0\\ \log(2p(1-p)^2) & \text{ if }U=1\\ \log(p(1-p)) & \text{ if }U=2\\ \log(2p^2(1-p)) & \text{ if }U=3\\ \log(p^3) & \text{ if } U=4 \end{array}\right\}$

$\implies \log(f(U;p)) = \left\{ \begin{array}{ll} \log((1-p)^3) & \text{ with probability } (1-p)^3\\ \log(2p(1-p)^2) & \text{ with probability } 2p(1-p)^2\\ \log(p(1-p)) & \text{ with probability } p(1-p)\\ \log(2p^2(1-p)) & \text{ with probability } 2p^2(1-p)\\ \log(p^3) & \text{ with probability } p^3\end{array}\right\}$

Then, the Fisher's information you need will be $$E\left[-\dfrac{d^2}{dp^2}\log(f(U;p))\right]= -\sum_{i=0}^4 \left(\dfrac{d^2}{dp^2} \log(f(i;p))\times P(U=i) \right)$$