Let $X_1,...,X_n$ be Bernoulli($p$) where $p$ is unknown, and $n>2$, and let $T=X_1+X_2$. My task is to calculate the information about $p$ in the entire sample and compare it to the information about $p$ given by the statistic.
After a few lines of work, I obtain the following expression for information contained in the sample: $$I_X(p)= n*E_p[(\frac{X_1p^{-1}(1-p)+X_1+1}{1-p})^2] $$
To calculate this, I first calculated the information given by one observation, and then multiplied that information by $n$.
Now, after some work, I obtained the following expression for information contained in $T$: $$I_T(p)=E_p[(\frac{Tp^{-1}(1-p)-1}{1-p})^2]$$
Now, I know from previous questions that $T$ is sufficient for $p$. Hence, $I_X(p)$ should equal $I_T(p)$. However, I have no idea how I am to compare these two quantities, because one has an $n$, and the other has an $X_2$ (in the $T$). Any advice?
Fisher Information Matrix (FIM), is the negative of the Expectation of the Hessian of the log likelihood function, namely \begin{equation} I(p) = - E H(p) \end{equation} where $I(p)$ is the FIM and $H(p)$ is the Hessian. Your likelihood function for independent samples is \begin{equation} L(p) = f(x_1\vert p) \ldots f(x_n \vert p) \end{equation} where \begin{equation} f(x_i \vert p) = p^{x_i} (1 - p)^{1 - x_i} \end{equation} Both equations above give us the likelihood function \begin{equation} L(p) = p^{ \sum x_i} (1 - p)^{n - \sum x_i} \end{equation} Log likelihood becomes \begin{equation} l(p) = \log L(p) = \log \big( p^{ \sum x_i} (1 - p)^{n - \sum x_i} \big) \end{equation} Denoting $\bar{x} = \frac{1}{n} \sum x_i$ for sake of simple presentation, one gets: \begin{equation} l(p) = n\bar{x} \log(p) + n(1 - \bar{x})\log(1-p) \end{equation} The score function (gradient of the log likelihood) is \begin{equation} s(p) = l'(p) = n \frac{\bar{x}}{p} - n \frac{(1- \bar{x})}{1 - p} \end{equation} The Hessian becomes \begin{equation} H(p) = s'(p) = - \frac{n(1-2p)\bar{x} + np^2}{p^2(1-p)^2} \end{equation} The FIM becomes \begin{equation} I(p)= -E H(p) = E\big[ \frac{n(1-2p)\bar{x} + np^2}{p^2(1-p)^2} \big] = \big[ \frac{n(1-2p)E\bar{x} + np^2}{p^2(1-p)^2} \big] \end{equation} But $E\bar{x} = p$ so \begin{equation} I(p) = E\big[ \frac{n(1-2p)\bar{x} + np^2}{p^2(1-p)^2} \big] = \big[ \frac{n(1-2p)p + np^2}{p^2(1-p)^2} \big] = \frac{n}{p(1-p)} \end{equation} Now, \begin{equation} T = X_1 + X_2 \end{equation} and the distribution of $T$ is \begin{equation} g(T=t\vert p) = Pr(T = t) = C_2^t p^t(1-p)^{n-t} \end{equation} Using Bayes theorem, we know that \begin{equation} f(x_1 \ldots x_n \vert T = t, p) = \frac{f(x_1 \ldots x_n,t=T \vert p)}{g(T= t \vert p)} \end{equation} Let's get $f(x_1 \ldots x_n,t=T \vert p)$, the likelihood function becomes \begin{equation} f(x_1 \ldots x_n , T = t\vert p) = p^{t + \sum\limits_{i=3}^n x_i}(1-p)^{n - t - \sum\limits_{i=3}^n x_i} \end{equation} \begin{equation} f(x_1 \ldots x_n \vert T = t, p) = \frac{f(x_1 \ldots x_n,t=T \vert p)}{g(T= t \vert p)} = \frac{p^{t + \sum\limits_{i=3}^n x_i}(1-p)^{n - t - \sum\limits_{i=3}^n x_i}}{C_2^t p^t(1-p)^{n-t}} \end{equation} which is \begin{equation} f(x_1 \ldots x_n \vert T = t, p) = \frac{p^{ \sum\limits_{i=3}^n x_i}(1-p)^{ - \sum\limits_{i=3}^n x_i}}{C_2^t} = \frac{1}{C_2^t} \Big(\frac{p}{1-p}\Big)^{\sum\limits_{i=3}^n x_i} = h(p) \end{equation} which is a function of $p$.