Let be $X:=\left(X_1,\dots, X_n\right)$ a vector of independent random variables and $P_X(\theta)=f(X_1,\theta)\cdot f(X_2,\theta)\dots \cdot f(X_n,\theta)$ their probability distribution. We assume that all $f(X_i,\theta)$ are differentiable and strictly positive. Compute the Fisher information $I_P(\theta):=\mathbb{E}_{\theta}\left(\left(\frac{d \ln\left(P_X(\theta)\right)}{d\theta}\right)^2\right)$.
There is one important step in our sample solution which I don't understand
$$ \mathbb{E}_{\theta}\left(\left(\sum\limits_{i=1}^n\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}\right)^2\right)\overset{???}{=}\sum\limits_{i=1}^n\mathbb{E}_{\theta}\left(\left(\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}\right)^2\right). $$
I made some simple example to get an idea of it but it seems that the summands for $1\leq i\neq j\leq n $ that consists of $$2\cdot\mathbb{E}\left(\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}\cdot\frac{d\ln\left(f(X_j,\theta)\right)}{d\theta}\right)$$ don't disappear. Where is the trick?
My Approach:
Maybe this works:
First, we recall that if $X_1,\dots,X_n$ are indepedent random variables, then functions of those random variables, $F(X_1),\dots,F(X_n)$ are again independent random variables (I assume that this statement is well known).
I found a statement in our lecture that says $\mathbb{E}_{\theta}\left(\frac{d \ln\left(P_X(\theta)\right)}{d\theta}\right)=0$, where $X$ is a random variable or a vector of random variables. \begin{align*} &\mathbb{E}_{\theta}\left(\left(\sum\limits_{i=1}^n\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}\right)^2\right)=\mathbb{E}_{\theta}\left(\left(\sum\limits_{i=1}^n\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}-0\right)^2\right)\\ &=\mathbb{E}_{\theta}\left(\left(\sum\limits_{i=1}^n\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}-\mathbb{E}_{\theta}\left(\sum\limits_{i=1}^n\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}\right)\right)^2\right)\\ &=\mathbb{V}_{\theta}\left(\sum\limits_{i=1}^n\left(\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}\right)\right)\\ &=\sum\limits_{i=1}^n\mathbb{V}_{\theta}\left(\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}\right)+2\sum\limits_{1\leq i<j\leq n}\text{Cov}\left(\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta},\frac{d\ln\left(f(X_j,\theta)\right)}{d\theta}\right)\\ &=\sum\limits_{i=1}^n\mathbb{V}_{\theta}\left(\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}\right)+0=\sum\limits_{i=1}^n\mathbb{E}_{\theta}\left(\left(\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}-\mathbb{E}_{\theta}\left(\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}\right)\right)^2\right)\\ &=\sum\limits_{i=1}^n\mathbb{E}_{\theta}\left(\left(\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}-0\right)^2\right)=\sum\limits_{i=1}^n\mathbb{E}_{\theta}\left(\left(\frac{d\ln\left(f(X_i,\theta)\right)}{d\theta}\right)^2\right). \end{align*}
I am not sure if there is a shorter or more elegant proof. Any hints are welcome :)
Using the fact that $X_1,...,X_n$ are independent, we can say that $$\begin{eqnarray*}\mathbb{E}_{\theta}\left(\left(\sum_{i=1}^n\frac{\mathrm{d}\ln f(X_i,\theta)}{\mathrm{d}\theta}\right)^2\right)&=&\mathbb{V}_{\theta}\left(\sum_{i=1}^n\frac{\mathrm{d}\ln f(X_i,\theta)}{\mathrm{d}\theta}\right)-\left(\mathbb{E}_{\theta}\left(\sum_{i=1}^n\frac{\mathrm{d}\ln f(X_i,\theta)}{\mathrm{d}\theta}\right)\right)^2\\ &=& \sum_{i=1}^n\mathbb{V}_{\theta}\left(\frac{\mathrm{d}\ln f(X_i,\theta)}{\mathrm{d}\theta} \right)-\left(\sum_{i=1}^n\mathbb{E}_{\theta}\left(\frac{\mathrm{d}\ln f(X_i,\theta)}{\mathrm{d}\theta}\right)\right)^2 \\ &=& \sum_{i=1}^n\mathbb{V}_{\theta}\left(\frac{\mathrm{d}\ln f(X_i,\theta)}{\mathrm{d}\theta} \right) \\ &=& \sum_{i=1}^n \left[\mathbb{E}_{\theta}\left(\left(\frac{\mathrm{d}\ln f(X_i,\theta)}{\mathrm{d}\theta}\right)^2\right)-\left(\mathbb{E}_{\theta}\left(\frac{\mathrm{d} \ln f(X_i,\theta)}{\mathrm{d}\theta}\right)\right)^2\right] \\ &=& \sum_{i=1}^n\mathbb{E}_{\theta}\left(\left(\frac{\mathrm{d} \ln f(X_i,\theta)}{\mathrm{d}\theta}\right)^2\right)\end{eqnarray*}$$