Suppose we have that $X_1, \ldots, X_n$ are iid from a distribution with ONE parameter, $\theta$. Then, under regulatory conditions, the Fisher Information may be written as:
$$ \mathcal{I}(\theta) = - \operatorname{E} \left[\left. \frac{\partial^2}{\partial\theta^2} \log f(X;\theta)\right|\theta \right] $$
I know that for the iid case, we have that:
$$ \mathcal{I}(\theta) = n\mathcal{I}_1(\theta) $$
Where $\mathcal{I}_1(\theta)$ is the Fisher Information for the first sample.
Now, I am wondering why this doesn't hold in a multi-variate setting, where instead of having ONE parameter, we have multiple.
Suppose now that $X_1, \ldots, X_n$ are iid from a distribution with parameters $\theta = (\theta_1, \ldots, \theta_k)$. In this case, it is my understanding that additivity will still hold, that is:
$$ \mathcal{I}(\theta) = \mathcal{I}_1(\theta)+ \ldots + \mathcal{I}_n(\theta) $$
However, why do we not have that case that $\mathcal{I}(\theta) = n\mathcal{I}_1(\theta)$ anymore?
Thanks!
In the multi-parameter case, $\theta = (\theta_1, ..., \theta_k)$, Fisher information is a $k\times k$ matrix where the $ij$ entry is given by $$ \mathcal{I}(\theta)_{ij} = -E[\frac{\partial}{\partial\theta_i \partial \theta_j} \log f(X;\theta)], $$ so, assuming that the sample is i.i.d, the additivity is still holds component-wise in the information matrix.