Fisher Information for a misspecified model

Question

Up until a few days ago I was thinking that the following two forms of the Fisher Information are "always" equivalent: $$(1) \quad \mathcal{I(\theta)}= E_\theta [\frac{\partial \log \ell(y;\theta)}{\partial \theta} \frac{\partial \log \ell(y; \theta)}{\partial \theta'}],$$ $$(2) \quad \mathcal{I(\theta)}= E_\theta [-\frac{\partial^2 \log \ell(y;\theta)}{\partial \theta\partial \theta'} ],$$ where $\theta$ is the parameter vector, $y$ is the vector of observations, and $\ell$ is the likelihood function.

I heard that (1) is the original definition of the Fisher Information, however (2) only holds when the model is not "misspecified".

I consulted some sources, among them Statistics and Econometrics Models (Gourieroux and Monfort 1995, Vol 1, p83 property 3.8), but there is no mention of any regularity conditions needed to hold for the second definition of $\mathcal{I}(\theta)$ to be equivalent to the original definition (1).

How does misspecification of a model cause this equivalency to break?

Answer 1

There are regularity conditions so that the two expressions are equal. In Statistical Theory of F.Abramovich and Y. Ritov, they give these two regularity conditions:

1. The support of $f(x,\theta)$, namely $K:=\{x\ | \ f(x,\theta)>0 \ \}$, does not depend on $\theta$.
2. One has $$\frac{d}{d\theta}\int_{K} g(x) f(x,\theta)dx=\int_{K} g(x)\frac{\partial}{\partial\theta}f(x,\theta)dx$$ for any $g(\cdot)$ for which $E[|g(X)|]$ is finite.

So actually one must fulfill the conditions of differentiating under the integral sign. As you might know one gets the other form of the Fisher Information which is not always right by differentiating under the integral sign, see for example this page.

I'll show you an example where both conditions are not fulfilled and how it goes wrong.

Example.

Let $X\sim \text{Unif}[0,\theta]$. So $f(x,\theta) = \frac{1}{\theta}\textbf{1}_{[0,\theta]}$. The support is $[0,\theta]$ and therefore dependent on $\theta$. We got a problem with the first condition. Now let's look to the second condition. Take $g(x)=1$: \begin{align} \frac{d}{d\theta}\int_0^\theta f(x,\theta)dx=\frac{d}{d\theta}\int_0^\theta \frac{1}{\theta}dx = \frac{d}{d\theta} 1 =0 \end{align}

On the other hand: \begin{align} \int_0^\theta \frac{\partial}{\partial\theta} f(x,\theta)dx = \int_0^\theta -\frac{1}{\theta^2} dx \end{align} So: $$\frac{d}{d\theta}\int_0^\theta f(x,\theta)dx\neq\int_0^\theta \frac{\partial}{\partial\theta}f(x,\theta)dx$$ So condition 2 is also not fullfilled. Now calculate the fisher number: \begin{align} E\bigg[\left(\frac{\partial}{\partial\theta}\log f(X,\theta) \right) ^2 \bigg] =E\bigg[\left(\frac{\partial}{\partial\theta}-\log\theta \right) ^2 \bigg] = \frac{1}{\theta^2} \end{align} On the other hand: \begin{align} E\bigg[-\frac{\partial^2}{\partial\theta^2}\log f(X,\theta) \bigg] = E\bigg[\frac{\partial^2}{\partial\theta^2}\log\theta \bigg] = -\frac{1}{\theta^2} \end{align} And so the expressions are not equal.