Fisher information of product model

69 Views Asked by At

Consider the following regular statistical model: $(\mathbb{R}^n, \mathcal{B}(\mathbb{R}^n), \mathbb{P}_{\theta}: \theta \in \mathbb{R}_+ )$. Assume $\mathbb{P}_{\theta}$ has the density $$\rho(\theta,.) = \frac{1}{\theta} f\left(\frac{x}{\theta}\right)$$, where $f(x)>0$ and $f'(x)$ exists.

I want to show that the fisher- information is given by:

$$I(\theta) = \int_{\mathbb{R}}\frac{n}{\theta^2} \frac{(xf'(x)+f(x))^2}{f(x)} dx $$

Therefore I have to compute $$E[U_{\theta}^2]$$ $U_{\theta}$ is the score, which is given by:

$$U_{\theta} = \partial_{\theta} ln(\rho(\theta,x))$$ for $x \in \mathbb{R}^n$

Then we have: $$\partial_{\theta} ln(\prod_{i=1}^n \rho(\theta,x_i)) = \sum_{i=1}^n\partial_{\theta} \frac{1}{\theta} f\left(\frac{x_i}{\theta}\right) = \sum_{i=1}^n \frac{\frac{-1}{\theta^2}f\left(\frac{x_i}{\theta}\right)+ \frac{1}{\theta}f'\left(\frac{x_i}{\theta}\right)\frac{-x_i}{\theta^2}}{\frac{1}{\theta}f\left(\frac{x_i}{\theta}\right)}$$

How can I go on from there. How can I make it independent from $i$ as the solution suggests to compute $$E[U_{\theta}^2]$$

1

There are 1 best solutions below

2
On BEST ANSWER

You are on the good track. I will start from where you left, i.e.:

$$\partial_{\theta} \ln \left(\prod_{i=1}^{n} \rho\left(\theta, x_{i}\right)\right)=\sum_{i=1}^{n} \partial_{\theta} \log{\frac{1}{\theta} f\left(\frac{x_{i}}{\theta}\right)}=\sum_{i=1}^{n} \frac{\frac{-1}{\theta^{2}} f\left(\frac{x_{i}}{\theta}\right)+\frac{1}{\theta} f^{\prime}\left(\frac{x_{i}}{\theta}\right) \frac{-x_{i}}{\theta^{2}}}{\frac{1}{\theta} f\left(\frac{x_{i}}{\theta}\right)} $$

Therefore what is requested to compute is the following: $$ \mathbb{E} \left[U_{\theta}^{2}\right] $$ $U_{\theta}$ is the score, which is given by: $$ U_{\theta}=\partial_{\theta} \ln (\rho(\theta, x)) $$ for $x \in \mathbb{R}^{n}$, this exactly means:

$$ \mathbb{E} \left[U_{\theta}^{2}\right] = \mathbb{E} \left[ \sum_{i=1}^{n} \left(\partial_{\theta} \log{\frac{1}{\theta} f\left(\frac{x_{i}}{\theta}\right)} \right)^2 + \sum_{i\neq j} \partial_{\theta} \log{\frac{1}{\theta} f\left(\frac{x_{i}}{\theta}\right)} \partial_{\theta} \log{\frac{1}{\theta} f\left(\frac{x_{j}}{\theta}\right)} \right] $$

Now recall that the samples are considered to be i.i.d., then means that all the crossed terms for $i\neq j$ are zero because we can split the expectations by independence and it is a well known fact that the expectation of the log score is zero:

$$\sum_{i\neq j} \mathbb{E} \left[\partial_{\theta} \log{\frac{1}{\theta} f\left(\frac{x_{i}}{\theta}\right)} \partial_{\theta} \log{\frac{1}{\theta} f\left(\frac{x_{j}}{\theta}\right)} \right] \stackrel{i.i.d.}{=} \sum_{i\neq j} \underbrace{\mathbb{E} \left[\partial_{\theta} \log{\frac{1}{\theta} f\left(\frac{x_{i}}{\theta}\right)} \right]}_{=0} \underbrace{\mathbb{E} \left[ \partial_{\theta} \log{\frac{1}{\theta} f\left(\frac{x_{j}}{\theta}\right)} \right]}_{=0} =0$$

Thus we are left only with the first summation of the squared terms, but again since the samples are i.i.d. and we are taking expectation results into:

$$ \mathbb{E} \left[U_{\theta}^{2}\right] = \mathbb{E} \left[ \sum_{i=1}^{n} \left( \partial_{\theta} \log{\frac{1}{\theta} f\left(\frac{x_{i}}{\theta}\right)} \right)^2 \right] = n \mathbb{E} \left[ \left( \partial_{\theta} \log{\frac{1}{\theta} f\left(\frac{x}{\theta}\right)} \right)^2 \right] = \int n \left( \frac{\frac{-1}{\theta^{2}} f\left(\frac{x}{\theta}\right)+\frac{1}{\theta} f^{\prime}\left(\frac{x}{\theta}\right) \frac{-x}{\theta^{2}}}{\frac{1}{\theta} f\left(\frac{x}{\theta}\right) }\right)^2 \frac{1}{\theta} f\left(\frac{x}{\theta}\right) dx =$$

Now we can make a change of variable and set $y = \frac{x}{\theta}$, thus $dy = \frac{1}{\theta} dx$ and obtain:

$$= \int n \left( \frac{\frac{-1}{\theta} f\left(y\right)+f^{\prime}\left(y\right) \frac{-y}{\theta}}{ f\left(y\right) }\right)^2 f\left(y\right) dy$$

Now note that the minuses inside the square are not of any use, and we pull outside $\frac{1}{\theta}$ and obtain:

$$= \int \frac{n}{\theta^2} \left( \frac{f\left(y\right)+f^{\prime}\left(y\right) y}{ f\left(y\right) }\right)^2 f\left(y\right) dy= \int \frac{n}{\theta^2} \frac{\left(f\left(y\right)+f^{\prime}\left(y\right) y\right)^2 }{ f\left(y\right) } dy$$

Now just call my $y$, $x$ and you get the answer.

(Please if this answer was useful leave positive feedback and/or accept the answer, as it took me quite some time :) ).