I am trying to compute the Fisher Information for θ in the following scenario:
X_1, X_2, ... , X_n are independent and identically distributed from a Uniform (0, θ)
There are 2 different but equivalent formulas for the Fisher info:
-E(∂^2/∂θ^2 (ln (f(x|θ))) = E(∂/∂θ (ln (f(x|θ)))^2
Now, with the first formula I get -1/θ^2, while with the Second one I get 1/θ^2. (the sign changes)
How is it possible?
Thank you!
Fisher's Information is defined only for Cramer-Rao regular families of distributions. One of the Cramer-Rao regularity conditions is:
$$\frac{\delta}{\delta \theta}f_{\theta}(x) \forall x, \forall \theta$$
exists finite, which in turn implies the support of $x$ being independent of $\theta$. Per contraposition, if such condition is violated, then the derivative doesn't exist finite, hence the distribution is not Cramer-Rao regular and Fisher's Information doesn't exist. Also, it is not always true the two definitions to be interchangeable, an additional regularity condition is needed.
Let me add - hence moving from comment to answer for space - that when computing the derivative of a distribution with respect to a parameter, you can't ignore the fact that it is in the support, since it instead becomes not computable.
To make it clear:
$$X \sim U(0, \theta) \Rightarrow f_{\theta}(x) = \frac{1}{\theta}I(0 < x < \theta)$$
whose derivative is not computable. Finally, note that the two definitions of the Fisher are equivalent if and only if:
$$\frac{\delta^{2}}{\delta \theta^{2}}\int f_{\theta}(x)dx =\int \frac{\delta^{2}}{\delta \theta^{2}} f_{\theta}(x)dx $$
which is not necessarily true. If does not hold, the definition of Fisher Information is:
$$I^{x}(\theta) = E_{\theta} \left [ \left ( \frac{\delta}{\delta \theta} ln(f_{\theta}(x)) \right )^{2} \right ] $$