Determinant of Fisher information

Question

In information geometry, the determinant of the Fisher information matrix is a natural volume form on a statistical manifold, so it has a nice geometrical interpretation.

But what is it in statistics? Does it measure anything meaningful? (For example, I would say that if it is zero, then the parameters are not independent. Does this go any further?)

Also, is there any closed form to compute it?

Thanks.

Update: I posted a similar question on stats.se.

Answer 1

If the log-likelihood is quadratic (i.e., the estimator is normally distributed), then the Fisher information is the reciprocal of the variance of the estimator (hence, the lower the variance of the estimator, the more "information" is provided by the data about the parameter...loosely speaking). The square root of the reciprocal of the Fisher Information is the standard error of an estimator (assuming approx. quadratic log-likelihood).

However, if the log-likelihood is not quadratic, then its reciprocal no longer represents the variance of the estimator. You either need to transform the estimator to give it a quadratic log-likelihood or use computational methods to determine the error.

Answer 2

You can generalize @Eupraxis1981's answer to all natural exponential families, that is distributions whose density has the form:

\begin{align} f(x \mid \eta) &= \exp\bigl({T(x) \cdot \eta - g(\eta)} + h(x)\bigr), \qquad x \in \Omega \end{align}

Their Fisher information is the Hessian of the (always strictly convex and $C^\infty$ differentiable) log-normalizer:

\begin{align} \mathcal I(\eta) &= -E(\nabla^2_{\eta} \log f(x \mid \eta) \mid \eta) \\ &= -E\Bigl(\nabla^2_{\eta} \bigl({T(x) \cdot \eta - g(\eta)} + h(x)\bigr) \mathrel{}\Bigm\vert\mathrel{} \eta\Bigr) \\ &= E\bigl(\nabla^2_{\eta} g(\eta) \mathrel{}\bigm\vert\mathrel{} \eta\bigr) \\ &= \nabla^2_{\eta} g(\eta). \end{align}

Therefore the curvature of the log-normalizer tells you how informative realizations of a distribution whose parameters are $\eta$ is. It makes sense that location parameters don't affect the Fisher information.