Need help on the Proof of Lemma 7.3.11 in Statistical Inference Casella&Berger (v2)

487 Views Asked by At

The Lemma states that if $f(x|\theta)$ satisfies

$\frac{d}{d\theta}\mathbb{E}_\theta\left(\frac{\partial}{\partial\theta} \log(X|\theta)\right) = \int \frac{\partial}{\partial\theta} \left[\left(\frac{\partial}{\partial\theta}\log f(x|\theta)\right) f(x|\theta)\right] dx$

then
$\mathbb{E}_\theta \left( \left(\frac{\partial}{\partial\theta} \log f(X|\theta)\right)^2\right) = -\mathbb{E}_\theta \left( \frac{\partial^2}{\partial \theta^2}\log f(X|\theta)\right). $

The proof of the Lemma is left as an Exercise (7.39). what I get is

$\mathbb{E}_\theta \left( \frac{\partial^2}{\partial \theta^2}\log f(X|\theta)\right) = \int \left( \frac{\partial^2}{\partial \theta^2}\log f(X|\theta)\right) \log(x|\theta) dx $

$=\int \frac{\partial}{\partial\theta} \left[\left(\frac{\partial}{\partial\theta}\log f(x|\theta)\right) f(x|\theta)\right] dx - \int\left(\frac{\partial}{\partial\theta}\log f(x|\theta)\right)\frac{\partial}{\partial\theta} f(x|\theta) dx $

Note that

$\mathbb{E}_\theta \left( \left(\frac{\partial}{\partial\theta} \log f(X|\theta)\right)^2\right) =\int\left(\frac{\partial}{\partial\theta}\log f(x|\theta)\right)\frac{\partial}{\partial\theta} f(x|\theta) dx,$

So basically I need that $\int \frac{\partial}{\partial\theta} \left[\left(\frac{\partial}{\partial\theta}\log f(x|\theta)\right) f(x|\theta)\right] dx=0$, for which I am unable to derive from the assumption of the Lemma.

Am I missing anything?

Thanks.

1

There are 1 best solutions below

0
On

It's been a long time since you asked, but it's still useful to have a proper answer I think.

If we do the tedious algebra carefully with the hypothesis we get \begin{equation} \begin{aligned} \frac{d}{d\theta}\int_{-\infty}^{\infty}\frac{d}{d\theta}f_{\theta}(x)dx&=\frac{d}{d\theta}E_{\theta}\left(\frac{d}{d\theta}\log f_{\theta}(X)\right)\\&=\int_{-\infty}^{\infty}\frac{d}{d\theta}\left[\left(\frac{d}{d\theta}\log f_{\theta}(x)\right)f_{\theta}(x)\right]dx\\&=\int_{-\infty}^{\infty}\frac{d^2}{d\theta^2}f_{\theta}(x)dx\text{, } \end{aligned} \end{equation} then, as you said, we have that \begin{equation} \begin{aligned} E_{\theta}\left(\frac{d^2}{d\theta^2}\log f_{\theta}(X)\right)&=\int_{-\infty}^{\infty}\frac{d^2}{d\theta^2}f_{\theta}(x)dx-E_{\theta}\left(\frac{d}{d\theta}\log f_{\theta}(X)\right)^2\\&=\frac{d}{d\theta}\int_{-\infty}^{\infty}\frac{d}{d\theta}f_{\theta}(x)dx-E_{\theta}\left(\frac{d}{d\theta}\log f_{\theta}(X)\right)^2\text{, } \end{aligned} \end{equation} also, we can apply Leibniz's theorem of differentiation under the integral sign, as long as $f_{\theta}(x)$ is integrable for all $\theta$, derivable with respect to $\theta$ for all $x$, and $\frac{d}{d\theta}f_{\theta}$ in integrable for all $\theta$, so \begin{equation} 0=\frac{d}{d\theta}1=\frac{d}{d\theta}\int_{-\infty}^{\infty}f_{\theta}(x)=\int_{-\infty}^{\infty}\frac{d}{d\theta}f_{\theta}(x)\text{, } \end{equation} then, substituting this last result, we have the lemma proved.

You can also check the book solutions for an alternative.