Let $p(x,\theta)$ be a density function and the KL-divergence be given by $$K\left(\theta,\theta^{'}\right)=\int \log \left( \frac{p(x,\theta)}{p(x,\theta^{'})} \right) p(x,\theta)dx$$ Let the fisher information be given by $$I\left(\theta\right)=\int \left\{\frac{\partial}{\partial\theta}\log p(x,\theta)\right\}^2 p(x,\theta)dx$$ To prove that
$$\frac{d^2}{d\theta^{'2}}K\left(\theta,\theta^{'}\right)|_{\theta^{'}=\theta}=I(\theta)$$
I'm trying to solve it but the left hand side always comes out to be $0$ due to miscalculation. Any help would be appreciated. Thank you.
We have $$ \partial_{\theta'} K(\theta,\theta') = \int -\frac{1}{p(x,\theta')}\partial_{\theta'}p(x,\theta') p(x,\theta)dx \\ \partial^2_{\theta'} K(\theta,\theta') =\int\left(\frac{1}{p(x,\theta')^2}(\partial_{\theta'}p(x,\theta'))^2 -\frac{1}{p(x,\theta')}\partial^2_{\theta'}p(x,\theta')\right)p(x,\theta)dx \\\to_{\theta'\to\theta}\int (\partial_{\theta}\log(p(x,\theta)))^2p(x,\theta)dx - \partial^2_{\theta}\int p(x,\theta)dx\\ = \int (\partial_{\theta}\log(p(x,\theta)))^2p(x,\theta)dx -0$$ where the last term is zero since $\int p(x,\theta)dx = 1.$