What is second Bartlett identity?

Question

I came across the term second Bartlett identity in the wikipedia link: https://en.wikipedia.org/wiki/Variance_function However could not find detail about it. Can anyone help me to understand what this identity is about. More importantly I want to know under what assumption this identity is true.

Answer 1

The Bartlett identities are identities related to expectations of derivatives of the log likelihood function. I will stay in the setting where a random variable $X$ has a non-zero density $f$ that depends on one parameter $\theta$, but these ideas easily extend to the setting where you have a parameter vector. The Bartlett identities now arise when considering the expectation of derivatives of $\ell(\theta;x) = \log(f(x;\theta) )$ with respect to $\theta$. For every $\theta$ we have $$ \int_{\mathbb{R}} f(x;\theta) dx = 1. $$ Assuming (!) that you can interchange partial derivatives and integrals you now get by differentiating both sides $$ \frac{\partial}{\partial \theta} \int_{\mathbb{R}} f(x;\theta) dx = \int_{\mathbb{R}}\frac{\partial}{\partial \theta} f(x;\theta) dx = 0. $$ We can slightly rewrite the term in the middle to relate it to the expectation of the first derivative of $\ell$ as follows $$ \int_{\mathbb{R}}\frac{\partial}{\partial \theta} f(x;\theta) dx = \int_{\mathbb{R}}\frac{\frac{\partial}{\partial \theta} f(x;\theta)}{f(x;\theta)} f(x;\theta) dx = \mathbb{E}\left[\frac{\partial}{\partial \theta} \log(f(X;\theta))\right] $$ leading to $$ \mathbb{E}\left[\frac{\partial}{\partial \theta} \ell(\theta;X)\right]= 0. $$ This is your first Bartlett identity. But you don't have to stop there. Consider the second derivative $$ \frac{\partial^2}{\partial \theta^2} \ell(\theta;x) = \frac{f''(x;\theta)}{f(x;\theta)} - \left( \frac{f'(x;\theta)}{f(x;\theta)}\right)^2 $$ With the same approach as above (again interchanging derivatives and integrals) and taking a second derivative now yields $$ \int_{\mathbb{R}} \frac{\partial^2}{\partial \theta^2} f(x;\theta)dx = 0, $$ or using the same trick as above $$ \mathbb{E}\left[\frac{f''(X;\theta)}{f(X;\theta)}\right] = 0. $$ Using the second derivative of $\ell$ we thus have $$ \mathbb{E}\left[\frac{\partial^2}{\partial \theta^2} \ell(\theta;X)\right] + \mathbb{E}\left[\left(\frac{\partial}{\partial \theta} \ell(\theta;X)\right)^2\right] = 0. $$ Given that $\mathbb{E}\left[\frac{\partial}{\partial \theta} \ell(\theta;X)\right]= 0$ the second term can be identified as the variance of $\frac{\partial}{\partial \theta} \ell(\theta;X)$. This leads to the formula encountered on wikipedia $$ \mathbb{E}\left[\frac{\partial^2}{\partial \theta^2} \ell(\theta;X)\right] + \mbox{Var}\left(\frac{\partial}{\partial \theta} \ell(\theta;X)\right) = 0. $$ and is referred to as the second Bartlett identity. You don't have to stop there either. In fact (infinitely) more relations can be obtained, see Equation (1) in the original paper: M. S. Bartlett. Approximate Confidence Intervals. Biometrika, Vol. 40, No. 1/2 (Jun., 1953), pp. 12-19. The Bartlett identities are further related to maximum likelihood theory (Cramer-Rao lower bound, Fisher information) and can be used to construct or improve statistical tests (see for example: Cordeiro, Gauss M., Cribari-Neto, Francisco: An Introduction to Bartlett Correction and Bias Reduction. 2014.).