We define the one-parameter exponential family of distribution functions as those whose pmf/pdf can be written as $$\exp\{c(\theta)T(x) + d(\theta) + s(x)\}$$ I would like to show that if c is twice differentiable with a positive derivative and $E(T(X))= \theta$ then $I(\theta) = \dfrac{1}{\text{var}(T(X))}$
I tried directly computing the fisher information of theta, but I do not see why the equality holds.
Any help would be appreciated
$\hat{\theta}=T(x)$ is an unbiased estimate for $\theta$, since $E[T(x)]=\theta$. The Cramer-Rao lower bound indicates that
$$\text{Var}(T(x))= \text{Var}(\hat{\theta})\geq \frac{1}{I(\theta)}$$
The equality holds if $Z = \frac{\partial \log f}{\partial \theta}$ and $\hat{\theta} = T(x)$ are linearly related.
For the exponential family density function we have:
$$Z=\frac{\partial \log f(x,\theta)}{\partial \theta}= \frac{\partial (c(\theta)T(x)+d(\theta)+s(x))}{\partial \theta} = c'(\theta)T(x)+d'(\theta) $$
which is linear and hence, the equality is achieved for exponential family distributions, which means:
$$\text{Var}(T(x))= \frac{1}{I(\theta)}$$
You may also refer to this paper for the proof.
Note: To study the equality condition of Cramer-Rao inequality, please see the proof in this lecture note. It includes a Cauchy-Schwartz inequality where:
$$\text{Cov}[Z,T]\leq \text{Var}[Z]\times \text{Var}[T]$$ with $Z=\frac{\partial log f}{\partial \theta}$ and $T = T(x)$. The equality can be achieved if $Z$ is a linear function of $T$.