Estimator for Gamma distribution is biased or unbiased

Question

I have $X_1,...,X_n$ which are all iid and follow a $\Gamma(\theta,\theta)$

Let $\hat{\theta}=\sqrt{\frac{1}{n} \cdot \sum X_i}$ be an estimator for $\theta$

How do I determine if $\hat{\theta}$ is biased or unbiased?

I know that $\sum X_i \sim \Gamma(n\cdot\theta, \theta/n)$ but not sure how to handle the square root

Answer 1

If the estimator were unbiased, this would have to be true for any $n$ and for any $\theta$. So to show that it’s biased, it suffices to show that the expected value in some simple case is not $\theta$. The simplest case is $n=1$, $\theta=1$, where we have a single random variable exponentially distributed with parameter $1$, and the expected value is

$$ \mathsf E[\hat\theta]=\int_0^\infty\mathrm e^{-x}\sqrt x\,\mathrm dx=\frac{\sqrt\pi}2\ne1\;, $$

so the estimator is biased.

Answer 2

If $$X_i \sim \operatorname{Gamma}(\theta, \theta)$$ where the parametrization is by shape and scale, i.e. $$f_X(x) = \frac{x^{\theta-1} e^{-x/\theta}}{\theta^\theta \Gamma(\theta)}, \quad x > 0,$$ then $$\sum_{i=1}^n X_i \sim \operatorname{Gamma}(n\theta, \theta)$$ and $$\bar X = \frac{1}{n} \sum_{i=1}^n X_i \sim \operatorname{Gamma}(n\theta, \theta/n)$$ with PDF $$f_{\bar X}(x) = \frac{x^{n\theta-1} e^{-nx/\theta}}{(\theta/n)^{n\theta} \Gamma(n\theta)}, \quad x > 0.$$ Then the monotone transformation $$\hat \theta = g(\bar X) = \sqrt{\bar X},$$ gives $$f_{\hat \theta}(x) = f_{\bar X}(g^{-1}(x)) \left|\frac{dg^{-1}}{dx}\right| = \frac{(x^2)^{n\theta-1} e^{-nx^2/\theta}}{(\theta/n)^{n\theta} \Gamma(n\theta)} \cdot 2x = \frac{2x^{2n\theta - 1} e^{-nx^2/\theta}}{(\theta/n)^{n\theta} \Gamma(n\theta)}, \quad x > 0.$$ This is a generalized gamma distribution.

To compute the expectation, we can do so directly as follows:
$$\begin{align*} \operatorname{E}[\hat \theta] &= \int_{x=0}^\infty x^{1/2} f_{\bar X}(x) \, dx \\ &= \frac{1}{(\theta/n)^{n\theta} \Gamma(n\theta)} \int_{x=0}^\infty x^{n\theta - 1/2} e^{-nx/\theta} \, dx \\ &= \frac{(\theta/n)^{n\theta + 1/2} \Gamma(n\theta + 1/2)}{(\theta/n)^{n\theta} \Gamma(n\theta)} \int_{x=0}^\infty \frac{x^{n\theta - 1/2} e^{-nx/\theta}}{(\theta/n)^{n\theta + 1/2} \Gamma(n\theta + 1/2)} \, dx \\ &= \frac{(\theta/n)^{1/2} \Gamma(n\theta + 1/2)}{\Gamma(n\theta)}. \end{align*}$$ As this is clearly not a constant function of $n$, $\hat \theta$ is biased. For instance, for the choice $n = 1/\theta$ (possible if $\theta$ is the reciprocal of a positive integer), we have $$\operatorname{E}[\hat \theta] = \Gamma(3/2) \theta \ne \theta.$$