what value for $c$ yields the estimator for $σ^2$ with the smallest mean square error among all estimators of ....

712 Views Asked by At

If $S'^2 = \dfrac{\sum_{i=1}^n (Y_i - \bar{Y})^2}{n}$ and $S^2 = \dfrac{\sum_{i=1}^n (Y_i - \bar{Y})^2}{n-1}$

then $S^{'2}$ is a biased estimator of $σ^2$, but $S^2$ is an unbiased estimator of the same parameter. we sample from a normal population.

$S^2$ and $S^{'2}$ are two estimators for $σ^2$ that are of the form $c\sum_{i=1}^n (Y_i − \bar{Y} )^2$. What value for $c$ yields the estimator for $\sigma^2$ with the smallest mean square error among all estimators of the form $c\sum_{i=1}^n (Y_i − \bar{Y} )^2$ ?

I first set $\hat{\sigma}^2 = c \sum_{i=1}^n (Y_i − \bar{Y} )^2$. The I get $E(\hat{\sigma}^2 ) = c(n-1)\sigma^2$. Then I am given $V(\hat{\sigma}^2 ) = 2c^2(n-1)\sigma^4$. I am wondering how to find $E[S^4]$ since $V(\hat{\sigma}^2 ) = E(S^4) - [E(S^2)]^2$

1

There are 1 best solutions below

17
On

$$ \sum_{i=1}^n (Y_i-\bar Y)^2 \sim \sigma^2\chi^2_{n-1}. $$

"Everybody knows" that $\mathbb E(\chi^2_k)=k$. What is perhaps less widely known is that $\operatorname{var}(\chi^2_k)=2k$. Where that comes from I will say something about below.

That implies $$ \operatorname{var}\left( c\sum_{i=1}^k (Y_i-\bar Y)^2 \right) = c^2\sigma^4 2(n-1). $$ So \begin{align} & \mathbb E\left(\left( \sigma^2- c\sum_{i=1}^n(Y_i-\bar Y)^2 \right)^2\right) \\[10pt] = & \sigma^4 -2\sigma^2\Big(c(n-1)\sigma^2\Big) + c^2\mathbb E((\chi^2_{n-1})^2) \\[10pt] = & \sigma^4 -2\sigma^2\Big(c(n-1)\sigma^2\Big) + c^2 \sigma^4\Big( \operatorname{var}(\chi^2_{n-1}) + (n-1)^2 \Big). \tag 1 \end{align}

If I have the details right, the value of $c$ that makes $(1)$ as small as possible should be $1/(n+1)$ (thus an estimator with even more bias than the MLE is "best" in the mean-squared-error sense).

So where did we get $\operatorname{var}(\chi^2_k)= 2k\ {}$?

Since the chi-square random variable as distributed as $Z_1^2+\cdots+Z_k^2$ where the $Z$s are i.i.d. standard normals, the variance should be $k\operatorname{var}(Z_1^2)$. We have $$ \operatorname{var}(Z^2) = \mathbb E(Z^4) - \left(\mathbb E(Z^2)\right)^2 = \mathbb E(Z^4) - 1. \tag 2 $$ This is \begin{align} & \int_{-\infty}^\infty z^4 \varphi(z)\,dz = 2\int_0^\infty z^4 \varphi(z)\,dz \\[10pt] = & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty z^4 e^{-z^2/2} \, dz = 2\frac{1}{\sqrt{2\pi}} \int_0^\infty z^3 e^{-z^2/2} \Big( z \, dz \Big) \\[10pt] = & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty \sqrt{2}^3 u^{3/2} e^{-u} \, du = 2\frac{1}{\sqrt{2\pi}} \sqrt{2}^3 \Gamma\left(\frac 5 2 \right) \\[10pt] = & 2\frac{1}{\sqrt{2\pi}} \cdot \sqrt{2}^3\cdot \frac 3 2 \cdot \frac 1 2 \Gamma\left(\frac 1 2 \right) \end{align}

Recalling that $\Gamma(1/2)=\sqrt{\pi}$, the above reduces to $3$, so in view of $(2)$, we have $\operatorname{var}(\chi^2_1)=2$.

A postscript prompted by comments below: We had $$ \sigma^4 -2\sigma^2\Big(c(n-1)\sigma^2\Big) + c^2 \sigma^4\Big( \operatorname{var}(\chi^2_{n-1}) + (n-1)^2 \Big). \tag 1 $$ Then we found that $\operatorname{var}(\chi^2_{n-1})=2(n-1)$, so $(1)$ becomes $$ \sigma^4 -2\sigma^2\Big(c(n-1)\sigma^2\Big) + c^2 \sigma^4\Big(2(n-1) + (n-1)^2 \Big) $$ This is $\sigma^4$ times $$ 1 -2 \Big(c(n-1) \Big) + c^2 \Big(2(n-1) + (n-1)^2 \Big). $$ By routine algebra this becomes $$ g(c) = (n-1)(n+1)c^2 -2(n-1)c + 1. $$ As a function of $c$, this is a parabola that opens upward, so the lowest point is the one value of $c$ at which $g'(c)=0$. So find $$ g'(c) = 2(n-1)(n+1) c - 2(n-1) = 2(n-1)\Big( (n+1)c - 1 \Big). $$ Clearly the expression in the $\Big(\text{big parentheses}\Big)$ is $0$ precisely when $c = \dfrac{1}{n+1}$.

(It can also be done by completing the square.)