The maximum-likelihood estimators of $\sigma^2$

208 Views Asked by At

A sample of size $n$ is drawn from each of four normal populations, all of which have the same variance $\sigma^2$. The means of the four populations are $a+b+c$, $a+b-c$, $a-b+c$ and $a-b-c$.

What are the maximum-likelihood estimators of $c$, $b$, $a$ and $\sigma^2$.

$a$, $b$, $c$ solved.

But for $\sigma^2$. Is this answer correct $\sigma^2 = \frac{1}{4}(s^2_1+s^2_2 + s^2_3 + s^2_4)$.

1

There are 1 best solutions below

0
On BEST ANSWER

It's somewhat more involved, since there is a linear constraint on the set of four population means. Instead of the usual $\text{“ }s_i^2\text{ ''}$, which is the average of the squares of observed deviations from the mean of the $i$th population, one must use an expression defined similarly where the deviations are from the MLE of the mean of that population. We have \begin{align} \mu_{11} = a+b+c & & \mu_{10} = a+b-c \\ \mu_{01} = a-b+c & & \mu_{00} = a-b-c \end{align} and so the means satisfy the constraint $$ \mu_{00}+\mu_{11}-\mu_{10}-\mu_{01} = 0. \tag 1 $$

One way to view this is as a linear regression problem: $$ Y_{ijk} = a + b \text{ row}_i + c \text{ column}_j + \text{error}_{ijk} \quad \text{for }i,j \in \{0,1\}, \quad k=1,\ldots,n, $$ where \begin{align} \text{row}_0 & = -1, & \text{row}_1 & = 1, \\ \text{column}_0 & = -1, & \text{column}_1 & = 1. \end{align}

Since the errors are i.i.d. and distributed as $N(0,\sigma^2)$, the MLEs for $a$, $b$, and $c$ will be the least-squares estimates $\hat a$, $\hat b$, $\hat c$ of $a$, $b$, $c$.

The fitted values are then $$ \hat Y_{ij} = \hat a + \hat b \text{ row}_i + \hat c \text{ column}_j = \begin{cases} \hat a+\hat b+\hat c & \text{if }i=j=1, \\ \hat a-\hat b+\hat c & \text{if }i=0,\ \&\ j=1, \\ \hat a+\hat b-\hat c & \text{if }i=1\ \&\ j=0, \\ \hat a-\hat b-\hat c & \text{if }i=j=0. \end{cases} $$ Note: The fitted value $\hat Y_{ij}$ is not just the average $$ \bar Y_{ij} = \frac 1 n \sum_{k=1}^n Y_{ijk} $$ because that would not satisfy the constraint $(1)$ above.

The MLE for $\sigma^2$ (not to be confused with the best unbiased estimator of $\sigma^2$, which is bigger) is then $$ \hat\sigma^2 = \frac 1 {4n} \sum_{i=0}^1 \sum_{j=0}^1 \sum_{k=1}^n (Y_{ijk} - \hat Y_{ij})^2. $$ For the MLEs for $a$, $b$, and $c$, I'm getting this: \begin{align} \hat a & = \frac 1 4 \left( \bar Y_{11} + \bar Y_{10} + \bar Y_{01} + \bar Y_{00} \right) \\[10pt] \hat b & = \frac 1 4 \left( \bar Y_{11} + \bar Y_{10} - \bar Y_{01} - \bar Y_{00} \right) \\[10pt] \hat c & = \frac 1 4 \left( \bar Y_{11} - \bar Y_{10} + \bar Y_{01} - \bar Y_{00} \right) \end{align}

PS: In a practical situation, you might want to do a statistical test of the null hypothesis that the linear constraints are satisfied. You can do that by means of an F-test in which $s_1^2+s_2^2+s_3^2+s_4^2$ is in the denominator and in the numerator you'd have $\widehat \sigma^2$ as derived above minus $\frac 1 4(s_1^2+s_2^2+s_3^2+s_4^2)$ and you divide by appropriate numbers of degrees of freedom.