Calculating Expected value of MSE for ANOVA Variance Components

Question

For a given anova model

$$Y_{ijk}=\mu +\alpha_i + \beta_j +\alpha\beta_{ij} +\epsilon_{ijk}$$

where the model can be mixed, random or fixed. I have been trying to calculate or find a source that can help me calculate the expected value of Sum of square components. For example if I assume a random model with $\alpha_i\sim N(0,\sigma_A)$, $\beta_j\sim N(0,\sigma_B)$, $\alpha\beta_{ij}\sim N(0,\sigma_{AB})$ , and $\epsilon_{ij}\sim N(0,\sigma_E)$.

I know that $$\sum_{i}\sum_{j}\sum_{k}(\bar{Y}_{i\cdot\cdot}-\bar{Y}_{\cdot\cdot\cdot})^2$$ will follow a scaled central $\chi^2$ with the scaled parameter being a linear combination on the variances of my random variables. I can find the formula in every textbook, but none actually derive results. Does anyone know where I can find the derivation?

Answer 1

Consider a two-way classified data with $m$ observations per cell. The random effects model is

$$y_{ijk}=\mu+\alpha_i+\beta_j+\gamma_{ij}+\varepsilon_{ijk}\quad;\small \,i=1,\ldots,p,j=1,\ldots,q,k=1,\ldots,m\,,$$

where $\mu$ is a general effect, $\alpha_i,\beta_j,\gamma_{ij}$ are additional random effects and $\varepsilon_{ijk}$ is random error.

Derivation of the distributions of different sum of squares is not difficult once you consider the assumptions under which they are usually derived.

We assume $\alpha_i\stackrel{\text{i.i.d}}\sim N(0,\sigma^2_\alpha),\beta_j\stackrel{\text{i.i.d}}\sim N(0,\sigma^2_\beta), \gamma_{ij}\stackrel{\text{i.i.d}}\sim N(0,\sigma^2_\gamma)$ and $\varepsilon_{ijk}\stackrel{\text{i.i.d}}\sim N(0,\sigma^2_\varepsilon)$. Moreover, suppose that $\alpha_i,\beta_j,\gamma_{ij}$ and $\varepsilon_{ijk}$ are all pairwise independent.

Now, $$\overline y_{i\cdot\cdot}=\mu+\alpha_i+\overline \beta+\overline \gamma_{i\cdot}+\overline\varepsilon_{i\cdot\cdot}$$

And $$\overline y_{\cdot\cdot\cdot}=\mu+\overline \alpha+\overline \beta+\overline \gamma_{\cdot\cdot}+\overline \varepsilon_{\cdot\cdot\cdot}$$

Therefore,

$$\sum_{i=1}^p (\overline y_{i\cdot\cdot}-\overline y_{\cdot\cdot\cdot})^2=\sum_{i=1}^p (u_i-\overline u)^2\,,$$

where $$u_i=\alpha_i+\overline\gamma_{i\cdot}+\overline\varepsilon_{i\cdot\cdot}$$

Now note that $$u_i \stackrel{\text{i.i.d}}\sim N(0,\sigma_u^2)\,,$$

with $$\sigma^2_u=\sigma^2_\alpha+\frac{\sigma^2_{\gamma}}{q}+\frac{\sigma^2_{\varepsilon}}{qm}=\frac{qm\sigma^2_\alpha+m\sigma^2_\gamma+\sigma^2_\varepsilon}{qm}$$

Thus,

\begin{align} E\left[qm\sum_{i=1}^p (\overline y_{i\cdot\cdot}-\overline y_{\cdot\cdot\cdot})^2\right]&=E\left[qm\sum_{i=1}^p (u_i-\overline u)^2\right] \\&=qm\left\{\sum_{i=1}^p E\left[u_i^2\right]-p\,E\left[\overline u^2\right]\right\} \\&=qm\left\{\sum_{i=1}^p \operatorname{Var}(u_i)-p\operatorname{Var}(\overline u)\right\} \\&=qm(p-1)\sigma^2_u \\&=(p-1)\left(qm\sigma^2_\alpha+m\sigma^2_\gamma+\sigma^2_\varepsilon\right) \end{align}

Or from general theory of normal distribution, you can directly say that

$$\frac1{\sigma_u^2}\sum_{i=1}^p (\overline y_{i\cdot\cdot}-\overline y_{\cdot\cdot\cdot})^2=\frac1{\sigma_u^2}\sum_{i=1}^p (u_i-\overline u)^2 \sim \chi^2_{p-1}$$ and hence find the expected sum of squares. Distributions of the other sum of squares and their expectations can be derived in a similar manner.

In this connection, there is a theorem that gives you a necessary-sufficient condition for a quadratic form of jointly normal variables to have a chi-square distribution. One version of this result is

Suppose $X\sim N_n(\mu,\Sigma)$ where $\Sigma$ is nonsingular and let $A$ be an $n\times n$ symmetric matrix. Then $X^TAX$ has a noncentral $\chi^2$ distribution if and only if $A\Sigma$ is idempotent (or equivalently $A\Sigma A=A$), in which case the degrees of freedom and noncentrality parameter are respectively $\operatorname{rank}(A)=\operatorname{rank}(A\Sigma)=\operatorname{tr}(A\Sigma)$ and $\mu^TA\mu$.