Consider a two-way ANOVA design with interaction: $y_{ijk} = \mu + \alpha_i + \beta_j + \gamma_{ij} + \epsilon_{ijk}$, $i=1,...,I$, $j=1,...,J$, $k=1,...,n$, where $\epsilon_{ijk} \sim N(0,\sigma^2)$ are independent.
I am trying to find out appropriate expressions for $A_I$ and $A_M$ from $$ y^T(I - n^{-1}J)y = y^T(I-H)y + y^T A_I y + y^T A_M y $$ ,where $y^TA_My$ is the sum of squares due to the main effect, $H$ is a hat matrix of $X$, and $J$ is a matrix of one.
I figured out that above equation can be expressed as SSTotal = SSError + SSA + SSB + SSInteraction where $SSTotal = \sum_i \sum_j \sum_k (y_{ijk} - \bar{y})^2$, $SSError = \sum_i \sum_j \sum_k (y_{ijk} - \bar{y}_{ij})^2$, $SSA = \sum_i \sum_j \sum_k (\bar{y}_{i} - \bar{y})^2$, $SSB = \sum_i \sum_j \sum_k (\bar{y}_{j} - \bar{y})^2$, $SSInteraction = \sum_i \sum_j \sum_k (\bar{y}_{ij} - \bar{y}_i - \bar{y}_j + \bar{y})^2$ and $\bar{y} = \sum_i \sum_j \sum_k \frac{y_{ijk}}{nIJ}$, $\bar{y}_i = \sum_j \sum_k \frac{y_{ijk}}{nJ}$, $\bar{y}_j = \sum_i \sum_k \frac{y_{ijk}}{nI}$, $\bar{y}_{ij} = \sum_k \frac{y_{ijk}}{n}$.
My two questions:
- How could I show that cross terms are zero from
$$ SSTotal = \sum_i \sum_j \sum_k (y_{ijk} - \bar{y})^2 = \sum_i \sum_j \sum_k \Big((y_{ijk} - \bar{y}_{ij} )+(\bar{y}_i - \bar{y}) + (\bar{y}_j - \bar{y}) + (\bar{y}_{ij} - \bar{y}_i - \bar{y}_j + \bar{y}) \Big)^2 $$
I only show the $\sum_i \sum_j \sum_k (\bar{y}_i - \bar{y})(\bar{y}_j - \bar{y})=0$.
- How could I express the above equation by using matrix?
We have
$$y_{ijk}-\overline y=(\overline y_{i\cdot\cdot}-\overline y)+(\overline y_{\cdot j\cdot}-\overline y)+(\overline y_{ij\cdot}-\overline y_{i\cdot\cdot}-\overline y_{\cdot j\cdot}+\overline y)+(y_{ijk}-\overline y_{ij\cdot})$$
When we square both sides and sum over $i,j,k$, we get
$$TSS=SSA+SSB+SS(AB)+SSE$$
This is because cross-product terms vanish. For instance,
$$\sum_{i,j,k}(\overline y_{i\cdot\cdot}-\overline y)(\overline y_{\cdot j\cdot}-\overline y)=n\underbrace{\sum_i(\overline y_{i\cdot\cdot}-\overline y)}_{I\overline y-I\overline y}\sum_j(\overline y_{\cdot j\cdot}-\overline y)=0$$
Again,
\begin{align} \sum_{i,j,k}(\overline y_{i\cdot\cdot}-\overline y)(\overline y_{ij\cdot}-\overline y_{i\cdot\cdot}-\overline y_{\cdot j\cdot}+\overline y) &=\sum_{i,j,k}(\overline y_{i\cdot\cdot}-\overline y)\{(\overline y_{ij\cdot}-\overline y_{i\cdot\cdot})-(\overline y_{\cdot j\cdot}-\overline y)\} \\&=\sum_{i,j,k}(\overline y_{i\cdot\cdot}-\overline y)(\overline y_{ij\cdot}-\overline y_{i\cdot\cdot})-C_1 \\&=n\left[\sum_{i,j}\overline y_{i\cdot\cdot}\overline y_{ij\cdot}-J\sum_i\overline y^2_{i\cdot\cdot}-\overline y\underbrace{\sum_{i,j}\overline y_{ij\cdot}}_{IJ\overline y}+J\underbrace{\overline y\sum_i\overline y_{i\cdot\cdot}}_{I\overline y}\right]=0\,, \end{align}
because $$\sum_{i,j}\overline y_{i\cdot\cdot}\overline y_{ij\cdot}=\sum_i\left(\overline y_{i\cdot\cdot}\sum_j\overline y_{ij\cdot}\right)=J\sum_i\overline y^2_{i\cdot\cdot}$$
And,
\begin{align} \sum_{i,j,k}(\overline y_{i\cdot\cdot}-\overline y)(y_{ijk}-\overline y_{ij\cdot}) &=\sum_{i,j,k}\overline y_{i\cdot\cdot}y_{ijk}-n\sum_{i,j}\overline y_{i\cdot\cdot}\overline y_{ij\cdot}-\overline y\underbrace{\sum_{i,j,k}y_{ijk}}_{nIJ\overline y}+n\overline y\underbrace{\sum_{i,j}\overline y_{ij\cdot}}_{IJ\overline y}=0\,, \end{align}
since $$\sum_{i,j,k}\overline y_{i\cdot\cdot}y_{ijk}=\sum_{i,j}\left(\overline y_{i\cdot\cdot}\sum_k y_{ijk}\right)=n\sum_{i,j}\overline y_{i\cdot\cdot}\overline y_{ij\cdot}$$
Regarding the matrix form of different sum of squares, rewrite the ANOVA model as
$$\boldsymbol y^{m\times 1}=X^{m\times p}\boldsymbol \theta^{p\times 1}+\boldsymbol\varepsilon^{m\times 1}\,,$$
where $\boldsymbol y=(y_{ijk})_{i,j,k}$ and $\boldsymbol \theta=(\mu,\boldsymbol\alpha^T,\boldsymbol\beta^T,\boldsymbol\gamma^T)^T$ . Here, $m=nIJ$ and $p=1+I+J+IJ$.
Now, $$TSS=\boldsymbol y^T\left(I_m-\frac1m\boldsymbol1_m\boldsymbol1_m^T\right)\boldsymbol y \quad,\quad SSE=\boldsymbol y^T\left(I_m-H\right)\boldsymbol y$$
Note that $X$ does not have full column rank, so the hat matrix is written with a generalized inverse: $$H=X(X^TX)^{-}X^T$$
For the other sum of squares, I think you should have something like
\begin{align} SSA&=\boldsymbol y^T \left[\left(I_I-\frac1I \boldsymbol1_I\boldsymbol1_I^T\right)\otimes \left(\frac1{nJ}\boldsymbol1_{nJ}\boldsymbol1_{nJ}^T\right)\right]\boldsymbol y\,, \\ SSB&=\boldsymbol y^T \left[\left(\frac1{nI}\boldsymbol1_{nI}\boldsymbol1_{nI}^T\right)\otimes \left(I_J-\frac1J \boldsymbol1_J\boldsymbol1_J^T\right)\right]\boldsymbol y \,, \quad\text{ etc. } \end{align}
Here $\boldsymbol 1_n$ is an $n\times 1$ column vector of all-ones and '$\otimes$' is the Kronecker product.