ANOVA is used to analyze differences within and between groups and was developed by the statistician Ronald Fisher. Essentially, ANOVA is a special case of the linear model.
Let's assume that for $I \geq 2$ groups, each with $J \geq 2$ observations, we have: $$ Y_{ij} = \mu + \alpha_i + \epsilon_{ij} \quad \text{for } i = 1, \ldots, I \text{ and } j = 1, \ldots, J $$ where $\epsilon_{ij} \sim N(0, \sigma^2)$ and the constraint $\sum_{i=1}^{I} \alpha_i = 0$ holds.
Furthermore, the following notations from the lecture are valid: $$ \begin{array}{l} \hat{\mu} = \bar{Y}_{\bullet\bullet} = \frac{1}{IJ}\sum_{i=1}^{I}\sum_{j=1}^{J} Y_{ij}, \quad \hat{\mu}_i = \bar{Y}_{i\bullet} = \frac{1}{J}\sum_{j=1}^{J} Y_{ij}, \quad \hat{\alpha}_i = \hat{\mu}_i - \hat{\mu} \\ S_G = J\sum_{i=1}^{I} \hat{\alpha}_i^2 = J\sum_{i=1}^{I}(\bar{Y}_{i\bullet} - \bar{Y}_{\bullet\bullet})^2 \quad \text{between-groups variance} \\ S_R = \sum_{i=1}^{I}\sum_{j=1}^{J} (Y_{ij} - \bar{Y}_{i\bullet})^2 \quad \text{within-groups variance} \end{array} $$
(a) Write the model in matrix notation, i.e., in the form $Y = X\beta + \epsilon$. How could you rewrite the model so that the design matrix has full rank?
(b) Show that in the model with full rank, $ \hat{\beta} = \left(\bar{Y}_{1}, \ldots, \bar{Y}_{I \bullet}\right)^{T} $. Hint: Consider $ \hat{\beta} = \left(X^{T} X\right)^{-1} X^{T} Y $.
Attempt/Idea:
(a) In matrix notation, the model can be represented as follows:
$$ \begin{pmatrix} Y_{11} \\ Y_{12} \\ \vdots \\ Y_{IJ} \end{pmatrix} = \begin{pmatrix} 1 & 1 & 0 & 0 & \dots & 0 \\ 1 & 1 & 0 & 0 & \dots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & 1 & 0 & 0 & \dots & 0 \\ 1 & 0 & 1 & 0 & \dots & 0 \\ 1 & 0 & 1 & 0 & \dots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & 0 & 1 & 0 & \dots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & 0 & 0 & 0 & \dots & 1 \\ 1 & 0 & 0 & 0 & \dots & 1 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & 0 & 0 & 0 & \dots & 1 \\ \end{pmatrix} \begin{pmatrix} \mu \\ \alpha_1 \\ \alpha_2 \\ \vdots \\ \alpha_I \end{pmatrix} + \begin{pmatrix} \epsilon_{11} \\ \epsilon_{12} \\ \vdots \\ \epsilon_{IJ} \end{pmatrix} $$
To rewrite the model in order for the design matrix to have full rank, we can omit one of the group indicators and use it as the reference group. This reduces the number of columns in the design matrix by one. The remaining group indicators are then encoded relative to the reference group.
Assuming we choose the first group as the reference group, we can rewrite the model as follows:
$$ Y = X\beta + \epsilon $$
where the design matrix $X$ and coefficient vector $\beta$ are given by:
$$ X = \begin{pmatrix} 1 & 0 & 0 & \ldots & 0 \\ 1 & 0 & 0 & \ldots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & 0 & 0 & \ldots & 0 \\ 1 & 1 & 0 & \ldots & 0 \\ 1 & 1 & 0 & \ldots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & 1 & 0 & \ldots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & 0 & 0 & \ldots & 1 \\ 1 & 0 & 0 & \ldots & 1 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 1 & 0 & 0 & \ldots & 1 \\ \end{pmatrix} \quad \beta = \begin{pmatrix} \mu \\ 0 \\ \alpha_2 \\ \vdots \\ \alpha_I \\ \end{pmatrix} $$
By making this transformation, the design matrix has full rank, as the remaining columns are linearly independent. The coefficients $\beta$ have the same interpretation as before, with $\alpha_1$ now implicitly included in the coefficients of the reference group.
Question:
Is the matrix from (a) chosen correctly so that you can continue with (b)? If so, how could you start with (b)?