Residual sum of squares and variance.

Question

So, from the ANOVA table one can get an output of residual sum of squares. Should the RSS be divided by degrees of freedom in ANOVA + 1 or should it be divided by df in ANOVA - 1? Because the residual variance formula is: $\frac{\left((\hat{y}_i - y_i) - \overline{(\hat{y}_i - y_i)}\right)^2 }{n-2}$. Thanks in advance.

Answer 1

The RSS in ANOVA should be divided by the number observations $n$ minus the number of groups $K$, i.e., $$ MSE = \frac{RSS}{n-K}, $$ the reason is that RSS is the weighted sum of the variances of each group. For every group you estimate its mean $\mu_j$ with $\bar{x}_{n_j\cdot}$, hence you "lose" one degree of freedom every estimator, i.e., $RSS = \sum_{j=1}^K (n_j-1) S^2_j$, where $S_j^2 = \frac{1}{n_j -1}\sum_{i=1}^{n_j}(x_{ij} - \bar{x}_{n_j\cdot})^2$