I am looking at Simple Linear Regression where our model has two parameters, i.e. $$ E[Y|x_{i}] = b_{0} + b_{1}x_{i} $$
I am having trouble understanding where the idea that $$ \frac{SSR}{\sigma^{2}}\sim X_{1}^{2} \text{ and } \frac{SSRes}{\sigma^{2}}\sim X_{n-2}^{2} $$
I am going to start off laying out the knowledge that I already have and hopefully you can pick up where I am going wrong.
Here are some quick facts to know that make things easier.
- Let $ X\sim N(\mu,\sigma^{2}). $ Then $ \frac{X-\mu}{\sigma} \sim N(0,1) $
- Let $ Z\sim N(0,1).$ Then $ Z^{2} \sim X_{1}^{2} $
- Let $ C \sim X_{n_{1}}^{2} \text{ and } K \sim X_{n_{2}}^{2}. $ Then $ (C+K) \sim X_{n_{1}+n_{2}}^{2} $
- Degrees of Freedom are defined to be the sample size minus the number of estimated parameters.
- We assume $ \epsilon_{i}\sim N(0,\sigma^{2}) $ where $\epsilon_{i}$ are the unseen population errors
The first time degrees of freedom started to make sense as a concept was seeing Cochran's Theorem which says $$ \sum \left( \frac{X_{i}-\mu}{\sigma} \right)^{2} = \sum \left( \frac{X_{i}-\bar{X}}{\sigma} \right)^{2} + \left( \frac{\bar{X}-\mu}{\sigma/\sqrt{n}} \right)^{2} $$
$$ X_{n}^{2} = X_{?}^{2} + X_{1}^{2} $$
Of course, a (n-1) belongs where the question mark is. The algebra says something along the lines of "Deviation from the Population Mean = Deviation from the Sample Mean + Bias of $\overline{X}$ from $\mu.$"
Cochran's Theorem seems to be a specific instance of the Bias-Variance Tradeoff and shows the idea that calculating a sample estimator reduces the Degrees of Freedom.
Next, I see how $ SST = SSR + SSRes .$ So $$ \frac{SST}{\sigma^{2}} = \frac{SSR}{\sigma^{2}} = \frac{SSRes}{\sigma^{2}} $$ $$ \left( \frac{y_{i}-\overline{y}}{\sigma} \right)^{2} = \left( \frac{\hat{y_{i}}-\overline{y}}{\sigma} \right)^{2} + \left( \frac{{y_{i}}-\hat{y}_{i}}{\sigma} \right)^{2} $$
From Cochran's Theorem, we know that $ \frac{SST}{\sigma^{2}} \sim X_{n-1}^{2} $ but I have not been able to determine what degrees of freedom for $ \frac{SSR}{\sigma^{2}} $ or $ \frac{SSRes}{\sigma^{2}} $ are.
My next thought was to try messing with the Sum of Squared Errors. $$ \sum \left( \frac{y_{i}-E[Y|x_{i}]}{\sigma} \right)^{2} = \sum \left( \frac{\epsilon_{i}-E[\epsilon_{i}]}{\sigma^{2}} \right)^{2} = \frac{SSE}{\sigma^{2}} \sim X_{n}^{2} $$
Both in Cochran's Theorem and ANOVA, the technique of adding 0 is used. The value being adding must be a sampled estimator though. So I tried adding $ (\overline{y}-\overline{y}) $ and $ (\hat{y_{i}}-\hat{y_{i}}) $ into the SSE expression but this hasn't seemed to work.
So overall, I have two main questions:
- How do I show that $ \frac{SSR}{\sigma^{2}} \sim X_{1}^{2} $ or $ \frac{SSRes}{\sigma^{2}} \sim X_{n-2}^{2} $
- Are there any ideas worth exploring by partitioning $ \frac{SSE}{\sigma^{2}} $
EDIT:
Found the answer HERE and HERE
The answer is to decompose the Sum of Squared Errors where $$ \frac{SSE}{\sigma^{2}} = \frac{SSRes}{\sigma^{2}} + \left( frac{\overline{\epsilon}}{\sigma/\sqrt{n}} \right)^{2} + \left( \frac{\hat{b_{1}}-E[\hat{b_{1}}]}{\sigma\sum (x_{i}-\overline{x})^{2}} \right)^{2} $$
$$ \sum \left( \frac{\epsilon_{i}}{\sigma} \right)^{2} = \sum \left( \frac{\hat{\epsilon_{i}}}{\sigma} \right)^{2} + \left( \frac{\overline{\epsilon}}{\sigma/\sqrt{n}} \right)^{2} + \left( \frac{\hat{b_{1}}-E[\hat{b_{1}}]}{\sigma\sum (x_{i}-\overline{x})^{2}} \right)^{2} $$
$$ X_{n}^{2} = X_{?}^{2} + X_{1}^{2} + X_{1}^{2} $$
You first have to calculate $ \overline{\epsilon} $ and use the fact that $ \sum(\epsilon_{i}-\overline{\epsilon})^{2} = SSE - n\overline{\epsilon} $