How to simplify OLS formulas?

26 Views Asked by At

Let $\lbrace x_i,y_i\rbrace_{i=1}^n$ be a random sample. I am trying to simplify the following expression

$$\frac{\sum_{i=1}^n x_i y_i - n \bar{X}\bar{Y}}{\sum_{i=1}^n x_i^2 -n\bar{X}^2}$$ to show it equals $$\frac{S_{xy}}{S_x^2}$$

I've been manipulating this for almost 2 hours and haven't made any progress.

How can I simplify the first expression to yield the second?

2

There are 2 best solutions below

2
On BEST ANSWER

Hint. Observe that $$ \frac{\sum_{i=1}^n x_i y_i - n \bar{x}\bar{y}}{\sum_{i=1}^n x_i^2 -n\bar{x}^2} =\frac{\frac1n\sum_{i=1}^n x_i y_i - \bar{x}\bar{y}}{\frac1n\sum_{i=1}^n x_i^2 -\bar{x}^2}\tag1 $$ then we are left with proving that $$ S_x^2=\frac1n\sum_{i=1}^n \left(x_i -\bar{x}\right)^2=\frac1n\sum_{i=1}^n x_i^2 -\bar{x}^2 \tag2 $$ and that $$ S_{xy}=\frac1n\sum_{i=1}^n \left(x_i -\bar{x}\right)\left(y_i -\bar{y}\right)=\frac1n\sum_{i=1}^n x_iy_i -\bar{x}\bar{y}. \tag3 $$

Let's see how to prove $(2)$. We have $$ \begin{align} \frac1n\sum_{i=1}^n \left(x_i -\bar{x}\right)^2&=\frac1n\sum_{i=1}^n \left(x^2_i -2x_i\bar{x}+\bar{x}^2\right) \\\\&=\frac1n\sum_{i=1}^n x^2_i -2\bar{x}\frac1n\sum_{i=1}^nx_i+\frac1n\sum_{i=1}^n\bar{x}^2 \\\\&=\frac1n\sum_{i=1}^n x^2_i -2\:\bar{x}\times \bar{x}+\frac1n \times n\:\bar{x}^2 \\\\&=\frac1n\sum_{i=1}^n x^2_i -\bar{x}^2. \end{align} $$ Similarly one gets $(3)$.

Thus, from $(1)$ we obtain $$ \frac{\sum_{i=1}^n x_i y_i - n \bar{x}\bar{y}}{\sum_{i=1}^n x_i^2 -n\bar{x}^2} =\frac{\frac1n\sum_{i=1}^n x_i y_i - \bar{x}\bar{y}}{\frac1n\sum_{i=1}^n x_i^2 -\bar{x}^2}=\frac{\frac1n\sum_{i=1}^n \left(x_i -\bar{x}\right)\left(y_i -\bar{y}\right)}{\frac1n\sum_{i=1}^n \left(x_i -\bar{x}\right)^2}=\frac{S_{xy}}{S_x^2}.\tag4 $$

0
On

divide top and bottom by n.

this gives you

$\dfrac{\frac{1}{n}\sum x_iy_i-E[X]E[Y]} {\frac{1}{n}\sum x_i^2-E[X]^2}$

Now let's look at the defintion of Variance.

$\frac{1}{n}\sum (x_i-E[X])^2 =$$ \frac{1}{n}\sum (x_i^2-2x_iE[X]+E[X]^2)\\ \frac{1}{n}\sum x_i^2- E[X]\frac{1}{n}\sum 2x_i+E[X]^2\\ \frac{1}{n}\sum x_i^2- 2E[X]^2+E[X]^2)\\ \frac{1}{n}\sum x_i^2- E[X]^2$

Now perform similar algebra to show that: $\frac{1}{n}\sum (x_i-E[X])(y_i-E[Y]) = \frac{1}{n}\sum x_iy_i-E[X]E[Y]$