How to find a and b estimates in regression from least square error formula

49 Views Asked by At

In regression, we want to minimize $1/n \sum_1^n(y_i-(a+bx_i))^2 $. To minimize, we use gradients: $$\begin{align} \partial/\partial b = 0 \Rightarrow \sum_i^n x_i(y_i-(a+bx_i)) =0 \Rightarrow \sum_1^nx_iy_i-a\sum x_i-b\sum x_i^2=0 \end{align}$$ $$\partial/\partial a = 0 \Rightarrow \sum_i^n (y_i-(a+bx_i))=0 \Rightarrow \\ \sum y_i - \sum a -b\sum x_i = 0 \Rightarrow \\ \sum y_i - na -b\sum x_i = 0 $$

which eventually in the second equation we have $na$ which I can't find a way to properly get rid of to get to the formalization of $\hat b = \frac{\overline {XY} -\overline X \overline Y}{\overline {X^2} - \overline X^2}$ where $\overline {XY} ,$ $\overline X,$ and $\overline Y $ represent averages. Can someone explain where I am going wrong?

1

There are 1 best solutions below

3
On BEST ANSWER

your first equation can be written as $\overline{XY}-a\overline{X}-b\overline{X^2}=0$ and then second can be written as $-a=b\overline{X}-\overline{Y}$ plugging $-a$ into the first $\overline{XY}-(b\overline{X}-\overline{Y})\overline{X}-b\overline{X^2}=0\iff b(\overline{X}^2-\overline{X^2})=\overline{X}\overline{Y}-\overline{XY}$ as desired