Linear regression question

Question

I don't understand the following derivation:

$$ e_i = y_i - ax_i - b$$

$$ e_i = (y_i - \bar{y}) - a(x_i - \bar{x}) - (b - \bar{y} + a \bar{x}) $$

I don't really understand what they do and why they do it.

To clarify:

$e_i = y_i - \hat{y}_i$, where $ \hat{y}_i$ is the regression function I believe it's called.

Answer 1

Some of the notation is not fully standard, and in particular, you seem not to be distinguishing between the (unobservable) errors $e_i$ and the (observable) residuals $\hat e_i$.

The usual notation runs like this:

• $\bar x =$ the average of $x_i$, $i=1,\ldots, n$
• $\bar y =$ the average of $y_i$, $i=1,\ldots, n$
• $y_i = ax_i+b+e_i$, where $e_i$ is the $i$th error. The values of $a$ and $b$ are unobservable because you see only the sample $(y_i,x_i)$, $i=1,\ldots,n$ and not the whole population, and $e_i$ is unobservable because $a$ and $b$ are unobservable.
• $\hat a$ and $\hat b$ are the least-squares estimates of $a$ and $b$. The least-squares estimates are observable because you can compute them base on the sample $(y_i,x_i)$, $i=1,\ldots,n$. They satisfy $\bar y=a\bar x + b$, i.e. the least-squares line passes through the point that is the average of $(y_i,x_i)$, $i=1,\ldots,n$.
• $\hat y_i = \hat a x_i + \hat b =$ the $i$th "fitted value".
• $\hat e_i=y_i-\hat y_i=$ the $i$th residual, not to be confused with the $i$th error. The residuals $\hat e_i$ are observable whereas the errors $e_i$ are not. The residuals necsessarily satisfy the two linear constraints $\hat e_1+\cdots+\hat e_n=0$ and $x_1 \hat e_1+\cdots + x_n \hat e_n=0$, whereas the errors, on the other hand, are often taken to be independent.
• The "regression function" is $x\mapsto y=ax+b$, whereas the fitted value $\hat y_i$ is the value of the regression function when the input is $x_i$.

Answer 2

Sorry people, I asked to soon:

$$ e_i = (y_i - \bar{y}) - a(x_i - \bar{x}) - (b - \bar{y} + a \bar{x}) = y_i - \bar{y} - ax_i + a\bar{x} - b + \bar{y} - a\bar{x} = y_i - ax_i - b$$

They just want to rewrite it to $$ e_i = (y_i - \bar{y}) - a(x_i - \bar{x}) - (b - \bar{y} + a \bar{x}) $$ because that is per definition $$ v_i - au_i - (b - \bar{y} + a\bar{x})$$ and that simpler expression will help with finding the regression line.