In a normal linear model (with intercept), show that if the residuals satisfy $e_i = a + \beta x_i$, then each residual is equal to zero.

Question

In a normal linear model (with intercept), show that if the residuals satisfy $e_i = a + \beta x_i$, for $i = 1\dots n$, where $x$ is a predictor in the model, then each residual is equal to zero.

I'm not really sure how to do this, I tried writing out $e_i = y_i - \hat{y}_i$; but I wasn't able to get anywhere.

Answer 1

Since your regression model has intercept, we can assume, the X matrix for the regression has the form $$ X = \begin{pmatrix} 1 & x_1^T \\ 1 & x_2^T \\ \vdots & \vdots \\ 1 & x_n^T \\ \end{pmatrix} $$

Note that the residual must be orthogonal to every vector in column space of $X$. This is because the predicted value, $\hat{Y} = P_X Y$ (where $P_X$ is the orthogonal projection matrix onto the column space of $X$), and hence the residual vector $e = Y - \hat{Y} = (I - P_X)Y $. So for any vector $c$ of appropriate dimension $$c^T e = (c^T - c^TP_X)Y.$$ Now if $c$ lies in the column space of $X$ then $$P_X c =c$$ or $$c^T = c^T P_X^T = c^T P_X \text{ since $P_X$ is symmetric}$$ and it follows for any $c$ in the column space of $X$ $$c^T e = 0.$$

Now, your condition implies $e$ itself lies in the column space of $X$ and hence must be orthogonal to itself, i.e., $e^Te = \|e\|^2 = 0$, i.e., $e = 0.$

Answer 2

If our linear model is given by

$$y_i=w_0+w_1x_i+e_i$$

we can substitute $e_i=\beta_0+\beta_1x_i$ to obtain $$y_i=w_0+w_1x_i+(\beta_0+\beta_1x_i)$$ $$\implies y_i=\left[w_0+ \beta_0\right]+\left[w_1+\beta_1 \right]x_i$$ $$\implies y_i=\tilde{w}_o+\tilde{w}_1x_i+\tilde{e}_i,$$

in which $\tilde{e}_i=0$. Hence, as the error is not random. The model can capture everything that is present in the error. Residual term $\tilde{e}_i\equiv 0.$