Is the sum of residuals in the weighted least squares equal to zero?

4.3k Views Asked by At

So I know that in OLS, the sum of the residuals is equal to zero. This makes sense. I also know that given any slope parameter its possible to rescale the intercept to where the sum of the u will be equal to zero. This makes sense. But in weighted least squares we give a different weight to each observation based on the variance structure, so would this still be true?

Heres my general attempt to think about this: If we weight an observation less and its far from the regression line, this seems like it would make the sum not equal to zero. Could it be that the sum of residuals AFTER the weights are applied sums to zero? This would make more sense to me. The problem is that the Assumption that E[u|x]=0 still holds in WLS. How does all this work?

1

There are 1 best solutions below

7
On BEST ANSWER

In weighted linear regression models with a constant term, the weighted sum of the residuals is $0$.

Suppose your regression model seeks to minimize an expression of the form $$\sum_i \omega_i(y_i-Ax_i -B)^2$$

Here the $\{\omega_i\}$ are your weights. Set the partial in $B$ to $0$ and suppose that $A^*$ and $B^*$ are the minimum. Then we have: $$-2\sum_i \omega_i(y_i-A^*x_i-B^*)=0$$ Dividing through by $-2$ we see that the weighted sum of the residuals is $0$, as desired.