Intercept in linear regression

Question

If i centre the predictor variable by its mean, does this have any effect on the intercept?

E.g if I have $$y_i = \alpha + \beta(x_i-\bar{x}) + \epsilon_i$$

---

See I have centred it above, but does this mean the alpha has changed?

Answer 1

Suppose initially we have

$$y_i = \alpha + \beta x_i + \epsilon_i$$

then

$$y_i = (\alpha + \beta \bar{x}) + \beta( x_i - \bar{x}) + \epsilon_i$$

Hence the intercept will be $\alpha + \beta \bar{x}$, assuming that you only perform centering for the predictor variable.

Answer 2

Lets calculate the value of those coefficients for both cases, with and without centering

With this model you have $$Y = bX+a+\epsilon, \epsilon\sim N(0,1)$$

minimizing $E[(Y-(bX+a))^2]$ we find:

$\frac{\partial E[(Y-(bX+a))^2]}{\partial a} = -2E[Y-bX-a]\implies $$a'=E[Y]-bE[X]$ $\frac{\partial E[(Y-(bX+a))^2]}{\partial b} = -2E[YX-bX^2-aX]\implies b'=\frac{E[YX]-E[Y]E[X]}{E[X^2]-E^2[X]} = \frac{Cov(X,Y)}{Var(X)}$

(where $a'$ and $b'$ are the coefficients that minimize the expected squared difference, i.e. making the derivative equals zero)

$a'=E[Y]-bE[X]$

$b' = \frac{Cov(X,Y)}{Var(X)}$

---

with the model $$Y = b(X-\bar{X})+a+\epsilon, \epsilon \sim N(0,1)$$

minimizing $E[(Y-(b(X-\bar{X})+a))^2]$ we find:

$\frac{\partial E[(Y-(b(X-\bar{X})+a))^2]}{\partial a} = -2E[Y-b(X-\bar{X})-a]\implies $$a'=E[Y]-bE[X-\bar{X}] = E[Y]$

$\frac{\partial E[(Y-(b(X-\bar{X})+a))^2]}{\partial b} =-2E[(Y-b(X-\bar{X})-a)(X-\bar{X})] =$

$ -2\{E[Y(X-\bar{X})-b(X-\bar{X})^2+a(X-\bar{X})]\} = $

$-2\{E[Y(X-\bar{X})]-bE[(X-\bar{X})^2]+aE[(X-\bar{X})]\} = $

$-2\{E[YX]-\bar{X}E[Y]-bVar(X)+aE[X]-aE[\bar{X}]\} = $

$-2\{E[YX]-\bar{X}E[Y]-bVar(X)\} \implies b' = \frac{E[XY]-\bar{X}E[Y]}{Var(X)} $

$a'=E[Y]$

$b' = \frac{E[XY]-\bar{X}E[Y]}{Var(X)}$

I guess they changed (unless I made a mistake somewhere)