What is the relationship between the function $\mathbb{E}(Y \mid X = x)$ and linear regression?

43 Views Asked by At

Consider the function

$$ r(x) = \mathbb{E}(Y \mid X = x) $$

This has been called the regression function in a textbook I'm using. I'm trying to figure out the relationship between this function and the classical linear regression model.

So, I know that it is a theorem* that we may write

$$ Y = r(X) + \epsilon $$

for some random variable $\epsilon$ s.t. $\mathbb{E}(\epsilon) = 0$.

Now suppose that we have

$$ Y = \beta_0 + \beta_1 X + \epsilon $$

This is the classical 1-dimensional regression function (assuming the $\beta_0$ and $\beta_1$ minimize the residual sum of squares).

Question: Is it then a mathematical theorem that if $Y$ is defined as above, that

$$ r(X) = \mathbb{E}(Y \mid X) = (\beta_0 + \beta_1 X)? $$

And is this why the function $\mathbb{E}(Y \mid X)$ is called the "regression function"?

EDIT: The theorem that I am making use of is as follows (from All of Statistics pg. 89):

Regression models are sometimes written as

$$ Y = r(X) + \epsilon $$

where $\mathbb{E}(\epsilon) = 0$. We can always rewrite a regression model this way. To see this, define $\epsilon = Y - r(X)$ and hence $Y = Y + r(X) - r(X) = r(X) + \epsilon$. Moreover, $\mathbb{E}(\epsilon) = \mathbb{E}\mathbb{E}(\epsilon \mid X) = \mathbb{E}(\mathbb{E}(Y - r(X)) \mid X) = \mathbb{E}(\mathbb{E} ( Y \mid X) - r(X)) = \mathbb{E}(r(X) - r(X)) = 0$.