Can we always write regression models Y = r(X) + e with E(e) = 0?

Question

I often read something like: all regression models can be written $$ Y = r(X) + e \quad \text{with } E(e) = 0 $$ I'm a little confused about this statement. Let $X,Y$ be two real-valued random variables. Assuming it exists, define $r(X) = E(Y|X)$. Define $e = Y - r(X)$. Then, by law of iterated expectations, you get: $$ E(e) = E(E(e|X)) = E(E(Y|X) - r(X)) = 0 $$ Fine. But we haven't done any modeling so far. We just used the definition of $r$, which carries no information. The act of modeling is assuming that $r \in \mathcal F$ where $\mathcal F$ is a specific set of functions. This is informative, but can be wrong. Imagine that I completely misspecify the model, e.g. I assume that $r$ belongs to the set $\mathcal F$ of linear functions, while $E(Y|X)$ (the true $r$) is highly non-linear. Can I still hope to write: $$ Y = r(X) + e \quad \text{with } E(e) = 0 \text{ and } r \in \mathcal F \text{ ?} $$

Answer 1

If you have a model where the error term has non-zero expected value, offsetting $r(X)$ by the appropriate constant gives you a model where the error term is centered. Here's to hoping your space $\mathcal{F}$ is at least expressive enough to adjust by a constant.

While the expected value is not the only measurement for the localization of a distribution, in most cases (especially when you're using the $L^2$ norm for your cost function) it doesn't make sense to be "off by a constant on average".