Looking at ordinary least squares (OLS) in matrix form, let X be an n x k matrix where we have observations on k independent variables for n observations. One of the columns contains only ones (constant term). Let ε be an n x 1 vector of errors.
I think that $X'\epsilon = 0$ because $E[\epsilon] = 0$, but that $X'\hat{\epsilon}$ is not necessarily 0. Is this reasoning right? Is there a better way of thinking about this?
If $X$ has the all ones vector as a column, then $X' \hat{\epsilon}$ is indeed zero.
Essentially this boils down to the fact that $\hat{\epsilon} = y - \hat{y}$ is the distance between $y$ and $\hat{y}$, where $\hat{y}$ is the projection of $y$ onto the column space of $X$. Then $\hat{\epsilon}$ is orthogonal to any vector in the column space of $X$, which implies $X' \hat{\epsilon}=0$.
A tangible way to justify this is to note that for any solution $\hat{\beta}$ to the equation $X' X\beta = X' y$ (this comes from setting the gradient of $f(\beta)=\|y-X\beta\|^2$ to zero), we will have $X'\hat{\epsilon}=X'(y - X \hat{\beta}) = 0$.