Find the OLS coefficient from a regression of y hat on X

Question

Let $\hat y$ be $\boldsymbol{X(X'X)^{-1}X'y}$. Find the OLS coefficient from a regression of $\hat y$ on $\boldsymbol X$.

I know I should start from SSE and then take the derivative to obtain an expression of $\beta$ but I do not know how to do with $\hat y$. Could you please do me a favor?

Answer 1

For the linear model $y=X\beta + \varepsilon$, the OLS estimator for $\beta$ is $\hat\beta:=(X^TX)^{-1}X^Ty$. This $\hat\beta$ is the coefficient vector from a "regression of $y$ on $X$". So if we wish to regress $\hat y$ on $X$, where $\hat y:=X(X^TX)^{-1}X^Ty$, then the coefficient vector is $$(X^TX)^{-1}X^T\hat y=(X^TX)^{-1}X^TX(X^TX)^{-1}X^Ty=(X^TX)^{-1}X^Ty=:\hat\beta.$$ That is, the coefficient vector is again $\hat\beta$. One consequence of this fact is that the predicted values under this second regression are the same as the $\hat y$ that we started with, and the residuals are all zero. In general if $y$ happens to be a linear combination of the columns of $X$ then regressing $y$ on $X$ will return the original $y$'s as predicted values.

Answer 2

You can understand this geometrically. The fitted vector $\hat y$ is simply the projection of $y$ onto the column space of $X$. Projecting the projection doesn't change anything so the OLS coefficient from regressing $\hat y$ on $X$ is the same as that from regressing $y$ on $X$.