given the normal equations of the least squares estimator, how to recover X?

Question

Given the least squares normal equations

$\hat{\beta} = (X^TX)^{-1}X^Ty$

$X\hat{\beta} = X(X^TX)^{-1}X^Ty$

$\hat{y} = X(X^TX)^{-1}X^Ty$

and knowing $\hat{\beta}$, $y$, and $\hat{y}$, and that $y$ and $X$ are respectively of form

$(y_1 ... y_i)$

and

$\begin{pmatrix} 1 & x_1\\ ... & ...\\ 1 & x_i \end{pmatrix}$

I would like to recover X. As I understand from the many explanations about solving $Ax = B$ on the internet, it seems there is no single solution for X. However, is it possible to define the space containing $X$?

Answer 1

You can see that $X^T ( y - \hat{y} )= 0$. So the column space of $X$ is contained in the orthogonal complement of the column space of $y - \hat{y}.$