Multiple regression - how to calculate SSE given only $X^TX$ and $X^T y$

Question

Suppose we are given only A = $X^TX$ and B = $X^T y$, that is, we do not know X and y, how can we work out the SSE and other related error criterias?

I know $\beta = (X^TX)^{-1}X^Ty$, but that doesn't seem helpful.

Answer 1

No can do. You can get the predictions $\hat y$ using $$ \hat y := X\beta = X(X^TX)^{-1}X^Ty= Hy $$ where $H$ denotes the hat matrix $H:=X(X^TX)^{-1}X^T$. The hat matrix is symmetric and idempotent so the SSE can be written $$ \text{SSE} :=(y-\hat y)^T(y-\hat y) = y^T(I-H)^T(I-H)y=y^Ty - y^THy. $$ With the information you have, you can compute the rightmost term $y^THy=B^TA^{-1}B$, but from $A$ and $B$ alone it is impossible to determine $y^Ty$, and therefore SSE is undetermined. The reason is that you can obtain the same $B$ with different $y$ vectors.

Example: Suppose the design matrix is $$ X:=\left(\begin{matrix}1 &0\\0&1\\1 &1\end{matrix}\right),\qquad X^T=\left(\begin{matrix}1 &0&1\\0&1&1\end{matrix}\right). $$ Then the choices $ y:=\left(\begin{matrix}0\\1\\0\end{matrix}\right) $ and $ y:=\left(\begin{matrix}1\\2\\-1\end{matrix}\right) $ have different values for $y^Ty$ but both yield the same value for $$B:=X^Ty=\left(\begin{matrix}0\\1\end{matrix}\right).$$