Relation between projection matrix and linear span

879 Views Asked by At

Let $X$ be an $m \times n$ matrix. Define the orthogonal projection onto the column space of $X$ as

$$P(X):=X(X'X)^{-1}X'$$

Also, define the linear span of a set of $m\times 1$ column vectors of $X$ as $\mbox{span}(X)$.

I'm wondering what the relation between $P(X)$ and vector space $\mbox{span}(X)$ is. Can you help me with an intuitive explanation? Intuitive and easy interpretation would be appreciated.

1

There are 1 best solutions below

0
On

You have used the "regression" tag, so I assume that the context in linear regression.

The columns of design matrix $X$ form a vector space if there is no intercept in the model $Y= X\beta$, in a case of an intercept this is an affine space. The intuitive relation is that the hat matrix $H = X(X'X)^{-1}X'$ projects the $n$ dimensional response vectors $y$ into the space that is spanned by your explanatory variables. Namely, $Hy=\hat{y}$ gives you the "closest" vector that can be uniquely represented by a linear combination of the columns of $X$ (explanatory variables).