Define the image of the projection matrix $Q:=X(X^{T}X)^{-1}X^{T}$

Question

I recently came across the following claim:

Let $X \in \mathbb{R}^{n \times p}$ ($p\leq n$) be a matrix with linearly independent columns. Then $Q:=X(X^{T}X)^{-1}X^{T}$ is a projection matrix onto $span\{x_{1},\ldots,x_{p}\}$ ($x_{i}$ is the i-th column of $X$).

The projection properties are easy to verify:

1. $Q$ is symmetric: $Q^{T}=(X(X^{T}X)^{-1}X^{T})^{T}=X(X^{T}X)^{-1}X^{T}=Q$
2. $Q$ is idempotent: $QQ=X(X^{T}X)^{-1}X^{T}X(X^{T}X)^{-1}X^{T}=X(X^{T}X)^{-1}X^{T}=Q$

Hence $P:\mathbb{R}^{n}\rightarrow \mathbb{R}^{n}, x \mapsto Qx$ is a projection.

However I do not see why the range of $Q$ is given by $span\{x_{1},\ldots,x_{p}\}$. Any suggestions?

Answer 1

Set $y:=X e_i$ and compute $Qy$ to obtain $x_i$ ($i$-th column of X) as an image vector.