For $H \in \mathbb{R}^{d\times d}$ & $H^T = H; HH=H$, then for any vector $v \in \mathbb{R}^d$ show that:

Question

Show that $Hv$ is the projection of v onto the column space of H

Answer: Let $S_H$ be the column space of $H$ and $X$ be the matrix that contains the column vectors that span $S_H$. Then the projection of $v$ onto $S_H$ can be found by:

$\rm proj_{S_H}v = X(X^TX)^{-1}X^Tv$. We can see that $X(X^TX)^{-1}X^T$ is

symmetric: $(X(X^TX)^{-1}X^T)^T = (X^T)^T(X(X^TX)^{-1})^T = X(X^TX)^{-1}X^T$.

and idempotent:

$X(X^TX)^{-1}X^T X(X^TX)^{-1}X^T = X(X^TX)^{-1}(X^TX)(X^TX)^{-1}X^T = X(X^TX)^{-1}X^T$
Given this, $H = X(X^TX)^{-1}X^T$ and $Hv$ is the projection of $v$ onto the column space of $H$

Answer 1

$H$ is indeed equal to $X(X^TX)^{-1}X^T$. They are indeed symmetric and idempotent and they are indeed the same projection onto the column space of $H$, but I don't think your proof is valid. Your argument essentially boils down to this:

1. Assume that $H$ is symmetric and idempotent.
2. Prove that $X(X^TX)^{-1}X^T$ is symmetric and idempotent projection onto the column space of $H$..
3. Claim (without proof) that $H=X(X^TX)^{-1}X^T$.
4. Conclude that $H$ is a projection onto the column space of $H$.

Actually, by definition, a projection matrix is an idempotent matrix. So, $H$ is assumed to be a projection right at the beginning. You don't need any proof. But I guess you want to show that $H$ is an orthogonal projection.

Steps 2 looks a bit pointless. $H$ is already assumed to be symmetric and idempotent. Why would you want to prove that it is equal to another symmetric idempotent matrix? Also, if you want to use the fact that $X(X^TX)^{-1}X^T$ is an orthogonal projection onto the column space of $X$ (which is proven in many textbooks), you should mention the relevant theorem, otherwise you haven't shown that $H$ is an orthogonal projection.

Step 3, although not difficult to prove, is less trivial. The mere fact that two matrices have the same column spaces doesn't mean that they are equal. You need to make use of the symmetry and idempotence.

Anyway, unless you are interested in expressing $H$ in terms of $X$, your whole proof seems to me a very roundabout attempt. The proof can be more succinct if one only wants to prove that $H$ is an orthogonal projection: $H$ is a projection because it is idempotent, and it is an orthogonal projection because for any vector $v$, its projected part $Hv$ is orthogonal to the residual part $v-Hv$: \begin{aligned} \langle Hv,\,v-Hv\rangle &=\langle Hv,v\rangle-\langle Hv,Hv\rangle\\ &=\langle Hv,v\rangle-\langle H^THv,v\rangle\\ &=\langle Hv,v\rangle-\langle H^2v,v\rangle\\ &=\langle Hv,v\rangle-\langle Hv,v\rangle=0. \end{aligned}