How to derive the orthogonal projection form $\mathbf P_H=\mathbf H (\mathbf H^H \mathbf H)^{-1}\mathbf H^H$

Question

I saw the formula below in this paper: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.331.6329&rep=rep1&type=pdf

It said Assume that

$\mathbf H$ is a complex matrix of size $n \times m $,having full column rank

$\mathbf S$ is a complex matrix of size $n \times t$,having full column rank

the well-known formula to build an orthogonal projection whose range is $<\mathbf H>$ is

$\mathbf P_H=\mathbf H (\mathbf H^H \mathbf H)^{-1}\mathbf H^H-----(a)$

Now if we examine the matrix $(\mathbf H \mathbf S)$,the orthogonal projection onto the linear subspace $<\mathbf H \mathbf S>$ is

$$ \mathbf P_{HS}=(\mathbf H \mathbf S)\begin{pmatrix} \mathbf H^H \mathbf H & \mathbf H^H \mathbf S\\ \mathbf S^H \mathbf H & \mathbf S^H \mathbf S\\ \end{pmatrix}^{-1}\begin{pmatrix}\mathbf H^H \\ \mathbf S^H\end{pmatrix} -------(b) $$

I want to ask is that how does the (a) come?because in my thinking,the orthogonal projection is like

$proj_\mathbf u(\mathbf v)=\frac{<\mathbf v, \mathbf u,>}{<\mathbf u, \mathbf u,>}\mathbf u$

Second,$\mathbf H$ become $\mathbf H \mathbf S$,so $H^H$ become\begin{pmatrix}\mathbf H^H \\ \mathbf S^H\end{pmatrix},i can also understand.However,why will the $(\mathbf H^H \mathbf H)^{-1}$ become $$ \begin{pmatrix} \mathbf H^H \mathbf H & \mathbf H^H \mathbf S\\ \mathbf S^H \mathbf H & \mathbf S^H \mathbf S\\ \end{pmatrix}^{-1} $$

Answer 1

Your formula for $proj_u$ is correct when you project along the line defined by $u$. Here we are projecting in a subspace spanned by the column of $H$. You start from the property that $v - proj_{H}(v)$ is orthogonal to any vector spanned by the columns of $H$: $$ H^T \left( v - proj_{H}(v) \right) = 0 $$ or $$ H^T v = H^T proj_{H}(v) $$ But $proj_{H}(v) = H x$ as it is spanned by the columns of $H$ $$ H^T H x = H^T v \quad \Rightarrow \quad x = (H^T H)^{-1} H^T v $$ And you get $$ proj_{H}(v) = H x = H (H^T H)^{-1} H^T v $$

Based on your dimensions, $\mathbf{HS}$ denotes $\left[ \mathbf{H}, \; \mathbf{S} \right]$ the matrix of dimension $n \times (m + t)$. The rest of the derivation is correct assuming that $\left[ \mathbf{H}, \; \mathbf{S} \right]$ remains full rank.

Answer 2

If $\ \mathbf{z}\ $ is any $\ n\times 1\ $ column vector, then $\ \mathbf{z}_P=\mathbf{P}_H \mathbf{z}\ $ is in the column space of $\ \mathbf{H}\ $, and

$$ \left\langle \mathbf{z}_P,\,\mathbf{z}- \mathbf{z}_P\right\rangle=\left\langle \mathbf{P}_H \mathbf{z},\,\mathbf{z}-\mathbf{P}_H \mathbf{z}\right\rangle = \left\langle \mathbf{z},\, \mathbf{P}_H\mathbf{z}-\mathbf{P}_H ^{\,2}\mathbf{z}\right\rangle = 0\ , $$ because $\ \mathbf{P}_H\ $ is hermitian, and $\ \mathbf{P}_H ^{\,2}=\mathbf{P}_H\ $. So $\ \mathbf{z}_P\ $ is the orthogonal projection of the vector $\ \ \mathbf{z}\ $ onto the column space of $\ \mathbf{H}\ $.

Your example of $\ proj_{\mathbf{u}}\left(\mathbf{v}\right)\ $ is just the special case of when $\ m=1\ $, and the column space of $\ \mathbf{H}\ $ is one-dimensional—spanned by $\ \mathbf{u}\ $.