Orthogonal projection onto Hyperplane

Question

Let $Ax = y$ be a linear system with $A \in \Bbb R^{n,m}, n<m$. Consider $H_i := \{x \in \Bbb R^{m}: A_ix = y_i\}, i \in \{1,\dots,n\}, y_i \in \Bbb R$ where $A_i$ denotes the $i$-th row of $A$. Now, regard $\mathcal{P}_{H_i}z := z - {A_i}^{\intercal} {y_i - A_i z \over \|A_i\|^2} $. This is apparently an orthogonal projection onto the hyperplane $H_i$ but I fail to show it. Any help is appreciated.

Answer 1

$Px$ is the orthogonal projection onto $H$ if and only if $Px\in H$ and $\langle x-Px,z-Px\rangle = 0$ for all $z\in H$.

If $Px = x + \frac{y_i-A_ix}{\|A_i\|^2}A_i^T$ (there must be a plus!), then $A_iPx = A_ix + \frac{y_i-A_ix}{\|A_i\|^2}A_iA_i^T = A_ix + (y_i-A_ix) = y_i$, hence $Px\in H_i$. Also, if $z\in H_i$, \begin{align} \langle x-Px,y-Px\rangle &= -\frac{y_i-A_ix}{\|A_i\|^2}\left\langle A_i^T,z-x-\frac{y_i-A_ix}{\|A_i\|^2}A_i^T\right\rangle\\ &= -\frac{y_i-A_ix}{\|A_i\|^2}\left(A_iz - A_ix - \frac{y_i-A_ix}{\|A_i\|^2}A_iA_i^T\right)\\ &= -\frac{y_i-A_ix}{\|A_i\|^2}\left(A_iz - A_ix - y_i + A_ix\right)\\ &= -\frac{y_i-A_ix}{\|A_i\|^2}\left(A_iz - y_i\right) = 0, \end{align} since $A_iz=y_i$. So, $P$ is indeed the orthogonal projection onto $H_i$.

Answer 2

Hint: To show it, it's enough to prove $\mathcal P_{H_i}(z)$ is $z$ if $z\in H_i$ and is $0$ if $z\perp H_i$ (i.e. if $z=\lambda A_i^T$).