In $\mathbb{R}^4$, let $\underline{a} = (1, 2,-2, 0)$ , $\underline{b} = (2, 1, 0, 4)$ and $\underline{c} = (5, 7, 3, 2)$. I want to determine that the orthogonal projection of $\underline{c}$ on $< \underline{a}, \underline{b} >^\perp$ is $(2,4,5,-2)$.
We already know that $\underline{a}$ and $\underline{b}$ are orthogonal to this space, so they are precisely the vectors that we need. I tried using:
$$C_{proj}=c- \frac{a\cdot c}{|a|^2}a -\frac{b\cdot c}{|b|^2}b$$
And then I get $$(5,7,3,2)-\frac{13}{9}(1,2,-2,0)-\frac{25}{21}(2,1,0,4) \neq (2,4,5,-2)$$ Is there something wrong in my method of orthogonal projection? It would be very conventient if these fators would be $-1$ and $-1$ instead.
If $x$ and $y$ are orthogonal vectors and $S:=\mathrm{span}\{x,y\}$ is the subspace spanned by $x$ and $y$, then $$ (*)\quad \mathrm{proj}_{S^\perp}(z) = z - \frac{z\cdot x}{|x|^2}x - \frac{z\cdot y}{|y|^2}y $$ (where $|x|$ is the length of vector $x$) is the projection of the vector $z$ onto the subspace $S^\perp$.
In your case, $\underline{a}$ and $\underline{b}$ are not orthogonal, so we find another vector $\underline{d}\in \mathrm{span}\{\underline{a}, \underline{b}\}$ which is perpendicular to $\underline{a}$. $$ \mathrm{proj}_{\underline{a}^\perp}\left(\underline{b}\right) = \underline{b} - \frac{\underline{b}\cdot \underline{a}}{|\underline{a}|^2}\underline{a} $$ $$ = (2, 1, 0, 4) - \frac{(2, 1, 0, 4)\cdot (1, 2, -2, 0)}{|(1, 2, -2, 0)|} (1, 2, -2, 0) $$ $$ = \left(\frac{14}{9},\frac{1}{9},\frac{8}{9},4\right). $$ So, $\underline{d}=(14,1,8, 36)$ is in $S$ and it is perpendicular to $\underline{a}$. We can now apply formula (*) to compute the projection of $\underline{c}$ on to $S^\perp$. $$ \mathrm{proj}_{S^\perp}\left(\underline{c}\right) = \underline{c} - \frac{\underline{c}\cdot \underline{a}}{|\underline{a}|^2}\underline{a} - \frac{\underline{c}\cdot \underline{d}}{|\underline{d}|^2}\underline{d} $$ $$ = (5, 7, 3, 2)- 13/9 (1, 2, -2, 0) - 173/1557 (14, 1, 8, 36) $$ $$ = (2, 4, 5, -2). $$