What's the correct representation of projection matrix between $UU^T$ and $U(U^TU)^{-1}U^T$

Question

I have seen those two representation of projection matrix $UU^T$ and $U(U^TU)^{-1}U^T$, where column of $U$ are orthonormal basis of subspace $S$. Which one is more accurate?

For projection matrix $UU^T$, $P_S(x)=UU^Tx$, then $x-UU^Tx$ is orthognal to any vector $y:=Uc\in S$, in the notes I read, it derive $||x-y||^2_2$ as: $$ \begin{eqnarray} ||x-y||^2_2&=&||x-UU^Tx+UU^Tx-y||^2_2 \\ &=& ||x-UU^Tx||^2_2+||UU^Tx-y||^2_2 \quad\textrm{By Pythagoras's theorem}\\ \end{eqnarray} $$

I couldn't solve the product of two vectors to zero. Here's how I derive the middle part:

$$ \begin{eqnarray} (x-UU^Tx)(UU^Tx-y)^T &=& (x-UU^Tx)(UU^Tx)^T-y^T(x-UU^Tx) \\ &=&(x-UU^Tx)x^TUU^T - 0 \\ \end{eqnarray} $$ Where did I go wrong?

--Edit--

1. For orthonormal columns, $UU^T=UIU^T=U(U^TU)^{-1}U^T$, two equations are the same.

2. As pointed out by @user1551 Dot product has a scaler result,so the equation should be: $$ \begin{eqnarray} (x-UU^Tx)^T(UU^Tx-y) &=& (x-UU^Tx)^T(UU^Tx)-(x-UU^Tx)^Ty \\ &=&(x^TUU^Tx-x^TUU^TUU^Tx)- 0 \\ &=& x^TUU^Tx-x^TUU^Tx = 0 \end{eqnarray} $$

Answer 1

Let $s$ be the dimension of $S$ and $U$ be a $n\times s$ matrix whose colums form an orthonormal basis of $S$. This is the same as saying that : $$U^T U = I_s$$ where $I_s$ is the $s\times s$ identity matrix.

Then, as OP shows, $P_S = U U^T = U (U^T U )^{-1} U^T$ is an $n\times n$ matrix and the orthogonal projection on $S$.

I couldn't solve the product of two vectors to zero.

Since $y \in S$, there is $c\in \mathbb K^s$ such that $y = Uc$. Then : \begin{align} (x - UU^T x )\cdot (y - UU^T x) &= (x^T - x^TUU^T)U(c - U^T x) \\ &= (x^T U - x^TU(U^TU))(c-U^Tx) \\ &= (x^T U - x^TU)(c-U^Tx) \\ &= 0 \end{align}

Now, if the columns of $U$ are not orthogonal, then $U^T U = I_s$ does not hold anymore and $UU^T$ is not a projection, since $(UU^T)^2 = U(U^TU) U^T \neq UU^T$.

Given any $s\times n$ matrix $V$ such that $VU = I_s$ you can check that $P_{S,V} = UV$ is a projection onto $S$ (but not necessarily orthogonal). It is orthogonal if, and only if, $P_{S,V}^T = P_{S,V}$, which rewrites as : \begin{align} V^T U^T =UV &\Leftrightarrow U^T = U^TUV \\ &\Leftrightarrow V = (U^TU)^{-1} U^T \end{align}

So the orthogonal projection onto $S$ is : $$P_{S,\perp} = U(U^TU)^{-1} U^T$$