This one has most of us in class scratching our heads. I not looking for a full solution, just a way to get started. If I can get the form for P I think I can get it from there. Here's the problem:
I did find a solution online, but I didn't understand it, so it's not really useful to me. They started like this
But I don't understand how they got that nor how they used it in the rest of their solution. (Judging by the rest of the solution, I think that last isn't x^n but x^T ... I think - handwriting).
Can anyone offer some advice?
Consider a vector $y$ which doesn't belong to Col(X) then $Xw = y$ has not an exact solution. Thus we look for $Xw = \bar y$ where $\bar y$ is the projection of $y$ in $Col(X)$.
The error is $e=y-\bar y=y-Xw$ and it is miminized when $e$ is orthogonal to $Col(X)$ that is
$$X^Te=X^T(y-Xw)=0\implies X^Ty=X^TXw\implies w=(X^TX)^{-1}X^Ty$$
and therefore
$$\bar y=X(X^TX)^{-1}X^Ty=Py$$
and the projection is orthogonal since $P^T=P$.
Take also a look to this reference by MIT Projection.