In an $n$-dimensional inner product space $V$, I have $k$ ($k\le n$) linearly independent vectors $\{b_1,b_2,\cdots,b_k\}$ spanning a subspace $U$. The $k$ vectors need not be orthogonal. Then I was told that the projection of an arbitrary vector $c$ onto $U$ is given by $$P_Uc=A(A^TA)^{-1}A^Tc$$ where $A$ is the $n\times k$ matrix with column vectors $\left((b_1)(b_2)\cdots(b_k)\right)$.
My questions are
(1) How to prove that it is the projection? (EDIT: which has been answered, thanks!)
(2) how can one be sure that the matrix $A^TA$ is invertible?
The image of $A$ is $U$ so any vector in $y\in U$ may be written as $y=Ax$ with $x\in {\Bbb R}^k$. The orthogonal projection of $c$ onto $y=Ax\in U$ is the one which minimizes the distance from $c$ to $U$, or equivalently such that $c-y$ is orthogonal to the image of $A$. This last condition may be written as: $$ A^T (c - y) = A^T(c - Ax)= 0 \ \ \Leftrightarrow \ \ x=(A^T A)^{-1} A^T c$$ from which you deduce $$ y= A x=A(A^T A)^{-1} A^T c$$
Suppose that there is $x\in {\Bbb R}^k$ so that $A^TAx=0$ whence so that $|Ax|^2 = x^T A^T A x=0$ so we must have $Ax=0$ but the columns in $A$ are linearly independent so $x=0$. The kernel of $A^TA$ is thus trivial and the (square) matrix is invertible.