I have a projection matrix: $P = A(A^tA)^{-1}A^t$ where A's columns are base of a vector space V.
I understand that the projection must be done on a previously orthonomalized base, but I do not understand how $Px$ does that.
How does P orthonormalize the base?
Actually $P$ is an operator so it doesn't orthonormalize any vector.
Yet, you can choose a orthonormal basis on $V$ to express the matrix of $P$ in that basis.
Actually we like to make orthogonal projection which are projection that project a vector onto orthogonal basis.
For example :
$$ P : f \in \mathcal{C}^0[0,1]\to \int_0^1 f(t)(1+\dfrac{1}{2}t) dt$$ an orthogonal projection from $\mathcal{C} ^0$ on $ Vect(1,\dfrac{X}{2}) $
$$ Q : f \in \mathcal{C}^0[0,1]\to \int_0^1 f(t)(1+t) dt$$ is a projection from $\mathcal{C} ^0$ on $ Vect(1,X) $ But not orthogonal because $(1,X)$ is not orthonormal for the scalar product $<f,g>=\int_0^1f(t)g(t)dt$
In your example you want $P$ to be orthonormal, so you have to make a Gram-Schmidt orthonomalization method on $A$ (which means the orthonormalization of the column vector of $A$ that are, as you mention here, a base of $V$.)
So let $R$ the orthonormalized matrix. $Q$ transition matrix from previous basis to $R$.
$$A=QRQ^{-1} \ \ RR^t=1 $$
$$P= QRQ^{-1}(QRQ^{-1}QR^{-1}Q^{-1})QR^{-1}Q^{-1}=Id$$
So $P$ must be the identity, that are the trvial orthogonal projection
For more precision you can look here :
Look here : https://mathworld.wolfram.com/Gram-SchmidtOrthonormalization.html and here: https://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process