Let $A$ a $n \times m$ matrix with $r(A)=m \le n$ and $b \in R^n$.
How can I prove that the normal equation system $$ A^TAx=A^Tb $$ has unique solution denoted by $u$.
If someone can help please.I do not know how to proceed this problem. Thanks for your time and help.
For any nonzero vector $v$ in $\Bbb R^m$, you have
$$v^TA^TAv=(Av)^T(Av)=||Av||_2^2$$
Since $A$ has rank $m$ by hypothesis (that is, full column rank), $Av$ can't be zero, hence $||Av||_2^2\neq0$ for all $v\neq0$, and your matrix $A^TA$ is positive definite (hence invertible).