Let $ w $ be a solution to $A^T Ax = A^Tb $. Which of the following statement is false?
$\quad $ (1) $ Aw - b $ is orthogonal to the column space of A.
$\quad $ (2) For any vector $ v \in R^n, ||Aw - b|| <= || Av - b|| $
$\quad $ (3) $ w $ is a least square solution to Ax = b, which always exists
$\quad $ (4) $ w $ must be the unique solution to $ A^TAx = A^Tb $
(4) is false, because the columns of A need to be linearly independent for w to be the only solution. Is my reasoning correct?
$A^T Aw = A^Tb$ $\Rightarrow$ $A^T(Aw-b)=0$
Let $A_1,...,A_n$ be columns of A, according to definition of transpose and matrix multiplication
$A^T(Aw-b)=\begin{pmatrix} <A_1,Aw,b> \\ \vdots \\ <A_n,Aw-b> \end{pmatrix}=\begin{pmatrix} 0 \\ \vdots \\ 0 \end{pmatrix}$
$<c_1A_1+...+c_nA_n,Aw-b>=c_1<A_1,Aw-b>+...+c_n<A_n,Aw-b>=0$