Let $A = \begin{pmatrix} 2 & 0 & 2 \\ 1 & 1 & 2 \\ 1 & 1 & 2 \\ 2 & 0 & 2 \end{pmatrix}$. Determine the least squares solution for $Ax = e_4.$
The least squares solution can be found from $A^TAx = A^Te_4$. This is problematic since $A^TAx = \begin{pmatrix}10&2&12\\ 2&2&4\\ 12&4&16\end{pmatrix}x = \begin{pmatrix}2\\ 0\\ 2\end{pmatrix}$, but $x$ cannot be solved from here since I cannot multiply the RHS by $A^{T^{-1}}$... What can I do here?
As the matrix
$$B=A^TA=\begin{pmatrix}10&2&12\\ 2&2&4\\ 12&4&16\end{pmatrix}$$ is not invertible, the solution of the least square problem is a linear subspace of dimension greater or equal than $1$. Indeed the kernel of $B$ is a linear line.
You can solve the system $$Bx = \begin{pmatrix}2\\ 0\\ 2\end{pmatrix}$$ to find this linear line.