Find the least square solution $x$ for $Ax = b$ if $$A =\left(\begin{array}{rrr} 2 & 0\\ 1 & 3 \\ 0 & 2\end{array}\right) \quad b =\left(\begin{array}{rrr} 7\\ 0\\-1\end{array}\right) $$
My Solution:
I found $A^T A =\left(\begin{array}{rrr} 7 & -1\\ -1 & 8 \\ \end{array}\right) $
$$A^T b =\left(\begin{array}{rrr} 2\\ 2\\ \end{array}\right) $$
$$\left(\begin{array}{rrr} 5 & -1\\ -1 & 5 \\ \end{array}\right) x = \left(\begin{array}{rrr} 2\\ 2\\ \end{array}\right) $$
But I can't solve this equation to find $x$ as there is no solution.
My solution after finding $x$ was to then find this:
$$Ax = \left(\begin{array}{rrr} 2 & 0\\ 1 & 1 \\ 0 & 2\end{array}\right) \left(\begin{array}{rrr} x\\ x\\ \end{array}\right) = \left(\begin{array}{rrr} a\\ b\\c\end{array}\right) $$
and then use $b-Ax^ = \sqrt{1+0+1} - \sqrt{a^2 + b^2 +c^2} = \textrm{final answer}$
Where am I going wrong with my approach? I am stuck as I can't solve for $x$?
Your first error is in computing $A^\top b$. We have $$A^\top b = \begin{pmatrix} 2 & -1 & 0 \\ 0 & 1 & 2 \end{pmatrix}\begin{pmatrix} 1 \\ 0 \\ -1\end{pmatrix} = \begin{pmatrix} 2 \\ -2 \end{pmatrix}$$
So, we need to solve $$\begin{pmatrix} 5 & -1\\ -1 & 5 \end{pmatrix}\begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 2\\ -2 \end{pmatrix}.$$
I don't know where you went wrong specifically here, but there is a unique solution, as $A^\top A$ is invertible (compute its determinant to see this). We can use the old shortcut to compute the inverse of this matrix: $$\begin{pmatrix} a & b \\ c & d\end{pmatrix}^{-1} = \frac{1}{ad - bc}\begin{pmatrix} d & -b \\ -c & a\end{pmatrix}.$$ In particular, $$\begin{pmatrix} 5 & -1 \\ -1 & 5\end{pmatrix}^{-1} = \frac{1}{24}\begin{pmatrix} 5 & 1 \\ 1 & 5\end{pmatrix},$$ so $$\begin{pmatrix} x \\ y \end{pmatrix} = \frac{1}{24}\begin{pmatrix} 5 & 1\\1 & 5\end{pmatrix}\begin{pmatrix} 2\\-2\end{pmatrix} = \begin{pmatrix}\frac{1}{3} \\ -\frac{1}{3}\end{pmatrix}.$$ Therefore, the closest point to $(1, 0, -1)^\top$ in the columnspace of $A$ is $$A\begin{pmatrix}\frac{1}{3} \\ -\frac{1}{3}\end{pmatrix} =\begin{pmatrix} 2 & 0\\-1 & 1 \\0 & 2\end{pmatrix}\begin{pmatrix}\frac{1}{3} \\ -\frac{1}{3}\end{pmatrix} = \begin{pmatrix} \frac{2}{3} \\ -\frac{2}{3} \\ -\frac{2}{3} \end{pmatrix}.$$