Hi I'm looking for the least squares solutions of....
$$ \begin{pmatrix} 1&-1 \\ -1&2 \\ -1&0 \end{pmatrix} \begin{pmatrix} x\\y \end{pmatrix} = \begin{pmatrix} 3 \\ 4 \\ 5\end{pmatrix} $$
So Assuming this goes by $ A\vec{x}=\vec{b}$
Then using.... $A^T A \vec{x} = A^T \vec{b} $
$$ \begin{pmatrix} 1&-1&-1 \\ -1&2&0 \end{pmatrix} \begin{pmatrix} 1&-1 \\ -1&2 \\ -1&0 \end{pmatrix} \vec{x}= \begin{pmatrix} 1&-1&-1 \\ -1&2&0 \end{pmatrix} \begin{pmatrix} 3 \\ 4 \\ 5\end{pmatrix} $$
Which solves to be $$ \begin{pmatrix} 3&-3 \\ -3&5 \end{pmatrix} \vec{x} = \begin{pmatrix} -6 \\ 5 \end{pmatrix} $$
which i solve to get $$ \vec{x} = \begin{pmatrix} -7/3 \\ 3/2 \end{pmatrix} $$
Since you are only asking for the solution...
$$ \left( \begin{vmatrix} 1 & -1 \\ -1 & 2 \\ -1 & 0 \end{vmatrix}^\intercal \begin{vmatrix} 1 & -1 \\ -1 & 2 \\ -1 & 0 \end{vmatrix} \right) \begin{pmatrix} x \\ y \end{pmatrix} = \left( \begin{vmatrix} 1 & -1 \\ -1 & 2 \\ -1 & 0 \end{vmatrix}^\intercal \begin{pmatrix} 3 \\ 4 \\ 5 \end{pmatrix} \right) $$
$$ \begin{vmatrix} 3 & -3 \\ -3 & 5 \end{vmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} -6 \\ 5 \end{pmatrix} $$
$$ \begin{pmatrix} x = -\frac{5}{2} \\ y = -\frac{1}{2} \end{pmatrix} $$