I have the following lineal regression matrix:
$$A\hat{x}=b$$ $$\begin{bmatrix} -2 &1 \\ -1 &1 \\ 0 &1 \\ 1 &1 \\ 2 &1 \\ \end{bmatrix} \begin{bmatrix}m \\b\end{bmatrix} = \begin{bmatrix}-4 \\-1 \\2 \\4 \\5 \end{bmatrix}$$ Normally, I would simply use the formula that states: $A^TA\hat{x}= A^Tb$
However, I realized that the columns of A where orthogonal, which meant that if I were to make them orthonormal, I could simply my equation to:·$\hat{x}=Q^Tv$ (because $QQ^T$ si equal to $I$) And so, I found each of A's column length to be: $\sqrt{10}$ and ${\sqrt{5}}$ respectively. And I all I have to do is divide each column by said constant. I simply left it alone for later in order to simply notation
And so later I mutliply: $A^T b$, which gets me to: $\begin{bmatrix}23 \\6\end{bmatrix}$. And all I have to is divide said matrix with the length, giving me: $\begin{bmatrix}\frac{23}{\sqrt{10}} \\\frac{6}{\sqrt{5}} \end{bmatrix}$.
However, the correct answer is: $\begin{bmatrix}\frac{23}{{10}} \\\frac{6}{{5}} \end{bmatrix}$.
Is my reasoning wrong? Am I wrong in doing: $\frac{v}{||v||}$ in order to find the orthonormal vector (give v is orthogonal to the other vectors in the set)?
PS: By $||v||$, I mean: $\sqrt{v_1^2 + (...) + v_5^2}$ in this case
There is indeed an arithmetic error embedded in your analysis. In this problem, in which the columns of $A$ are orthogonal (but not orthonormal), the matrix $A^t A =D$ is a square diagonal matrix (a standard concept whose definition is readily available elsewhere).
In fact the diagonal entries of $D$ are just the squares of the lengths of the columns of $A$ ( 5 and 10 in your example). Thus the equation $ A^t A\hat x= A^t b$ is easily solved: $\hat x= D^{-1} (A^t b)$. Your error can be traced to the fact that in effect you mistakenly used $\sqrt{D}$ instead.
Additional comments. In general, the least-squared error minimization problem of approximating a given column vector $b$ by an expression of the form $ A\hat x =\hat y$ can be interpreted as follows. The matrix product $A \hat x$ is simply a linear combination of the column vectors in $A$. The totality of all possible such expressions spans a linear subspace (the column space of $A$). The vector $\hat y$ that is the best or closest fit to $b$ can be obtained by orthogonal projection of $b$ onto this subspace. In order to compute that projected vector $\hat y$, one can if one pleases use Gram-Schmidt to first find a change of basis for that column space that results in an orthonormal basis. These orthonormal vectors can be organized as the columns of a matrix $O$. The fact that the columns of $A$ and $O$ are expressible as linear combinations of one another means simply that there exists a change of basis matrix $C$ (in your case $C$ is a 2x2 matrix) such that $A= OC$; hence ($O= A C^{-1}$.
Thus the equation $\hat y= A\hat x$ can be written as $\hat y= O (C \hat x)$. Thus the optimal solution vector $\hat y$ can be expressed as a linear combination of the orthonormal basis entries in $O$, if we take the 2 entries of the matrix product $C\hat x$ as the coefficients in this new basis. Because we have two different bases, we have two different ways to write the same vector $\hat y$. That is, the answer to one of your comments is Yes.
Another comment. Note that $ A^t A= (OC)^t (OC)= C^t O^t O C= C^t I C) = C^tC$. In your example you will discover that $C^tC= D$ is the diagonal matrix whose diagonal entries are $10$ and $5$. The matrix $C$ happens to be diagonal with entries $\sqrt{10}$ and $\sqrt{5}$. Thus your arithmetic error arose because you used $C$ rather than $D$.