Generally the normal equation is derived using a calculus-based approach of minimizing the least squares error. I'm trying to learn the linear algebra approach. My understanding so far is as follows:
Consider the standard linear regression problem: $Ab=y$ or $\sum_iA_{(i)}b^i=y$, where $A$ is an $m\times n$ matrix with $m>n$, and $A_{(i)}$ is the $i$-th column of $A$. Assuming $A$ to be full rank, its columns form a basis $C$ of an $n$-dimensional subspace of $\mathbb{R}^m$.
We already know the standard $\mathbb{R}^m$ basis representation of $y$ and we are trying to find its representation in the $C$ basis, such exact representation is possible to find only if $y\in\text{span}(C)$. Most likely $y$ won't lie in $\text{span}(C)$. In that case, the most natural alternative would be to find the $C$-representation of the next best thing - the projection of $y$ in $\text{span}(C)$. In other words, I need to find $$\hat b^i=\frac{\langle y,A_{(i)}\rangle}{\|A_{(i)}\|}$$ Now if only I had a matrix $\tilde A$ whose columns are the normalized versions of the columns of $A$, i.e., $\tilde A_{(i)}=A_{(i)}/\|A_{(i)}\|$, then I can just write $\hat b=\tilde A^Ty$. The question is: is it possible to get $\tilde A$ from multiplication/transpose/inverse operations performed on $A$?
Multiplying $A$ on the right by a diagonal matrix scales the columns of $A$. More concretely,
$$AD=\bar{A}$$
where $D$ is the diagonal matrix such that $D_{ii}=1/\lVert A_{(i)}\rVert$.