I've been diving more into linear algebra recently, but there is something I can't get my head around when using least squares via linear algebra.
Having a plane, we have a line $a$, a point $b$ and a point $p$ which is the projection from $b$ to the line $a$. Also $p$ is the closest distance from $b$ to $a$. The error will be the distance $b-p$ so: $ e = b - p $. We consider $a^Te=0$ (perpendicular). So $a^T(b-xa)=0$ and in matrix form $A^T(b-Ax)=0$ which leads to $A^TAx=A^Tb$.
If I'm using the closest projection (perpendicular) to the "line":
¿Why are the vertical distances minimised and not the perpendicular to the line when resolving $A^TAx=A^Tb$?
¿Is there a way to know that before using these equations?
Thank you.