Where is the square in the Least square regression method?

Question

I'm having a serious doubt in the least square regression problem. I guess its got to do with the notation of norm. Is the least square formulation $||b - \mathbf{A}x||^2$ or is it $||b - \mathbf{A}x||_2$. Or are they the same ?

Answer 1

Usually it is $||b - \mathbf Ax||_2$, but the problem of minimizing $||b - \mathbf Ax||_2$ is the same as minimizing $||b - \mathbf Ax||_2^2$. The notation $||b - \mathbf Ax||$ is short for $||b - \mathbf Ax||_2$ when it is understood that the norm being used is the $2$-norm.

Answer 2

Suppose $v=(a,b,c)$. Then $\|v\|$ and $\|v\|_2$ are both common notations for $\sqrt{a^2+b^2+c^2}$. If you minimize $\|v\|_2$ then you also minimize $\|v\|^2$, and vice versa.