Why can the function $f(x)=||A\vec{x}-\vec{b}||^2$ be rewritten as $\vec{x}^tA^tA\vec{x}−\vec{x}^tA^t\vec{b}−\vec{b}^tA\vec{x}+||\vec{b}||^2$

Question

Someone answered a question introducing this transformation of the function $f(x)=||A\vec{x}-\vec{b}||^2$ ; but I cannot get the idea why and how. Looks a bit like a binomial expansion, but I can't grasp the transpose multiplication.

Thanks in advance.

Answer 1

Note that for any real vector $v$, $\|v\|^2 = v^Tv$. So, now we have $$ \|Ax - b\|^2 = \\ (Ax - b)^T(Ax - b)=\\ (x^TA^T - b^T)(Ax - b) =\\ x^TA^TAx - x^TA^Tb - b^TAx - b^Tb =\\ x^TA^TAx - x^TA^Tb - b^TAx - \|b\|^2 $$ as desired.