why $(t-Gw)'(t-Gw) = t't - 2t'Gw + t'G'Gt$

Question

I think this question was asked many times, but I can not find the answer. So I can deduct $(t-Gw)'(t-Gw) = (t' - (Gw)')(t-Gw) = t't - t'Gw - (Gw)'t + w'G'Gw$

I don't quite understand how $t'Gw$ becomes $t'Gw$, with vectors one can deduct that $x'y = y'x$ because it's an inner product and as a result one will have a number. But I can not prove it.

Thanks in advance.

Answer 1

Let $a_{n\times 1}$, $b_{p\times 1}$ and $G_{n\times p}$, hence \begin{align} S(b)&=(a-Gb)'(a-Gb)\\ &=a'(a-Gb)-(Gb)'(a-Gb)\\ &=a'a-a'Gb-b'G'a+b'G'Gb, \end{align} where $(Gb)'=b'G'$. Now, note that $a'Gb$ is a scalar because $(a')_{1 \times n}G_{n \times p}=c_{1\times p}$, thus $a'Gb = c_{1 \times p}b_{p \times 1}=d, d \in \mathbb{R}$. Hence, $d'=(a'Gb)'=b'G'a=d$, so \begin{align} S(b)&=a'a-a'Gb-b'G'a+b'G'Gb=a'a-2b'G'a-b'G'Gb. \end{align}