Least-Squares Derivation Looks Different

37 Views Asked by At

Why does my least squares solution look different ? Here, $y$ and $w$ are vectors, while $X$ is a matrix.

$ \frac{1}{2}(y - Xw)^T(y-Xw) = y^Ty - y^TXw - w^TX^Ty + w^TX^TXw $

$ \frac{d}{dw} = -y^TX + X^TXw = 0$

$w = (X^TX)^{-1}y^TX$

but the solutions I see are

$w = (X^TX)^{-1}X^Ty$

1

There are 1 best solutions below

2
On BEST ANSWER

Note that $$\frac{d}{dw}(a^Tw)=\frac{d}{dw}(w^T a)=a\neq a^T.$$ Consequently, $$\frac{1}{2}\frac{d}{dw}(y^Ty - y^TXw - w^TX^Ty + w^TX^TXw)=\frac{1}{2}(-X^T y-X^T y+2 X^T X w)=0\implies?$$