least squares minimum

40 Views Asked by At

it's the age old least-squares problem:

Find the value of b for which the following sum has the minimum value (assuming that x and y are one-dimensional vectors of the same length):

$$ \sum_{i=1}^n (y_i - bx_i)^2 = min $$

This must have been asked before but I can't find it, would appreciate a redirect from somebody thanks

2

There are 2 best solutions below

6
On BEST ANSWER

Hint: Differentiate with respect to $b$ and equate to $0$.

4
On

If you know linear algebra, the LHS can be written as $$ (y-bx)^T(y-bx) $$ with $y=(y_0,y_1,\cdots y_n)^T$, $x=(x_0,x_1,\cdots x_n)^T$.

The minimization solution is given by $$ b=(x^Tx)^{-1}x^Ty $$

See this article for the general case.