Finding the minimum for this formula

Question

I think if we want to calculate $\min_x \sum_i (b_i - x)^2$, the answer should be the mean of $b_i$, right? Now if we add a weight to each term and make it $\min_x \sum_i w_i(b_i - x)^2$, what's the closed form solution for $x$ then?

Answer 1

It's just the weighted average $$ \sum_i w_i b_i $$ (I assume $\sum_i w_i=1$).

To see this, do some algebra, as follows. (For convenience we'll let $n$ be the number of terms, thereby sparing ourselves the need to repeatedly write $\sum\limits_i\ 1$.) $$ \begin{align} & {} \qquad \sum_{i=1}^n w_i(x - b_i)^2 = \sum_{i=1}^n w_i(x^2 - 2xb_i + b_i^2) = x^2 - \left(2x\sum_{i=1}^n (w_i b_i)\right) + \left(\sum_{i=1}^n w_i b_i^2\right) \\ \\ \\ & = \underbrace{\left(x^2 -2x\sum_{i=1}^n w_i b_i + \left(\sum_{i=1}^n w_i b_i \right)^2 \right)}_\text{a square} + \sum_{i=1}^n w_i b_i^2 - \left(\sum_{i=1}^n w_i b_i \right)^2\tag{completing the square} \\ \\ \\ & = \underbrace{\left( x - \sum_{i=1}^n w_i b_i \right)^2}_\text{a square} + \sum_{i=1}^n w_i b_i^2 - \left(\sum_{i=1}^n w_i b_i \right)^2. \end{align} $$ The term labeled "a square" is $0$ if $x =\text{the aformentioned weighted average}$, and is otherwise positive. In the succeeding terms, "${}\;x\;{}$" does not appear at all. Therefore the value of $x$ that minimizes the whole expression is the value of $x$ that minimizes the term labaled "a square", and that is the weighted average. the actual value of the minimum is then given by those last two terms.