Differentiating a specific cost function

Question

Consider the following cost function. $$J(x)=\sum_{i=1}^n [(x\,a_i+(1-x)\,b_i)-d_i]^2$$ How can I find the first and second derivatives of $J$, with respect to $x$? The variables $b_i$ are dependent on $x$, but $a_i$ and $d_i$ are independent of $x$, where $1\leqslant i \leqslant n$ and $0\leqslant x \leqslant 1$

Answer 1

Yes, we have to apply the chain rule for the first derivative:

$J(x)=\sum\limits_{i=1}^n ((x\cdot a_i+(1-x)\cdot b_i)-d_i)^2$

$J^{'}(x)=\sum\limits_{i=1}^n 2\cdot( (x\cdot a_i+(1-x)\cdot b_i)-d_i)\cdot (a_i-b_i)$

And the second derivative is

$J^{''}(x)=\sum\limits_{i=1}^n 2\cdot(a_i -b_i)\cdot (a_i-b_i)=2\cdot\sum\limits_{i=1}^n (a_i-b_i)^2$

Answer 2

Assuming that n is not being sent off towards infinity, then we can differentiate the sum. Since the sum is finite, we can expand the sum:

\begin{align*} \sum_{i = 1}^n ((xa_i + (1 - x)b_i) - d_i)^2 = (a_1x + b_1(1 - x) - d_1)^2 + \cdots + (a_nx + b_n(1 - x) - d_n)^2. \end{align*}

Then if we differentiate we have the following:

\begin{align*} \frac{d}{dx}\sum_{i = 1}^n ((xa_i + (1 - x)b_i) - d_i)^2 &= \frac{d}{dx}((a_1x + b_1(1 - x) - d_1)^2 + \cdots + (a_nx + b_n(1 - x) - d_n)^2)\\ &= 2(a_1x + b_1(1 - x) - d_1)(a_1 - b_1) + \cdots + 2(a_nx + b_n(1 - x) - d_n)(a_n - b_n)\\ &= \sum_{i = 1}^n 2(a_ix + b_i(1 - x) - d_i)(a_i - b_i). \end{align*}

Now by differentiating in the same manner, again we get \begin{align*} \frac{d}{dx}\sum_{i = 1}^n 2(a_ix + b_i(1 - x) - d_i)(a_i - b_i) &= \frac{d}{dx}(2(a_1x + b_1(1 - x) - d_1)(a_1 - b_1) + \cdots + 2(a_nx + b_n(1 - x) - d_n)(a_n - b_n))\\ &= 2(a_1 - b_1)(a_1 - b_1) + \cdots + 2(a_n - b_n)(a_n - b_n)\\ &= \sum_{i = 1}^n 2(a_i - b_i)^2. \end{align*}

If n is allowed to head off to infinity, this procedure may not necessarily hold.