How to make derivative when function has double summation

Question

I have a function which needs to be minimize, I know the basic idea of derivation and minimization but this expression killed me!

$min \sum_{j=1}^n\sum_{i=1}^n(x_{ij}w_j-w_i)^2$ s.t. $\sum_{i=1}^n w_i=1$ where $w_i \ge 0, i=1,2,...,n$

I proceed like this

$\partial L/\partial w_i = \partial (\sum_{j=1}^n\sum_{i=1}^n(x_{ij}w_j-w_i)^2)/\partial w_i +\partial \lambda (\sum_{i=1}^n w_i- 1 )]/\partial w_i$

I know partial derivative of last part is

$\partial \lambda (\sum_{i=1}^n w_i- 1 )]/\partial w_i =n\lambda $

but derivative of first part is really hard for me and I could not move even a little Thank you in advance

Answer 1

Thank you Micheal, using your suggestion here is what I got. $\frac{\partial\sum_j\sum_i (x_{ij}w_j-w_i)^2}{\partial w_k} =\frac{ \partial(\sum_{i,j : i \neq k, j\neq k} (x_{ij}w_j-w_i)^2 + \sum_{j: j \neq k} (x_{kj}w_j-w_k)^2 + \sum_{i:i\neq k} (x_{ik}w_k-w_i)^2 + (x_{kk}w_k-w_k)^2)}{\partial w_k}$

$= 0 -2 \sum_{j:j\neq k}(x_{kj}w_j-w_k)+0-2(x_{kk}w_j-w_k)$

$= -2 \sum_{j}(x_{kj}w_j-w_k)$

and from other way I got this solution

$ \frac {\partial \sum_{j=1}^n\sum_{i=1}^n(x_{ij}w_j-w_i)^2}{\partial w_i} = \frac { \sum_{j=1}^n \partial\sum_{i=1}^n(x_{ij}w_j-w_i)^2}{\partial w_i}$

$= -2 \sum_{j=1}^n \sum_{i=1}^n(x_{ij}w_j-w_i)$

I got two different result, did i make any mistake? in previous result, where my $w_i$ gone or can i simply change subsrcipt k to i? or later one has issue?

Answer 2

Note that in the expression \begin{align*} \sum_{j=1}^n\sum_{i=1}^n(x_{ij}w_j-w_i)^2 \end{align*} the variables $i$ and $j$ are bound indexvariables, which are visible only within the range of the summation symbol.

This invalidates your second approach, since the variable which is subject to differentiation must not be indexed by $i$ or $j$. Now the first approach in some detail.

Let $1\leq k \leq n$. We obtain \begin{align*} \frac{\partial}{\partial w_k}&\sum_{j=1}^n\sum_{i=1}^n(x_{ij}w_j-w_i)^2\\ &=\frac{\partial}{\partial w_k}\sum_{{j=1}\atop{j\neq k}}^n\sum_{{i=1}\atop{i\neq k}}^n(x_{ij}w_j-w_i)^2 +\frac{\partial}{\partial w_k}(x_{kk}w_k-w_k)^2\\ &\qquad+\frac{\partial}{\partial w_k}\sum_{{i=1}\atop{i\neq k}}^n(x_{ik}w_k-w_i)^2 +\frac{\partial}{\partial w_k}\sum_{{j=1}\atop{j\neq k}}^n(x_{kj}w_j-w_k)^2\\ &=0+(x_{kk}-1)^2\frac{\partial}{\partial w_k}w_k^2\\ &\qquad+\frac{\partial}{\partial w_k}\sum_{{i=1}\atop{i\neq k}}^n(x_{ik}w_k-w_i)^2 +\frac{\partial}{\partial w_k}\sum_{{j=1}\atop{j\neq k}}^n(x_{kj}w_j-w_k)^2\\ &=2w_k(x_{kk}-1)^2+2\sum_{{i=1}\atop{i\neq k}}^n(x_{ik}w_k-w_i)(x_{ik})+ 2\sum_{{j=1}\atop{j\neq k}}^n(x_{kj}w_j-w_k)(-1)\\ &=2w_k(x_{kk}-1)^2+2w_k\sum_{{i=1}\atop{i\neq k}}^nx_{ik}^2-2\sum_{{i=1}\atop{i\neq k}}^nx_{ik}w_i- 2\sum_{{j=1}\atop{j\neq k}}^nx_{kj}w_j+2(n-1)w_k\\ &=2w_k(x_{kk}-1)^2+2w_k\sum_{{i=1}\atop{i\neq k}}^nx_{ik}^2-2\sum_{{i=1}\atop{i\neq k}}^n(x_{ik}+x_{ki})w_i+2(n-1)w_k\\ &=2w_k(x_{kk}^2-2x_{kk}+n)+2w_k\sum_{{i=1}\atop{i\neq k}}^nx_{ik}^2-2\sum_{{i=1}\atop{i\neq k}}^n(x_{ik}+x_{ki})w_i\\ &=2w_kn+2w_k\sum_{{i=1}}^nx_{ik}^2-2\sum_{{i=1}}^n(x_{ik}+x_{ki})w_i\\ \end{align*}