differentiating covariance

75 Views Asked by At

$$\sum_{i=1}^n x_{i}^2C_{ii} + \sum_{i=1}^n \sum_{j=1 \atop{j\neq i}}^n x_{i}x_{j}C_{ij}$$ I am trying to differentiate the above expression with respect to $x_i$. I did partial differentiation to get $$2x_i C_{ii} + \sum_{j=1 \atop{j\neq i}}^n x_{j}C_{ij}$$. However the answer is given as $$2x_i C_{ii} + 2\sum_{j=1 \atop{j\neq i}}^n x_{j}C_{ij}$$.

Could I get assistance with the regards to the extra 2 before the summation and why it should be there?

1

There are 1 best solutions below

0
On BEST ANSWER

Note that

$$\frac{\partial x_p}{\partial x_i} = \delta_{ip} = \begin{cases}1, & i=p\\0,& i \neq p \end{cases},$$

and by relabeling the first index,

$$S= \underset{j \neq i}{\sum_{i=1}^n \sum_{j=1 }^n} x_ix_{j}C_{ij} = \underset{j \neq k}{\sum_{k=1}^n \sum_{j=1 }^n} x_kx_{j}C_{kj}$$

Hence,

$$\frac{\partial S}{\partial x_i} =\frac{\partial}{\partial x_i}\underset{j \neq k}{\sum_{k=1}^n \sum_{j=1 }^n} x_kx_{j}C_{kj} = \underset{j \neq k}{\sum_{k=1}^n \sum_{j=1 }^n} \frac{\partial (x_k x_{j })}{\partial x_i}C_{kj} \\ = \underset{j \neq k}{\sum_{k=1}^n \sum_{j=1 }^n} x_k\frac{\partial x_{j }}{\partial x_i}C_{kj}+ \underset{j \neq k}{\sum_{k=1}^n \sum_{j=1 }^n} x_j\frac{\partial x_{k }}{\partial x_i}C_{kj} = \underset{j \neq k}{\sum_{k=1}^n \sum_{j=1 }^n} x_k\delta_{ij}C_{kj}+ \underset{j \neq k}{\sum_{k=1}^n \sum_{j=1 }^n} x_j\delta_{ik}C_{kj} $$

For the second sum on the RHS, since $\delta_{ik} = 0$ if $k \neq i$, we have

$$\underset{j \neq k}{\sum_{k=1}^n \sum_{j=1 }^n} x_j\delta_{ik}C_{kj} = \underset{j \neq i}{\sum_{j=1 }^n}x_jC_{ij} $$

For the first sum on the RHS, since $\delta_{ij} = 0$ if $j \neq i$, we have

$$\underset{j \neq k}{\sum_{k=1}^n \sum_{j=1 }^n} x_k\delta_{ij}C_{kj}= \underset{k \neq j}{\sum_{j=1}^n \sum_{k=1 }^n} x_k\delta_{ij}C_{kj} = \underset{k \neq i}{\sum_{k=1}^n } x_kC_{ki} =\underset{j \neq i}{\sum_{j=1}^n } x_jC_{ji}$$

Thus,

$$\tag{*}\frac{\partial S}{\partial x_i} = \underset{j \neq i}{\sum_{j=1 }^n}x_jC_{ij} + \underset{j \neq i}{\sum_{j=1}^n } x_jC_{ji}$$

A covariance matrix is symmetric. Thus $C_{ij} = C_{ji}$ and (*) becomes

$$\frac{\partial S}{\partial x_i} = 2\underset{j \neq i}{\sum_{j=1 }^n}x_jC_{ij}$$