Differentation of vector

Question

in the below equation, $\mathbf w$ is a vector with components $w_0$ and $w_1$. $x^{(i)}$ and $y^{(i)}$ are constants.

how to differentiate $j(\mathbf w)$ w.r.t. $w_0$ and $w_1$

$j(\mathbf w) = \frac{1}{10}\sum_{i=1}^5(x^{(i)}+w_1y^{(i)}-w_0)$

Answer 1

Putting $$ \nabla j(\mathbf{w})= \begin{pmatrix} \dfrac{\partial j}{\partial w_0}\\ \dfrac{\partial j}{\partial w_1} \end{pmatrix}, $$ if $j(\mathbf w) = \frac{1}{10}\sum_{i=1}^5(x^{(i)}+w_1y^{(i)}-w_0)$ we have $$ \begin{split} \nabla j(\mathbf{w})= \begin{pmatrix} \dfrac{\partial j}{\partial w_0}\\ \dfrac{\partial j}{\partial w_1} \end{pmatrix} &= \begin{pmatrix} \dfrac{\partial }{\partial w_0}\bigg[\displaystyle\frac{1}{10}\sum_{i=1}^5(x^{(i)}+w_1y^{(i)}-w_0)\bigg]\\ \dfrac{\partial}{\partial w_1}\bigg[\displaystyle\frac{1}{10}\sum_{i=1}^5(x^{(i)}+w_1y^{(i)}-w_0)\bigg] \end{pmatrix}\\ \\ &=\begin{pmatrix} \displaystyle\frac{1}{10}\sum_{i=1}^5(-1)\\ \displaystyle\frac{1}{10}\sum_{i=1}^5y^{(i)} \end{pmatrix}=\begin{pmatrix} \displaystyle-\frac{1}{2}\\ \displaystyle\frac{1}{10}\sum_{i=1}^5y^{(i)} \end{pmatrix}, \end{split} $$ while if $j(\mathbf w) = \frac{1}{10}\sum_{i=1}^5(x^{(i)}+w_1y^{(i)}-w_0)^2$ $$ \begin{split} \nabla j(\mathbf{w})= \begin{pmatrix} \dfrac{\partial j}{\partial w_0}\\ \dfrac{\partial j}{\partial w_1} \end{pmatrix} &= \begin{pmatrix} \dfrac{\partial }{\partial w_0}\bigg[\displaystyle\frac{1}{10}\sum_{i=1}^5(x^{(i)}+w_1y^{(i)}-w_0)^2\bigg]\\ \dfrac{\partial}{\partial w_1}\bigg[\displaystyle\frac{1}{10}\sum_{i=1}^5(x^{(i)}+w_1y^{(i)}-w_0)^2\bigg] \end{pmatrix}\\ \\ &=\begin{pmatrix} \displaystyle-\frac{1}{5}\sum_{i=1}^5(x^{(i)}+w_1y^{(i)}-w_0)\\ \displaystyle\frac{1}{5}\sum_{i=1}^5y^{(i)}(x^{(i)}+w_1y^{(i)}-w_0) \end{pmatrix}, \end{split} $$ The method is always the same: you should differentiate termwise each term of the sum. This is also true if you want to calculate mixed derivatives (that in this case are invariant respect to the order of differentiation) $$ \dfrac{\partial^2 j(\mathbf{w})}{\partial w_0\partial w_1}=\dfrac{\partial }{\partial w_0}\bigg[\dfrac{\partial j(\mathbf{w})}{\partial w_1}\bigg]=\dfrac{\partial }{\partial w_1}\bigg[\dfrac{\partial j(\mathbf{w})}{\partial w_0}\bigg]=\dfrac{\partial^2 j(\mathbf{w})}{\partial w_1\partial w_0} $$ or, if $(w_0,w_1)=\big(w_0(t),w_1(t)\big)$, you want to calculate the total derivative respect to $t$: $$ \dfrac{\mathrm{d} j(\mathbf{w})}{\mathrm{d}t}=\dfrac{\partial j(\mathbf{w})}{\partial w_0}\dfrac{\mathrm{d} w_0(t)}{\mathrm{d}t}+\dfrac{\partial j(\mathbf{w})}{\partial w_1}\dfrac{\mathrm{d} w_1(t)}{\mathrm{d}t} $$

Answer 2

The differential of the sum is the sum of differential.

Then comes a square, use the power rule $(u^2)' = 2 u u'$.