Why is this simple partial derivative correct?

48 Views Asked by At

This example is taken from Geoffrey Hinton's Coursera on neural networks (seen at 13:15).

$$E=\sum_ip_i(t-y_i)^2,\\ \frac{\partial E}{\partial y_i}=p_i(t-y_i)$$

When I calculate this partial derivative I get:

$-2p(t-y)$

Does the sigma change how we calculate derivatives?

1

There are 1 best solutions below

0
On BEST ANSWER

The partial derivative is a linear operator, so we can always pull it inside a finite sum. Assuming your sum is finite, we have \begin{align*} \frac{\partial E}{\partial y_j} &= \frac{\partial }{\partial y_j} \sum_i p_i (t-y_i)^2 \\ &= \sum_i p_i \frac{\partial}{\partial y_j} (t-y_i)^2. \end{align*} Since \begin{equation*} \frac{\partial}{\partial y_j} (t-y_i)^2 = \begin{cases} 0, &\text{ if } i \neq j \\ -2 (t - y_j), &\text{ if } i = j, \end{cases} \end{equation*} we then have \begin{equation*} \frac{\partial E}{\partial y_j} = - 2p_j (t- y_j). \end{equation*} In general, the way I've done it (writing the derivative with a different index than the one in the sum and then pulling it inside the sum and computing the derivative by cases) is probably the best way to handle this. The screenshot you've posed seems incorrect to me, unless there's something I'm missing from the context.