Calculating derivation of logarithm of summation of products

Question

I am trying to grasp the idea discussed in this paper. In the second section of this paper it calculates the derivative of (1) which results in equation (2). I cannot figure out how the derivative of the second term is calculated. This term is:

$\log \sum_c \prod_m p_m(c|\theta_m)$

and it's derivative as mentioned in the paper is:

$\sum_c \Big(p(c|\theta_1 \dots \theta_n) \times \frac{\partial \log p_m(c|\theta_m)}{\partial \theta_m}\Big)$

Please help me to figure this out. MOLi

Answer 1

The trick is to know the following: $$ \begin{equation} \begin{split} \dfrac{\partial\log\left(\sum_{c}f(x, c)\right)}{\partial x}&=\dfrac{1}{\sum_{c}f(x, c)}\cdot\sum_{c}\left(f(x, c)\dfrac{\partial\log\left(f(x, c)\right)}{\partial x}\right)\\&=\sum_{c}\left(\dfrac{f(x, c)}{\sum_{c}f(x, c)}\dfrac{\partial\log\left(f(x, c)\right)}{\partial x}\right). \end{split} \end{equation} $$

Now apply this to your formula: You have $f(x, c)=\prod_{m}p_m(c\mid\theta_m)$. Your variable is now $\theta_m$ not $x$.

$$ \begin{equation} \begin{split} \dfrac{\partial\log\left(\sum_{c}\prod_{m}p_m(c\mid\theta_m)\right)}{\partial \theta_m}&=\sum_{c}\left(\dfrac{\prod_{m}p_m(c\mid\theta_m)}{\sum_{c}\prod_{m}p_m(c\mid\theta_m)}\dfrac{\partial\log\left(\prod_{m}p_m(c\mid\theta_m)\right)}{\partial \theta_m}\right)\\&=\sum_{c}\left(p(c\mid\theta_1, \cdots,\theta_n)\dfrac{\partial\log\left(p_m(c\mid\theta_m)\right)}{\partial \theta_m}\right). \end{split} \end{equation} $$

Because:

• By equation $(1)$ in your paper, you get: $$ \dfrac{\prod_{m}p_m(c\mid\theta_m)}{\sum_{c}\prod_{m}p_m(c\mid\theta_m)}=p(c\mid\theta_1, \cdots,\theta_n). $$
• By knowing that $\dfrac{\partial\log\left(f(x)\cdot g(y))\right)}{\partial x}=\dfrac{\partial\log\left(f(x))\right)}{\partial x}$: $$ \begin{equation} \begin{split} \dfrac{\partial\log\left(\prod_{m}p_m(c\mid\theta_m)\right)}{\partial \theta_m}&=\dfrac{\partial\log\left(p_m(c\mid\theta_m)\prod_{q\neq m}p_q(c\mid\theta_q)\right)}{\partial \theta_m}\\&=\dfrac{\partial\log\left(p_m(c\mid\theta_m)\right)}{\partial \theta_m}. \end{split} \end{equation} $$

I hope this helps you.