differentiation of the log of the sigmoid function.

188 Views Asked by At

I want to differentiate the following function with respect to a $w$. $$ L(w) = \frac{1}{|S^+| |S^-|} \sum_{x^+ \in S^+} \sum_{x^-\in S^-} \log(1+\exp(-t))$$ where $t = f_w(x^+) - f_w(x^-)$ and $f_w(x) = \langle w.x \rangle$

My steps are as follows. But I think there is a mistake. Can someone tell me where the mistake is please?

$$ L(w) = \frac{1}{|S^+| |S^-|} \sum_{x^+ \in S^+} \sum_{x^-\in S^-} \log(1+ \exp(-t))\\ L(w) = \frac{1}{|S^+| |S^-|} \sum_{x^+ \in S^+} \sum_{x^-\in S^-} \log( \frac{1 + \exp(t)}{\exp(t)})\\ \frac{d L(w)}{dw_n} = \frac{1}{|S^+| |S^-|} \sum_{x^+ \in S^+} \sum_{x^-\in S^-} \frac{\exp(t)}{1+ \exp(t)} \Big(\frac{\exp(t).\exp(t) - \exp(t)(exp(t)+1)}{\exp(t)^2}\Big) (x_{n}^{+} - x_{n}^{-})\\ = \frac{1}{|S^+| |S^-|} \sum_{x^+ \in S^+} \sum_{x^-\in S^-} \frac{1}{\exp(t)+1} \times \frac{-\exp(t)}{\exp(t)}\times (x_{n}^{+} - x_{n}^{-})\\ = \frac{1}{|S^+| |S^-|} \sum_{x^+ \in S^+} \sum_{x^-\in S^-} \frac{\exp(-t)}{\exp(-t)+1}(x_{n}^{-} - x_{n}^{+}) $$

I seem to be having an extra minus mark in my derivation. because in the final step I should be getting $\frac{1}{|S^+| |S^-|} \sum_{x^+ \in S^+} \sum_{x^-\in S^-} \frac{\exp(-t)}{\exp(-t)+1}(x_{n}^{+} - x_{n}^{-})$

or is my answer correct? What am I doing wrong? Thanks!