How is this derivative calculated?

Question

I'm working through Python Machine Learning myself and am currently on the chapter on Training a logistic regression model. In that book, the author calculates the partial derivative of a sigmoid function:

$$\frac{\partial}{\partial{z}}\phi(z) = \frac{\partial}{\partial{z}}\left(\frac{1}{1 + e^{-z}}\right) = \frac{1}{(1 + e^{-z})^{2}}e^{-z} = \frac{1}{1 + e^{-z}}\left(1 - \frac{1}{1 + e^{-z}}\right)$$

I don't understand the last two steps, how:

$$\frac{1}{(1 + e^{-z})^{2}}e^{-z} = \frac{1}{1 + e^{-z}}\left(1 - \frac{1}{1 + e^{-z}}\right)$$

Could someone please explain?

Answer 1

It's easier to see if you do it right to left:

$$\frac{1}{1 + a}\left(1 - \frac{1}{1 + a}\right) = \frac{1}{1+a} - \frac{1}{(1+a)^2} = \frac{1+a}{(1+a)^2} - \frac{1}{(1+a)^2} = \frac{1+a-1}{(1+a)^2} = \frac{1}{(1+a)^2}a $$ with $a=e^{-z}$.

Answer 2

$\frac{1}{1+e^{-z}} = (1+e^{-z})^{-1}$

From here, it's just the chain rule and the power rule :)

The other part is just fraction decomposition.

Answer 3

$$ \frac{1}{(1+e^{-z})^2}e^{-z} = \frac{1}{1+e^{-z}} \frac{e^{-z}}{1+e^{-z}}, $$ and then $$ \frac{x}{1+x} = \frac{1+x-1}{1+x} = 1-\frac{1}{1+x} $$ with $x$ replaced by $e^{-z}$ gives $$ \frac{1}{1+e^{-z}} \frac{e^{-z}}{1+e^{-z}} = \frac{1}{1+e^{-z}} \left(1 - \frac{1}{1+e^{-z}} \right). $$

Answer 4

The derivative was already calculated via the chain rule at the third expression. The final equality on the RHS is established by writing $$\begin{align*} \frac{e^{-z}}{(1+e^{-z})^2} &= \frac{1}{1+e^{-z}} \left(\frac{e^{-z}}{1+e^{-z}}\right) \\ &= \frac{1}{1+e^{-z}} \left( \frac{(1+e^{-z}) - 1}{1+e^{-z}} \right) \\ &= \frac{1}{1+e^{-z}} \left( \frac{1+e^{-z}}{1+e^{-z}} - \frac{1}{1+e^{-z}} \right) \\ &= \frac{1}{1+e^{-z}} \left( 1 - \frac{1}{1+e^{-z}} \right). \end{align*}$$