derivation of softmax

75 Views Asked by At

This is the derivation of softmax in Bishop's PRML:

$$ln(\frac{u_k}{1-\sum_ju_j}) = n_k$$

"Which we can solve for $u_k$ by first summing both sides over k and then rearranging and back-substituting to give"

$$ u_k = \frac{exp(n_k)}{1+\sum_jexp(n_j)} $$

Unfortunately, I fail to grasp the process by which the second equation is derived from the first? Can someone please help point out the steps?

Thanks.

1

There are 1 best solutions below

2
On

Exponentiate before you sum, then add $1$. Rearrange to get $\sum_k u_k$, so you can normalise the conditions $u_k\propto\exp n_k$.