I am trying to find the stationary points of the cross-entropy function for binary classification : $$ L(w) = -y \cdot \log(\sigma(wx)) - (1-y) \cdot \log (1-\sigma(wx)) $$ with $$ \sigma(wx) = \frac{1}{1+e^{-wx}} $$
If I develop and simplify I get: $$ L(w) = \log(1+e^{-wx}) + wx(1-y) $$
To find the stationary points in the close form I want to differentiate and solve for 0
$$ \frac{dL}{dw} = 0 $$ $$ \frac{dL}{dw} = \frac{-xe^{-wx}}{1+e^{-wx}} + x(1-y) = 0 $$ $$ \frac{xe^{-wx}}{1+e^{-wx}} = x(1-y) $$ $$ \frac{x}{1+e^{wx}} = x(1-y) $$ Here I start to face some problems:
- First, I am not sure that $x \neq 0$
- Second, $y \in (0;1)$, so obviously I am going to face some problems
What am I doing wrong and how can I solve this problem?
Thank you