Logistic Regression Optimization problem?

284 Views Asked by At

This is our optimization problem of Logistic Regression

$$\text{find optimal}\;(w) = argmax\;\;\sum_{i=1}^n\frac{1}{1+e^{-(y_iw^T.x_I)}}$$

I've read that sigmoid function is a monotonic function and it's differentiable.

Let's take log on the above optimization problem,

$$\Longrightarrow argmax\;\;\sum_{i=1}^n log\Big(\frac{1}{1+e^{-(y_iw^T.x_I)}}\Big)$$ $$\text{we know, }\;log \Big(\frac{1}{x}\Big) = -log(x)$$ $$\Longrightarrow argmax\;\;\sum_{i=1}^n - log(1+e^{-(y_iw^T.x_I)})$$ $$\Longrightarrow argmin\;\;\sum_{i=1}^nlog(1+e^{-(y_iw^T.x_I)})$$

After taking $log$ on our above optimization problem we've got this final alternative formulation for logistic regression.

$$\Longrightarrow argmin\;\;\sum_{i=1}^nlog(1+e^{-(y_iw^T.x_I)})$$

1) Why do we need to consider taking $log$ in the first place if our sigmoid ($\sigma$) function (i.e first formulation mentioned above) was already differentiable and can be used?