Why do we use logistic function to build logistic regression. I know the output value for logistic function bounds between 0 & 1, and bcos of this we can express as probability. Is this the reason we go for logistic function ?
Or is it because, if we plot the probabilities for an observation of being class 1 vs the observation, will form a S curve plot. And hence we use logistic function to model the behavior ? For example consider the case of credit card balance & default. If we plot the cumulative distribution of the probability of default given balance, then the plot we get is of shape S curve.
Could someone please explain this.
Regards,
Rohit
The flippant answer is that if we used another function for the regression, we wouldn't call it logistic regression. -- That's not entirely flippant, as logistic regression part of the broad class of "generalized linear models", which attaches a "link" function to the output of a linear regression model. This allows you to have non-linear fitting, but with computational tractability close to that of a linear model. For logistic regression this link function is the logit function, but you can certainly use other functions in a generalized linear model.
As you mention, the choice of logit (logistic regression) is due primarily to it's behavior of compressing the output between 0 and 1, with a smooth output between them. It is not, however, the only link function which does this. There are a number of other functions with this property, most notably the probit link function.
So why do so many people choose logistic regression, rather than some other generalized linear model? Well, that's mainly due to the interpretation of the output. For a logistic model, intermediate output values can be interpreted as the log odds ratio of being in the "1" category. Other link functions don't have such a clear probabilistic interpretation.