Why Logistic Regression for Classification Problems?

Question

In a class on machine learning, we covered classification problems. In such a problem, you are studying a property of some object, say malignity of tumors in a patient. You are first given a training set, which consists of a set of ordered pairs $(x^{(1)}, y^{(1)}), (x^{(2)}, y^{(2)}), ... (x^{(n)}, y^{(n)})$, where each $x^{(i)}$ is a vector of desired parameters (in the tumor example, size could be such a parameter), while each $y^{(i)}$ is either zero or one depending on the status of this property (in the tumor example, one might say a tumor is benign if $y^{(i)} = 0$, and malign if $y^{(i)}=1$. By fitting a function to this training set, one finds a hypothesis function, which hopefully predicts whether tumors are benign or malign in the future. But how to fit such a function? The professor of the class gave an example that wouldn't work, linear regression. Linear regression was shown as a poor method because outliers in the training set would influence the hypothesis too drastically. Then the professor said that the better method for classification problems was logistic regression. However, he did not explain this -- from an observer's point of view, it seemed logistic regression was chosen ad hoc as a fitting method. Could someone please ex

Answer 1

Maybe your professor should ex, since he's the one who introduced the logistic regression? One (possibly weak) reason to use the logistic curve is: it is one of the simplest functions one can think of that increases from 0 to 1, always staying between 0 and 1. Thus, its value at $x$ can be interpreted as the likelihood of $x$ belonging to the "$y=1$" group.