I have a poll with four answers (A,B,C,D) and response information about people who have taken that poll. I have created four models (one for each of the answers) in a one vs all. i.e. the model for answer A has answer A as a 1 and the rest of the responses as a 0, for the second model answer B is 1 and the rest are 0, and so on...
My question is if I predict a new response given a new responser's information, are all the probabilities guaranteed to equal 1? Written as:
$$P(A|X) + P(B|X) + P(C|X) + P(D|X) \stackrel{?}{=} 1$$
It should be noted that each of the models might have different levels of regularization. I have convinced myself that if all the models have full regularization or no regularization this will hold true. But I'm not sure in the general case (one test case I have does but I don't know if it's dumb luck)