It is known that the axiom of choice is required in order to construct a non-measurable set. I believe (though I'm very far from understanding the details) that the existence of Solovay's model shows that there are alternative set theories (of the form ZF + other stuff) in which "every set is measurable" is a theorem; and moreover that you can have this as a theorem while still keeping at least some of the more useful parts of real analysis.
So, assuming I'm correct about the above: suppose things had developed differently, and we had developed probability theory in a set theory where we can still do calculus, but there are no non-measurable sets. My question is, what practical differences would this make for applied probability theory and related fields such as statistics and information theory?
Presumably you wouldn't need to specify a sigma algebra when defining a measure, and in my naïve imagination this would make it generally simpler to reason about probability distributions over infinite sets. But would there be any major consequences beyond that? Would any important probability-related theorems become false, or conversely, would there be new theorems that are false in ZFC but could be useful in applied probability theory? Or would the differences necessarily be only minor?
(Note that for the purposes of this question I'm only interested in the impacts on probability theory and related topics. For example, I don't care whether every vector space has a basis or not, unless it can be shown that this would have an impact on a result in information theory or statistics, etc.)