Suppose we have some random phenomena. Is it true that any event concerning the phenomena has a fixed "correct" probability? That is, the correct probability is the relative number of occurrences of the event if the phenomena is repeated infinitely many times.
If that's the case, is it provable that the modern theory of probability will always compute this correct probability for any event? Let us just take the discrete probability space for now. Suppose we construct an arbitrary sample space, with intrinsic probability values for outcomes that are indeed the "correct" probability values in reality. These probabilities sum to unity. Suppose that through our various probability laws, we arrive at a theoretical probability for a more complicated event. Is it always the case that this theoretical probability coincides with the true probability?
Now, since each event has a "correct probability", given any sample space, we can assign intrinsic probability values that are necessarily "correct" in reality. The problem is that there can be multiple sample spaces. For instance, suppose we roll a 6-sided die. We can model the sample space as $\{1,2,3,4,5,6\}$ or as $\{\text{roll is odd}, 2, 4, 6\}$. We can roll the die infinitely many times and hence assign intrinsic probabilities of $\frac{1}{6}$ to each outcome of the former sample space and $\frac{1}{2}, \frac{1}{6}, \frac{1}{6}, \frac{1}{6}$ to each outcome of the latter respectively.
The question is, will each such sample space (with probabilities assigned "correctly" in accordance with reality) lead us to the same final theoretical (and hence correct) probabilities? Can we prove that any sample space leads us to the same result? For instance, consider the probability $P(\text{roll is 2}|\text{roll is odd or roll is 2})$. Through probability theory, we compute it is as $\frac{1}{6} \div (\frac{1}{6}+\frac{1}{6}+\frac{1}{6}+\frac{1}{6})=\frac{1}{4}$ in the former probability space and as $\frac{1}{6} \div (\frac{1}{2}+\frac{1}{6})=\frac{1}{4}$ in the latter. This clearly coincides, but will it work for any possible thinkable sample space (there are infinitely many)? Is this fact provable with mathematics?
Sorry about the long question, but in short, I would mainly like to find the answers for the bolded questions. If any of my assertions above are wrong, what exactly is provable with mathematics with regard to probability? It would be great if someone can enlighten me on these issues!
Euclid's axioms of geometry provide only one model, which works for most purposes in real life (building a pyramid, putting carpet in your house, etc.). If we need a different geometry for other purposes (e.g., mapping the globe, mapping the universe) we change axioms and try to make an appropriate non-Euclidean geometry. Maybe with more or less than 180 degrees in a triangle.
By contrast, Kolmogorov's axioms of probability accommodate many models for even very simple discrete sample spaces (rolling dice, drawing from urns, etc.). As long as probabilities assigned to all the distinct and countably many outcomes in the sample space add to 1, the axioms lead to consistent probabilities of compound events. If a particular probability assignment doesn't agree with someone's view of reality, he or she is free to make a new probability assignment that follows the axioms.
People use many different methods to arrive at practically useful probability assignments. Traditional assignments for fair dice games, balanced roulette wheels, draws from honest and well-shuffled card decks, etc. are usually taught first. Frequentist assignments made on the basis of data are also used (e.g., slightly more boys at birth, somewhat more women at age 65). Bayesian methods based on personal belief are also used. Consistently and carefully executed, many different methods can be used to arrive at a useful system of probabilities that is consistent with Kolmogorov's axioms for finding probabilities of compound events--and perhaps useful in a particular real-life situation.
Notes beyond your question: (1) For uncountably infinite sample spaces, technical problems arise. For example, one assigns probabilities only to a particular subclass of all possible events. (2) There have been explorations of the consequences of using only a finite version of the rule for adding disjoint events. This restricts the theorems that can be proved in uncountably infinite spaces, but enough results can be derived to please some people. (One reference: Dubins & Savage: How to gamble, if you must.)