I well know that in standard probability theory there is no way to define a uniform probability distribution over the natural numbers in the strict sense, but nonetheless, I'm trying to form an intuitive understanding of how such a distribution ( here I'm appealing more to an intuitive idea of probability than the formal one ) should behave, i.e. what should I expect to see, in an aggregated and structural sense, if I were to extract independently an infinite sequence of numbers using this distribution.
My intuition says that if I restrict myself to sequences of ordinal length $\omega$ (simple infinite sequences) then I should expect to observe nothing but pure chaos, with no regularity whatsoever. But if I were to keep extracting (independently, again in a not-quite-formal sense) numbers to obtain a sequence of ordinal length $\omega_1$, let's call it $\langle x_\alpha\rangle_{ \alpha < \omega_1}$, then great regularity would emerge. In particular, I would almost surely expect to see every natural number appearing in this uncountable sequence, and moreover, I would expect every natural number to appear cofinely many times (i.e. $\forall n \in \omega \ \forall \alpha < \omega_1 \ \exists \beta \in [\alpha, \omega_1)$ s.t. $x_\beta = n$). Also there would be no way to distinguish two numbers based on their appearances in the sequence (here the uniform character of the distribution).
Hoping that what I wrote is not too vague, my questions are:
- Is this intuition reasonable and acceptable in some way?
- Have some attempts been made to formalize this intuition?
More in general I'm getting interested in a critical understanding of what are the true reasons (of various nature) behind the fact that we could not develop a rich theory of probability comprising these "pathological" distributions and also other fringe notions like infinitesimal probabilities (I think these two concepts are intimately related). Is there some literature in this regard? I'm not asking for references to the thousands of non-standard probability theories available, but rather for a study of why these non-standard theories, which in general try to formalize some of the notions written above, all have serious and non-overseeable flaws that make them of no "practical" use, as if there are some structural and profound difficulties in formalizing these notions.
Thanks!
EDIT1: I know that by dropping countable additivity we could define a uniform distribution over the naturals, but dropping such an axiom has a serious (I'd dare to say devastating) impact on the mathematical power of the theory (an example of the flaws I was talking about). Also it does not shed any light on the intuition I wrote above.
EDIT2 (in response to Noah Schweber):
Regarding the first comment: I know that bringing the transfinite into the picture introduces serious difficulties (and I have no precise idea how to deal with it) but to me it is not simply another layer of difficulty, rather it is the necessary layer at which we are able to say something relevant about a possible uniform distribution over the naturals. More specifically I would expect that the "measure" on $\mathbb{N}^{\omega_1}$, once we formalize this uniform distribution over $\mathbb{N}$, should stem from a product-measure-like construction, as it is usually done in standard probability theory.
Regarding the second comment: I didn't say that dropping countable additivity is unnatural, what I say is that by dropping it we'd lose a lot of the theory's mathematical power, i.e. we could develop only very small fragments of Lebesgue's integration theory and would not be able to achieve even the most basic results in stochastic process' theory. And even if we were not interested in such theories, I don't see how we could say anything relevant regarding the uniform distribution over the naturals besides giving it a semblance of formal definition, in particular the intuition about the cofinality of all naturals in the $\omega_1$ sequence (which seems to me a critical property of such distribution) would not be even formalizable, let alone assessable.
I don't know if this helps you but although you cannot define a uniform probability measure in a countable set (easy to prove), you can define a uniform finitely additive probability measure on the naturals: Take a set $A\subset \mathbb N$, and the counting sequence of $A$ defined by $p_A^n=\#\{x\in A:a\leq n\}$, the measure would be $\mu(A)=\lim_n\frac{p_A^n}{n}$, you can easily see that this is finitely additive and $\mu(\{n\})=0$ for every $n\in\mathbb N$.
This is also a very intuitive measure, for example finite sets have measure $0$ and $\mu(n\mathbb N)=\frac{1}{n}$