Forgive the undoubted ignorance of this question. I am out of my element both in probability and in nonstandard analysis.
A mathematically curious layperson friend recently had a conversation with me where he wanted a notion of "pick a random integer" (meaning uniformly random). I told him that there isn't room for this idea in the theory of probability as developed from the Kolmogorov axioms, because a countable probability space can't have atoms of equal probability, because $\sum 0 = 0$, but $\sum \alpha = \infty > 1$ for $\alpha >0$ by the archimedean principle.
He asked, why couldn't the probability of picking an individual integer be $1/\infty$? "I know infinity isn't a number but in this context it feels like it could be," he said.
If you want $\infty$ and $1/\infty$ to be numbers, well, this is what nonstandard analysis was made for, right? So, my question is this:
Has anybody tried to formulate probability theory with probabilities in a field of hyperreal numbers rather than in $\mathbb{R}$? If so, what issues come up? Does the theory have significant differences with the standard theory or does this notion not really change anything substantive? Basically, what was the outcome?
If this has been done, I would be most interested in a sort of executive summary, but I would also accept a reference. Thanks in advance.
Addendum: Just found this related question at MO.
To answer your question about infinity specifically, one fixes an infinite hypernatural $H$, and works with the collection $$\{1,2,\ldots,n-1,n,n+1,\ldots,H-1, H\}.$$ As you suggested, one can assign probability $\frac{1}{H}$ to the occurrence of each individual number in this collection. This is the basic idea behind using infinitesimals in probability. Such sets are called hyperfinite.
The basic idea of Brownian motion, namely moving an infinitesimal amount in a random direction infinitely many times, finds a literal interpretation in Robinson's framework. There are two approaches to this that are technically somewhat different: one developed by Peter Loeb and the other by Edward Nelson. Here are some recent sources:
Nonstandard Analysis for the Working Mathematician. Editors: Loeb, Peter A., Wolff, Manfred P. H. (Eds.) Springer 2015. See here;
Herzberg, Frederik S. Stochastic calculus with infinitesimals. Lecture Notes in Mathematics, 2067. Springer, Heidelberg, 2013. See here.