Recently I learned about the discussion around the St. Petersburg paradox and how people try to explain why the calculated expected value differs so much from most people's intuition.
My question: Since the expected value for the St. Petersburg paradox is infinite the law of large numbers doesn't hold in that case. But doesn't that mean, that the expected value loses its meaning as what to expect in the long run and that we don't know where the average win will converge to?
The law of large numbers does not hold in that case, but you can prove a similar result. This is very much similar to finite and infinite limits: in the finite case, we know that there is a number where the average will converge, and in the infinite case, we know that the average will grow without limit.
Let $X_i$ be the $i$th outcome and let $\overline{X}_n = (X_1+X_2+\dots+X_n)/n$.
If the average of each $X_i$ is $\mu$, then we have the law of large numbers: $$ \forall \epsilon>0\quad \lim_{n\to\infty} \mathcal{P}(|\overline{X}_n -\mu | < \epsilon) = 1, $$ and $$ \mathcal{P}(\lim_{n\to\infty} \overline{X}_n = \mu ) = 1. $$
If the average is infinite, the above does not make any sense. But we can prove that $$ \forall M\quad \lim_{n\to\infty} \mathcal{P}(\overline{X}_n > M) = 1. $$
But doesn't that mean, that the expected value loses its meaning as what to expect in the long run and that we don't know where the average win will converge to?
We don't know where the average win will converge to, because it won't converge. But the expected value does not lose its meaning, because by knowing that the expected value is infinite, we are be able to know something else: that the average will grow without limit.