Why do we ask for *absolute* convergence of a series to define the mean of a discrete random variable?

Question

If $X$ is a discrete random variable that can take the values $x_1, x_2, \dots $ and with probability mass function $f_X$, then we define its mean by the number $$\sum x_i f_X(x_i) $$ (1) when the series above is absolutely convergent.

That's the definition of mean value of a discrete r.v. I've encountered in my books (Introduction to the Theory of Statistics by Mood A., Probability and Statistics by DeGroot M.).

I know that if a series is absolute convergent then it is convergent, but why do we need to ask for the series (1) to converge absolutely, instead of just asking it to converge? I'm taking my introductory courses of probabilty and so far I haven't found a situation that forces us restrict ourselves this way.

Any comments about the subject are appreciated.

Answer 1

It's because if the series is convergent but not absolutely convergent, you can rearrange the sum to get any value. Any good notion of "mean" or "expectation" should not depend on the ordering of the $x_i$'s.

For a more abstract reason, note that we define the expectation $E[X]$ of a random variable $X$ defined on a probability space $(\Omega, \mathcal{F}, P)$ as the Lebesgue integral $\int_{\Omega} X dP$. By definition of the Lebesgue integral, this is only well-defined if the integrand is absolutely integrable. If you learn more about measure theory, you will also learn why this definition makes sense. It is done to avoid strange situations like $\infty - \infty$ in the theory.

Answer 2

I think any explanation is going to make reference to the fact that without absolute convergence, the value of an infinite sum or an improper Riemann integral depends on the order in which the "pieces" are summed up. That alone may satisfy you, but it didn't fully satisfy me.

The more specific reason that satisfied me is "without absolute convergence the law of large numbers fails". The intuitive reason for this is that when you're taking sample averages, instead of integrating $x f(x)$ symmetrically, you're integrating it by Monte Carlo integration, literally picking locations randomly. As a consequence, if the integral for the mean only converges conditionally, then there is no guarantee that the sample averages have the same behavior along different sequences of samples, or even that the sample averages converge at all.

To see this on a computer, try running a program like this, which takes successive sample averages from the standard Cauchy distribution (which is symmetric about $0$, so its mean "would be zero if it made sense").

n=1e4;
x=pi*(rand(1,n)-1/2);
y=cumsum(tan(x))./(1:n);
plot(1:n,y)

This program as is will run in Matlab or Octave, but very similar programs can be run in other software with support for random numbers and plotting. What you see is quite dramatic jumps in the sample mean that occur when an entry of x gets too close to $\pi/2$ or $-\pi/2$, and which continue to occur even after thousands of samples have been drawn.

Answer 3

It's a great question, but unfortunately, I'm going to give an advanced probability/a measure theory answer that answers for random variables in general (OP's original title is actually about random variables. It was actually me, who edited it to discrete). However, I really can't imagine an elementary answer that consists of something more than simply 'Because' (or in Tagalog/Filipino 'Kasi'. I guess this is more precise because Tagalog/Filipino also has a word 'dahil' for a different kind of 'because').

Elementary probability peeps (including OP, I guess) may ignore this, except possibly the 'last part'. Or maybe elementary probability peeps can try to skim this and try to get an idea or something instead of understanding the specific details:

Defining expectations of random variables (or Lebesgue integrals of measurable functions) is in steps defined in the standard machine:

1. Expectation of indicator random variables
2. Expectation of simple random variables
3. Expectation of nonnegative random variables
4. Expectation of general random variables.

After 1 and 2, we get to 3. Let $X$ be a random variable. Here, we can define $E[X^{+}]$ and $E[X^{-}]$ for the nonnegative random variables $X^{+}$ and $X^{-}$, where $X^{+}$ and $X^{-}$ are something like, resp, 'the nonnegative part of $X$' and the negative of the 'nonpositive part of $X$' s.t. $X = X^{+} - X^{-}$. Then, given $|X| = X^{+} + X^{-}$, we can define $E[|X|] := E[X^{+}] + E[X^{-}]$. (Wait I forgot. Maybe you don't define this. Maybe it's really just decomposed by linearity of expectation/integration...)

And then we get to 4:

$$E[X] := E[X^{+}] - E[X^{-}]$$

Now, this is defined for either

1. $X$ that satisfies $E[|X|] < \infty$ --> Here, we say that $X$ is Lebesgue integrable.

2. $X$ that satisfies $E[X^{+}] < \infty$ or $E[X^{-}] < \infty$ --> Here, we say that the Lebesgue integral of $X$ exists.

Here, Condition 1 implies ($E[X^{+}] < \infty$ and $E[X^{-}] < \infty$, which implies) Condition 2 but not conversely. I guess analogous to regular limits in elementary calculus it's like: $\lim x^2 = \infty$, so $\lim x^2$ 'exists' but is not 'existable'. Meanwhile, $\lim \frac1x = 0$, so $\lim \frac1x$ is 'existable'. So 'existable' is the same as the usual 'exists' in elementary calculus, but 'exists' here is like including $\pm \infty$.

Finally, the aforementioned 'last part':

1. Anyway, I believe most texts on measure theory or advanced probability will consider specifically the Lebesgue integrable ones and not all of the Lesbesgue-integral-exists ones. On wiki, it says 'It turns out that this definition gives the desirable properties of the integral.' I assume this statement refers to Lebesgue integrable.

So now this begs the question as to why those texts refer to Lebesgue integrable random variables/measurable functions specifically. Well, there's probably (lol) some properties that Lebesgue integrable's satisfy that not all of the Lesbesgue-integral-exists ones do, but I figure it's just to avoid $\pm \infty$

2. AlohaSine mentioned about $\infty - \infty$ cases in h answer, but I think it doesn't apply to the Lesbesgue-integral-exists ones:

• For $E[X^{+}] < \infty$ and $E[X^{-}] = \infty$, $E[X] = - \infty$

• For $E[X^{+}] = \infty$ and $E[X^{-}] < \infty$, $E[X] = \infty$

Wait actually I just realised that even for elementary level probability, we use this kinds of definitions eg St. Petersburg paradox: $E[X]$ doesn't really 'exist' in the sense that $E[|X|] = \infty$, but we do have that $E[X]$ '$=\infty$' under the idea that $E[|X|] = \infty$ because $E[X^{+}] = \infty$ while $E[X^{-}] < \infty$ (specifically $E[X^{-}] =0 $) as I believe.

3. there's a lot of stuff in comments and answers about rearrangement in re conditional convergence. Not so familiar about this apart from wiki, but I think this (explicitly) explains only the case of discrete random variables and not for random variables in general (whenever their expectations exist...or are 'existable')

Answer 4

There is a distinction to be made in math between features that are an essential part of a mathematical object, versus features that are used in a more arbitrary manner simply as labels so that we can discuss the objects.

For instance, if we're encoding demographic data, and one of the variables is race, to encode that data as an integer requires assigning numbers to the different races. What number a race gets is not an essential feature of the race, but simply an arbitrary number given to keep track of it. It would be rather weird if our calculation of the average value of some metric over the different races gave a different answer depending on how we labeled the races.

Similarly, the index that we assign different $x$ values is not an essential part of the data. Even if there's an "obvious" order, it's not the only ordering, and we would want our definition to not depend on picking the right order, or there being an obvious ordering. Consider the set of rational numbers. This set is countable, so it's possible to come up with a labeling of the rational numbers by integers, but there are many different ways of going about this. If two people used different methods for labeling the rational numbers and got different numbers for the mean, that would be a problem.

The fact that the formula has $\Sigma$ rather than $\Sigma_{i=0}^{\infty}$ emphasizes this: the mean is an attribute of the events and their probabilities, not of any particular indexing. We can define the $\Sigma$ operator without any reference to an indexing. The sum over a finite set can easily be defined, and for an infinite set $X$, we can define the limit of its sum as being the number $L$ such that for any positive $\epsilon$, $X$ can be split into a finite set $H$ and infinite set $T$ such that the sum over $H$ plus the sum over any finite subset of $T$ is within $\epsilon$ of $L$. i.e.

$\Sigma X = L$ if $\forall \epsilon >0,\exists H,T:H \cup T = X, H \cap T= \emptyset, |H|<\infty, \forall S \subset T, |S| < \infty \rightarrow |\Sigma X + \Sigma S-L|<\epsilon$

This definition (unless I've messed up somewhere) is equivalent to "uniform convergence"; the term "uniform" is a qualifier we have to add when defining sums with respect to an indexing to make that sum independent of the indexing.