I read that discrete random variables form a vector space, but it doesn't seem possible. Help?

Question

I am a Linear Algebra teacher in high school. We just did a unit on vector spaces, and I want to create a good application to my students to try to understand a way in which something which isn't obviously a vector space could be viewed as a vector space. So, I was doing some work on random variables, and wondered whether there was some way to view random variables as being vectors in a vector space.

I did a little research, and found some references that suggest that random variables form a vector space under the "usual" definition of the sums and scaled products of two random variables. Here and Here.

The essence of my problem comes down to the fact that each citation that I find seems to be asserting that random variables form a vector space, but the operations used don't seem to behave according to the properties that these operations are supposed to be exhibiting! Either these references are wrong (which I doubt), or my understanding of these operations is wrong and I need to understand what the true operations are that people are using.

Let's say that $\mathbf X_1$ represents the discrete probability distribution for the rolling of a single six-sided fair die. Using the rules of a vector space, $\mathbf X_1+ \mathbf X_1=1 \mathbf X_1 +1 \mathbf X_1=(1+1)\mathbf X_1=2\mathbf X_1$. But, it is not the case that $\mathbf X_1 + \mathbf X_1$ is the same random variable as $2 \mathbf X_1$, since the former allows the event '7' and the latter allows only even results.

Is my Linear Algebra bad, is my understanding of Random Variable arithmetic bad, or am I just using the wrong definitions for these operations?

Answer 1

I would say that indeed it is true that $\mathbf X_1 + \mathbf X_1 = 2\mathbf X_1.$ But this can only be correctly understood by understanding what $\mathbf X_1$ represents.

A single random variable such as $\mathbf X_1$ is essentially a one-chance affair. We may say we don't know what its value is going to be, but whatever its value is is what its value is. It does not get a "second chance" to take on some other value.

In short, $\mathbf X_1 \stackrel{?}{\neq}\mathbf X_1$ is impossible.

If you want to roll two dice and add them, then each die is represented by its own random variable. For example, you might name these variables $\mathbf X_1$ and $\mathbf X_2.$ Then, since in general it is not true that $\mathbf X_2 \stackrel{?}{=} \mathbf X_1,$ it also is not true in general that $\mathbf X_1 + \mathbf X_2 \stackrel{?}{=} 2\mathbf X_1.$ The left hand side actually represents rolling two dice and adding them, while the right side represents rolling one die and doubling the result. So while it is possible to come up with the same result both ways, the equation has only a relatively small probability of being true, and a larger probability of being false.

Answer 2

Discrete random variables can be viewed as vectors indexed by the sample space $\Omega = \{ \omega_1, \cdots, \omega_n \} $

Then a r.v. $ X \equiv \left( X(\omega_1), \cdots, X(\omega_n) \right) $

The same holds for countable or uncountable sample space $ \Omega $ since a random variable is technically a function $X: \Omega \to \mathbb R$

A beautiful connection can be made for the vector space of (finitely) discrete random variables with mean zero and finite variance. This space is Euclidean, its norm is standard deviation and its dot product is covariance. The Cauchy-Schwarz inequality is equivalent to the correlation coefficient being comprised between -1 and 1.