Less than or equal on random variables

Question

Casella and Berger exercise 2.15 asks to prove ($Y$ and $X$ are any random variables): $E[\max(X,Y)] = E[X] + E[Y] - E[\min(X,Y)]$, in the solution it gives:

Assume without loss of generality that $X \le Y$. Then $\max(X, Y) = Y$ and $\min(X, Y) = X$. Thus $X + Y = \max(X, Y) + \min(X, Y)$. Taking expectations $E[X + Y] = E[\max(X,Y) + \min(X,Y)] = E[\max(X,Y)] + E[\min(X,Y)]$. Therefore $E[\max(X,Y)] = E[X] + E[Y] − E[\min(X, Y)]$.

How shall I think of $X \le Y$ "without loss of generality". Is this stochastic dominance (but then I would not think we can assume it without loss of generality)? Or does it mean something else?

Answer 1

You are correct that it is weird to assume $X \le Y$ or $X \ge Y$ for random variables; either one would be a massive loss of generality!

The solution is not careful about separating two steps of this argument.

1. First, we prove a general fact about real numbers: for all $x$ and $y$, $x+y = \min\{x,y\} + \max\{x,y\}$. Here, it is valid to say that "without loss of generality, $x \le y$", because we are talking about numbers.
2. Second, from this general fact, we conclude that $X+Y = \min\{X,Y\} + \max\{X,Y\}$ always holds (that is, it holds at every outcome of the random experiment ). From there on, we can take expectations, and it's business as usual.

In many problems, it is possible to combine these two steps, but in this case, it makes the "without loss of generality" weird, and we should keep the two steps separate.

Another way to make things "specific enough" to make the assumption is to talk about specific outcomes in the sample space. We can say:

Pick an arbitrary $\omega \in \Omega$, where $\Omega$ is the set of outcomes for the probability space where $X$ and $Y$ live. Without loss of generality, $X(\omega) \le Y(\omega)$. Then $\min\{X(\omega), Y(\omega)\} =X(\omega)$ and $\max\{X(\omega), Y(\omega)\} = Y(\omega)$. Thus $X(\omega) + Y(\omega) = \min\{X(\omega), Y(\omega)\} + \max\{X(\omega),Y(\omega)\}$. Since this holds for an arbitrary $\omega$, we can say that $X+Y = \min\{X,Y\} + \max\{X,Y\}$ as a general statement about the random variables.

The notation with $\omega$ is cumbersome and we'd often prefer to omit it. One way or the other, we should make it clear that in the "without loss of generality" statement, we are comparing $X$ and $Y$ at some specific outcome, not in general.

Answer 2

I agree that the given solution looks confusing. I suspect this is mostly because it's condensed a bit too far.

For someone like a lecturer or a teaching assistant grading student exercises, i.e. someone who already knows the topic well and just needs a reminder on what to look for when grading answers, this kind of condensed summary would be fine since they can mentally fill in the gaps. But a solution intended for students themselves to look at really should spell things out more clearly.

---

Anyway, the given solution looks a lot clearer if we break it up into explicit steps:

1. Assume without loss of generality that $X \le Y$. Then $\max(X, Y) = Y$ and $\min(X, Y) = X$. Thus $X + Y = \max(X, Y) + \min(X, Y)$.
2. Taking expectations $E[X + Y] = E[\max(X,Y) + \min(X,Y)] = E[\max(X,Y)] + E[\min(X,Y)]$.
3. Therefore $E[\max(X,Y)] = E[X] + E[Y] − E[\min(X, Y)]$.

In step 1, as Misha Lavrov notes, we prove a general equation that applies to any real numbers, and therefore also to real-valued random variables. No expectations appear in this step.

In particular, it should be clear that for any two real numbers $X$ and $Y$, either $X ≤ Y$ or $Y ≤ X$ (or possibly both!), and that since both $X$ and $Y$ here are just arbitrary labels for any two values, no generality is indeed lost by assigning $X$ to be the label for the smaller value of the two.

What may be potentially confusing here, however, is that if $X$ and $Y$ are random variables, then the direction of the inequality may vary depending on the observed outcome. However, since in this step we are proving a true pointwise equality (i.e. an equality in every outcome), rather than some weaker equivalence (like almost sure equality or equality in distribution) between $X$ and $Y$, we can (again without loss of generality!) just consider any arbitrary outcome and show that the equality holds regardless.

---

In step 2, we then take the general result that $X + Y = \max(X, Y) + \min(X, Y)$ and apply the facts that (a) equal random variables have equal expectations, and that (b) the expectation of a sum equals the sum of the expectations of the terms, to obtain the equation $E[X] + E[Y] = E[\max(X,Y)] + E[\min(X,Y)]$.*

Since we're now dealing with expectations over many possible outcomes, we can no longer focus on just one arbitrary outcome, and thus the assumption that $X ≤ Y$ from step 1 no longer makes sense. But that's OK, because we no longer need it! We have our equation from step 1, which applies regardless of which of $X$ or $Y$ happens to be greater in any given outcome, and that equation is all we need. So we simply stop assuming that $X ≤ Y$ between steps 1 and 2. The solution, as written, just doesn't say so explicitly.

Anyway, once we've done this, step 3 is just elementary arithmetic rearrangement (subtracting $E[\min(X,Y)]$ from both sides and switching the sides) to get the equation into the exact form that the exercise asks for.

---

*) There's also another tiny gap in the solution as written, where it skips from $E[X + Y]$ to $E[X] + E[Y]$ between steps 2 and 3 without explicitly noting that these expressions are also equal. Presumably the reader is also assumed to be able to fill that in, if they know about the linearity of expectation at all.