Change the order of expectation

Question

Sorry this might be a silly question, but I'm kind of confused and really want to make sure I'm correct.

Let $v_1,v_2,\dots,v_n$ be $n$ i.i.d. random variables with the same range of $[\underline{v}, \overline{v}]$ and the same CDF $F(v_i)$. Let $G(v_1,v_2,\dots,v_n)$ be a function of the $n$ random variables.

Is it true that we can always freely change the order of the expectations?

For example, is it true that

$E_1[E_2[E_3[\dots[G(v_1,v_2,\dots,v_n)]\dots]]] =E_1[E_3[E_2[\dots[G(v_1,v_2,\dots,v_n)]\dots]]]$ ?

If not, what conditions do I need for equalities like the one above to be true?

Thanks everyone!

Answer 1

If $v_1,v_2,\ldots,v_n$ are $n$ i.i.d random variables with the same range and the same CDF $F(v_i)$ and $G(v_1,v_2,\ldots,v_n)$ is some function of them, then $G(v_1,v_2,\ldots,v_n)$ is itself a random variable, and its expected value $\operatorname{E}(G(v_1,v_2,\ldots,v_n))$ is a scalar, not a non-constant random variable. It has no randomness it it. The expected value of a constant is itself, so one could say $\operatorname{E}(\operatorname{E}(G(v_1,\ldots,v_n)))=\operatorname{E}(G(v_1,\ldots,v_n))$.

Since they are i.i.d., one could say the expected value is $$ \operatorname{E}(G(v_1,\ldots,v_n)) = \int\cdots\int G(x_1,\ldots,x_n)\,dF(x_1)\cdots dF(x_n) = c. $$ If one iterates the expectation one is merely integrating a constant over a space of measure $1$: $$ \operatorname{E}(c) = \int\cdots\int c\,dF(x_1)\cdots dF(x_n) = c. $$ (The same c.d.f. $F$ appears $n$ times here, since they are identically distributed, and $\Pr(X_1\le x_1\ \&\ \cdots\ \&\ X_n\le x_n)$ is simply the product $F(x_1)\cdots F(x_n)$ because $v_1,\ldots,v_n$ are independent.)

As "zoli" has pointed out in a comment, the meaning of your subscripts in "$\operatorname{E}_i$" is unclear at best.

Answer 2

You are certainly talking about the following integral:

$$\iint \cdots \int G(x_1,x_2,\cdots x_n)\,dF_1\,dF_2\cdots dF_n.$$

"Freely changing the order of integration" is a calculus issue. Probability theory does not have specific rules in this respect.

Answer 3

I gather this non-standard shorthand is to indicate the order of integration, thusly:

$$\mathsf E_1(\mathsf E_2(\ldots \mathsf E_n(G(V_1,V_2,\ldots, V_n)\ldots)) = \iint\cdots\int_\Omega G(v_1,v_2, \ldots, v_n)\;\mathrm d\mathsf F(v_n)\cdots\mathrm d\mathsf F(v_2)\;\mathrm d\mathsf F(v_1)$$

If that's the case, then the Fubini–Tonelli theorem states that if $G$ is a measurable function over the joint support (which is a measure space), and $\mathsf E(\lvert G(\bullet)\rvert)$ is finite when integrated in any order, then the order of integration of $\mathsf E(G(\bullet))$ will be irrelevant to the result.