I've come across the following theorem in a computer science paper:
For random variables $X, Y$ and function $f(x, y)$ under suitable regularity conditions, $\mathbb{E}_{x \sim X, y \sim Y \mid x} [f(x, y)] = \mathbb{E}_{x \sim X, y \sim Y \mid x, x' \sim X \mid y} [f(x', y)]$.
The paper then goes on to prove this statement:
\begin{align*} \mathbb{E}_{x \sim X, y \sim Y \mid x} [f(x, y)] &= \int_{x} P(x) \int_y P(y \mid x) f(x, y) dy dx \\ &= \int_x \int_y P(x, y) f(x, y) dy dx \\ &= \int_x \int_y P(x, y) f(x, y) \int_{x'} P(x' \mid y) dx' dy dx \\ &= \int_x P(x) \int_y P(y \mid x) \int_{x'} P(x' \mid y) f(x', y) dx' dy dx \\ &= \mathbb{E}_{x \sim X, y \sim Y \mid x, x' \sim X \mid y} [f(x', y)] \end{align*}
As is true for many computer science papers, the mathematical details are a little sparse. I have in particular two questions:
i) What precisely should be the appropriate regularity conditions for $f(x, y)$ in the theorem?
ii) How do we go from line 3 to line 4 in the proof? It seems like we just substitute $x$ for $x'$ after pushing $f(x, y)$ into the integral over $x'$, but it's not clear to me why this can be done.