Hoeffding's lemma: hard to prove

247 Views Asked by At

This is a continuation of another post.

Let $F$ be the joint distribution function and $F_X,F_Y$ the marginal distribution function of the random variables $X,Y$ respectively. Let $(X,Y), (X_2,Y_2)$ be independent and distributed according to $F$. Assume $E\mid XY\mid, E\mid X \mid, E\mid Y \mid$ finite. Then

\begin{align} &E\bigg\{\int\int\Big[I(X\leq x)-I(X_2\leq x)\Big]\Big[I(Y\leq y)-I(Y_2\leq y)\Big]dxdy\bigg\}\\ &=2\int\int F_{X,Y}(x,y)-F_X(x)F_Y(y)dxdy \end{align} where $I$ is the indicator function.

My attempt

There are two challenging steps for me.

Firstly, I think I have to show that I can put the expectation inside the integrals using Fubini-Tonelli's theorem. Unfortunately I'm struggling to deal with it. Well, I know that the integrand is a (measurable) function of the random variables $X,Y,X_2$ and $Y_2$. So I saw the $L.H.S.$ of the above equality as

$$\int_{\mathbb{R}^4}\int\int\Big[I(x_1\leq x)-I(x_2\leq x)\Big]\Big[I(y_1\leq y)-I(y_2\leq y)\Big]dxdy P_{X,Y,X_2,Y_2}(dx_1dy_1dx_2dy_2)$$ I don't know if this allows me to interchange the integrals to obtain \begin{align} &\int\int\ E\bigg\{\Big[I(X\leq x)-I(X_2\leq x)\Big]\Big[I(Y\leq y)-I(Y_2\leq y)\Big]\bigg\}dxdy\\ \end{align} If everything is alright so far, then I need to show that $E\bigg\{\Big[I(X\leq x)-I(X_2\leq x)\Big]\Big[I(Y\leq y)-I(Y_2\leq y)\Big]\bigg\}=2Cov(I(X\leq x),I(Y\leq y))$, and the rest becomes straighforward to me. Thus I need to show (i) the independence of $I(X_2\leq x)\perp I(Y\leq y)$ and $I(X\leq x)\perp I(Y_2\leq y)$ given $X_2\perp Y$ and $X\perp Y_2$; and (ii) $E(I(Y\leq y))=E(I(Y_2\leq y))$ which is easy, and that $Cov(I(X\leq x),I(Y\leq y))=Cov(I(X_2\leq x),I(Y_2\leq y))$.

I wonder if you can help me with it.

UPDATE Since $X$ is independent of $Y_2$, by definition, $\sigma(X)=\{X^{-1}(B):B\in \mathcal{B}_\mathbb{R}\}$ and $\sigma(Y_2)$ are independent, meaning that $P(A\cap B)=P(A)P(B), \ \forall A\in \sigma(Y_2), B\in \sigma(X)$. Since $f=I_{(-\infty,x]}$ and $g=I_{(-\infty,y]}$ are $(\mathbb{R},\mathcal{B}_\mathbb{R})-(\mathbb{R},\mathcal{B}_\mathbb{R})$measurable functions, then $(f\circ X)^{-1}=X^{-1}(f^{-1}(A))\in \sigma(X), \forall A\in \mathcal{B}_\mathbb{R}$. The same holds for $g\circ Y_2$. It implies $\sigma(f\circ X)\subseteq \sigma(X)$ and $\sigma(g\circ Y_2)\subseteq \sigma(Y_2)$. Hence $\sigma(f\circ X)$ and $\sigma(g\circ Y_2)$ are also independent. That is the indicator functions (measurable) preserve the independence of the random variables.

Exploring one of the answers below, let $f,b:\mathbb{R}\rightarrow\mathbb{R}$ measurable functions (like the indicator functions I am interested in). Then $F_{f\circ X, g\circ Y}(x_1,y_1)=P\{f(X)\leq x_1, g(Y)\leq y_1\}=P(X\in f^{-1}(-\infty,x_1], Y\in g^{-1}(-\infty,y_1])=P_{XY}(f^{-1}(-\infty,x_1]\times g^{-1}(-\infty,y_1])=P_{X_2Y_2}(f^{-1}(-\infty,x_1]\times g^{-1}(-\infty,y_1])=F_{f\circ X_2, g\circ Y_2}(x_1,y_1)$. This immediately implies $Cov(f\circ X_2, g\circ Y_2)=Cov(f\circ X, g\circ Y)$. Just put $f=I_{(-\infty,x]}$ and $g=I_{(-\infty,y]}$. Finally, it is clear that the marginal probability distributions must be the same ($P_X=P_{X_2}$), by assumption. Therefore, $E(f\circ X)=\int_{\Omega} (f\circ X)(z) P(dz)=\int_{\mathbb{R}} f(w) P_X(dw)=\int_{\mathbb{R}} f(w) P_{X_2}(dw)=E(f\circ X_2)$, since $f$ is nonnegative measurable function (the indicator function)[see the Corollary 5.5.1, Resnick, A probability path, p. 138].

2

There are 2 best solutions below

1
On

i) Independence of $(X,Y)$ and $(X_2,Y_2)$ implies independence of $X$ and $Y_2$. This implies independence of $f(X)$ and $g(Y_2)$ for any measurable functions $f,g:\mathbb R \to \mathbb R$. Hence it implies independence of $I_{X_2 \leq x}$ and $I_{Y\leq y}$. Similarly we get independence of $I_{X \leq x}$ and $I_{Y_2\leq y}$.

ii) Since $(X,Y)$ and $(X_2,Y_2)$ have the same distribution it follows that $(f(X),g(Y))$ and $(h(X_2),k(Y_2))$ have the same distribution for any measurable functions $f,g,h,k:\mathbb R \to \mathbb R$. The equality of covariances follows from this immediately.

5
On

Exchange of order of integration will be an application of Fubini's theorem applied to $g(x,y,\omega)=\Big(\mathbb{1}(X(\omega)\leq x)-\mathbb{1}(X_2(\omega)\leq x)\big)\big(\mathbb{1}(Y(\omega)\leq y)-\mathbb{1}(Y_2(\omega)\leq y)\Big)$. This is $dP\otimes dx\otimes dy$ integrable; in fact, $$\int_\Omega\int_\mathbb{R}\int_\mathbb{R} |g(x,y,\omega)|dx \,dy\,P(d\omega)= E\big[|X-X_2||Y-Y_2|\big]\leq 2(E[|XY|]+E[|X|]E[|Y|])<\infty$$ by independence and the integrability assumptions.