The law of large numbers - limits of $\max$ vs $\max$ of a limit.

Question

Assume that $X_{1,1}, \dots , X_{1,n}, X_{2,1},\dots, X_{2,n}, \dots ,X_{n,1}, \dots , X_{n,n}$ are i.i.d. random variables, and that $\mathbb EX_{i,j}$ exists and is finite. From the strong law of large numbers we have $$\max_i\left\{\lim_{n\to\infty}\left\{\frac{1}{n}\sum_{j=1}^{n}X_{i,j}\right\}\right\} \overset{a.s.}{=} \mathbb EX_{i,j} ,$$ because $\max_i$ is redundant. However, do we have $$\lim_{n\to\infty}\left\{\max_i\left\{\frac{1}{n}\sum_{j=1}^{n}X_{i,j}\right\}\right\} \overset{a.s.}{=} \mathbb EX_{i,j} ? $$ I.e., can we change the order of $\max$ and $\lim$ without affecting the result?

Answer 1

IF the all moments exist, then the answer is as follows.

Without loss of generality (and by doing and affine transformation $X\rightarrow aX+b$), I assume $$\mathbb{E}[X]=0; \hspace{3mm}\mathbb{E}[X^2]=1$$ Then, the moment generating function $$g(s)\equiv \log\mathbb{E}[e^{sX}]$$ would satisfy $$g(0)=0; \hspace{3mm}g^\prime(0)=0; \hspace{3mm}g^{\prime\prime}(0)=1$$ For each $i$, the variate $Y_i\equiv\frac1n\sum_jX_{ij}$ has $$f(s)\equiv\log\mathbb{E}[e^{sY}]=ng(s/n)=\frac{s^2}{2n}+\mathcal{O}(\frac{1}{n^2})$$ Therefore the density for $Y$ is $$p(y)=\sqrt{\frac{n}{2\pi}}e^{-\frac{ny^2}{2}}+\frac{e_n(y)}{n^2}$$ For some finite function $e_n(y)$. Now the question is: What is the asymptotic behavior of $$Z\equiv\max_{i\in[n]}Y_i$$ Let's start by the cumulative distribution function $$F_Z(z)=F_Y^n(z)=\Big(F_G(z\sqrt n)+\frac{e_n(z)}{n^2}\Big)^n$$ $$= F_G^n(z\sqrt n)+\frac{e_n(z)}{n}+\mathcal{O}(\frac{1}{n^2})$$ where $F_G$ is the cumulative distribution for a standard gaussian. This approximation holds for all real $z$ unless $|z|\ll \frac{1}{\sqrt{n}}$.

Next, let us try to calculate the moments for $Z$ using known results about the maximum of a set of independent gaussian variables. $$\mathbb{E}[Z]=\int z F_Z^\prime(z)dz=\frac{C\log n}{\sqrt{n}}+\mathcal{O}(\frac{1}{n})$$ $$\mathbb{E}[Z^2]=\int z^2 F_Z^\prime(z)dz=\frac{C\log^2 n}{n}+\mathcal{O}(\frac{1}{n})$$ These mean $$\boxed{Z\overset{a.s.}{=}0}$$ There are (small) gaps in my solution but the bounty is due, I had to hurry!