This is Etemadi's proof of the strong law of large numbers. I did not understand the four steps marked in the red boxes. Can anyone explain me these steps?
Thanks in advance for your help.
Theorem 1. Let $\{X_n\}$ be a sequence of pairwise independent, identically distributed random variables. Let $S_n = \sum_{i=1}^{n} X_i$. Then
$$ E|X_1| < \infty \quad \Rightarrow \quad \lim_{n\to\infty} \frac{S_n}{n} = EX_1 \quad \text{a.s.} $$
Proof. Since $\{X_n^+\}$ and $\{X_n^-\}$ satisfy the assumptions of the theorem and $X_i = X_i^+ - X_i^-$, without loss of generality we can assume that $X_i \geq 0$. LEt $Y_i = X_i I\{X_i \leq i\}$ with $I$ the indicator function and $S_n^* = \sum_{i=1}^{n} Y_i$. Now for $\epsilon > 0$ let $k_n = [\alpha^n]$, $\alpha > 1$ and use Chebyshev's inequality to obtain
\begin{align*} \sum_{n=1}^{\infty}P\left( \left| \frac{S_{k_n}^* - ES_{k_n}^*}{k_n} \right| \geq \epsilon \right) &\leq c \sum_{n=1}^{\infty} \frac{\operatorname{Var}S_{k_n}^*}{k_n^2} = c \sum_{n=1}^{\infty} \frac{1}{k_n^2}\sum_{i=1}^{k_n} \operatorname{Var}Y_i \\ &\color{red}{\boxed{\leq}} c \sum_{i=1}^{\infty} \frac{E Y_i^2}{i^2} = c \sum_{i=1}^{\infty} \frac{1}{i^2} \int_{0}^{i} x^2 \, dF(x) \\ &= c \sum_{i=1}^{\infty} \frac{1}{i^2} \left( \sum_{k=0}^{i-1} \int_{k}^{k+1} x^2 \, dF(x) \right) \\ &\color{red}{\boxed{\leq}} c \sum_{k=0}^{\infty} \frac{1}{k+1} \int_{k}^{k+1} x^2 \, dF(x) \tag{1} \\ &\leq c \sum_{k=0}^{\infty} \int_{k}^{k+1} x \, dF(x)\\ &= cE X_1 < \infty, \end{align*}
where $F(x)$ is the distribution of $X_1$ and $c$ is an unimportant positive constant which is allowed to change. We also have
$$ EX_1 = \lim_{n\to\infty} \int_{0}^{n} x \, dF(x) = \lim_{n\to\infty} EY_n = \lim_{n\to\infty} \frac{E S^*_{k_n}}{k_n}. \tag{2} $$
Therefore by the Bore-Cantelli Lemma
$$ \lim_{n\to\infty} \frac{S_{k_n}^*}{k_n} = EX_1 \quad \text{a.s.} \tag{3} $$
Also
\begin{align*} \sum_{n=1}^{\infty} P(Y_n \neq X_n) = \sum_{n=1}^{\infty} P(X_n > n) &= \sum_{n=1}^{\infty} \int_{n}^{\infty} dF(x) = \sum_{n=1}^{\infty} \sum_{i=n}^{\infty} \int_{i}^{i+1} dF(x) \\ &\color{red}{\boxed{=}} \sum_{i=1}^{\infty} i \int_{i}^{i+1} dF(x) \leq \sum_{i=1}^{\infty} \int_{i}^{i+1} x dF(x) \\ &\leq EX_1 < \infty. \tag{4} \end{align*}
Hence by the Borel-Cantelli Lemma $X_n \neq Y_n$ only finitely many times. Consequently
$$ \lim_{n\to\infty} \frac{S_{k_n}}{k_n} = E X_1 \quad \text{a.s.} \tag{5}$$
Now from monotonicity of $S_n$ we can conclude that
$$ \color{red}{\boxed{ \frac{1}{\alpha} (E X_1) \leq \varliminf_{n\to\infty} \frac{S_n}{n} \leq \varlimsup_{n\to\infty} \frac{S_n}{n} \leq \alpha (E X_1) \quad \text{a.s.} }} \tag{6} $$
for every $\alpha > 1$ which gives us the desired result.
The original image can be seen here.
First three boxed parts are related to the double summation technique:
For instance, your third box follows from
\begin{align*} \sum_{n=1}^{\infty} \sum_{i=n}^{\infty} \int_{i}^{i+1} dF(x) &= \sum_{n=1}^{\infty} \sum_{i=1}^{\infty} \left( \int_{i}^{i+1} dF(x) \right) I\{n \leq i \} \\ &\overset{\text{(*)}}{=} \sum_{i=1}^{\infty} \sum_{n=1}^{\infty} \left( \int_{i}^{i+1} dF(x) \right) I\{n \leq i \} \\ &= \sum_{i=1}^{\infty} i \int_{i}^{i+1} dF(x). \end{align*}
And then your second box follows from
\begin{align*} \sum_{i=1}^{\infty} \frac{1}{i^2} \left( \sum_{k=0}^{i-1} \int_{k}^{k+1} x^2 \, dF(x) \right) &\overset{\text{(*)}}{=} \sum_{k=0}^{\infty} \left( \sum_{i=k+1}^{\infty} \frac{1}{i^2} \right) \int_{k}^{k+1} x^2 \, dF(x) \\ &\leq \sum_{k=0}^{\infty} \frac{c'}{k+1} \int_{k}^{k+1} x^2 \, dF(x), \end{align*}
where $c' > 0$ is a generic constant whose value is not important in this context. Similarly, the first box follows from
$$ \sum_{n=1}^{\infty} \frac{1}{k_n^2}\sum_{i=1}^{k_n} \operatorname{Var}Y_i \overset{\text{(*)}}{=} \sum_{i=1}^{\infty} \left( \sum_{n=1}^{\infty} \frac{1}{k_n^2}I\{ i \leq k_n \} \right) \operatorname{Var}Y_i $$
and the inner sum can be estimated by
$$ \sum_{n=1}^{\infty} \frac{1}{k_n^2}I\{ i \leq k_n \} \leq \sum_{n=1}^{\infty} \frac{c'}{\alpha^{2n}}I\{ i \leq \alpha^n \} \leq \frac{c'}{\alpha^{2( \log i/\log \alpha)}} = \frac{c'}{i^2}, $$
where again $c' > 0$ is a generic constant whose value may change from line to line.
For the last line, for each $n$ we pick $m = m_n$ such that $k_m \leq n \leq k_{m+1}$. Then
$$ \frac{k_m}{k_{m+1}} \cdot\frac{S_{k_m}}{k_m} = \frac{S_{k_m}}{k_{m+1}} \leq \boxed{\frac{S_n}{n}} \leq \frac{S_{k_{m+1}}}{k_m} = \frac{k_{m+1}}{k_m} \cdot \frac{S_{k_{m+1}}}{k_{m+1}} $$
Then the fourth red box $\text{(6)}$ follows from the fact that the LHS converges to $\frac{1}{\alpha} EX_1$ and the RHS converges to $\alpha (E X_1)$ as $n\to\infty$ (and hence $m = m_n \to \infty$).