Convergence of the random harmonic series $\sum_{n=1}^{\infty}\frac{X_{n}}{n}$

Question

Let $(X_{n})_{n \in \mathbb{N}}$ be independent with Rademacher distribution:

\begin{equation} \mathbb{P}(X_{n} = -1) = \frac{1}{2} = \mathbb{P}(X_{n} = 1). \end{equation}

I have to investigate

\begin{equation} \sum_{n=1}^{\infty}\frac{X_{n}}{n} \end{equation} for convergence. It was given in a textbook and I'm very interested in the solution. It is something between the harmonic series $\sum_{n=1}^{\infty}\frac{1}{n}$ and the series $\sum_{n=1}^{\infty}\frac{(-1)^{n}}{n}$, but I know the sign changes randomly.

Answer 1

Define for $N \in \mathbb{N}$ \begin{equation} Y_{N} = \sum_{n=1}^{N}X_{n} \end{equation} Now \begin{equation} \mathbb{E}(\frac{1}{n}X_{n}) = 0. \end{equation} Therefore \begin{equation} \mathbb{E}(Y_{N}) = 0. \end{equation} Also \begin{align} var(Y_{N}) &= var(\sum_{n=1}^{N}X_{n})\\ &= \sum_{n=1}^{N}\frac{1}{n^{2}}var(X_{n})\\ &=\sum_{n=1}^{N}\frac{1}{n^{2}}\\ &\leq\sum_{n=1}^{\infty}\frac{1}{n^{2}} = \frac{\pi^{2}}{6}. \end{align}

For $M < N$ we have \begin{align} \mathbb{E}[|Y_{N}-Y_{M}|^{2}] &= \mathbb{E}[|\sum_{n=M+1}^{N}\frac{1}{n}X_{n}|^{2}]\\ &=var(\sum_{n=M+1}^{N}\frac{1}{n}X_{n})\\ &=\sum_{n=M+1}^{N}\frac{1}{n^{2}} \xrightarrow{N,M \to \infty} 0, \end{align} i.e. $(Y_{N})_{N \in \mathbb{N}}$ is a cauchy-sequence with respect to the $\mathcal{L}^{2}$-norm. Now the Riesz–Fischer theorem says, that $\mathcal{L}^{2}$ is a banach-space. Especially every cauchy sequence in $\mathcal{L}^{2}$ converges. Now $\mathcal{L}^{2}$-convergence implies onvergence in probability and distribution. We also have almost sure convergence, since if $(X_{n})_{n \in \mathbb{N}}$ are independet random variables with $S_{n} = \sum_{i=1}^{n}X_{i} \xrightarrow{p} S$ then $S_{n} \rightarrow S$ almost surely.

Answer 2

It's clear that the sum converges in $L^2$. Because the terms are independent it follows that the sum converges almost surely. One can give a simple "trick proof" of this using Khinchine's Inequality:

Say $$X=\sum\frac{X_n}{n}$$(where the convergence is in $L^2$). We need to show that $E_n\to0$ almost surely, where $$E_n=X-S_n,$$ $$S_N=\sum_{n=1}^N\frac{X_n}{n}.$$

Khinchine's inequality (see below) says that $$||E_N||_4\le c||E_N||_2=c\left(\sum_{n=N+1}^\infty\frac1{n^2}\right)^{1/2}\sim\frac c{N^{1/2}}.$$So $$\Bbb E(E_n^4)\le\frac c{n^2}.$$So monotone convergence implies $\Bbb E\sum E_n^4<\infty$; hence $\sum E_n^4<\infty$ almost surely, which implies $E_n\to0$ almost surely.

Below: In fact the relevant special case of Khinchine's inequality is trivial:

Lemma. If $a_j$ is real then $\Bbb E (\sum a_jX_j)^4\le c(\sum a_j^2)^2$.

Proof: If you multiply out $(\sum a_jX_j)^4$ you get a lot of terms. By independence all the terms have expected value $0$ except for terms of the form $a_j^4X_j^4$ and $a_j^2a_k^2X_j^2X_k^2$. Noting that the first form is just the second form for $j=k$ you see that $$\Bbb E(\sum a_jX_j)^4\le c\sum_j\sum_ka_j^2a_k^2=c(\sum a_j^2)^2.$$

Answer 3

Use the following theorem about sums of zero-mean independent variables in $\mathcal L^2$ from David Williams' Probability with Martingales Chapter 12.2.

Let $\{X_n\}_{n \in \mathbb N}$ be a sequence of random variables. We have $$\sum_n X_n < \infty \ \text{a.s.}.$$ if the following 4 conditions hold:

1. $$\{X_n\}_{n \in \mathbb N} \text{is independent.}$$

2. $$E[X_n] = 0 \ \forall n \in \mathbb N$$

3. $$\sigma_n^2 := Var[X_n] < \infty \ \forall n \in \mathbb N$$

4. $$\sum_{n \in \mathbb N} \sigma_n^2 < \infty$$

In our case:

1. $$\{\frac{X_n}{n}\}_{n \in \mathbb N} \ \text{is independent} \ \because \{X_n\}_{n \in \mathbb N} \text{is independent.}$$

2. $$E[\frac{X_n}{n}]=\frac{1}{n}E[X_n] = \frac{1}{n} 0 = 0 \ \forall n \in \mathbb N$$

3. $$Var[\frac{X_n}{n}] = \frac{1}{n^2} Var[\frac{X_n}{1}] = \frac{1}{n^2} (1) = \frac{1}{n^2} < \infty \ \forall n \in \mathbb N$$

4. $$\sum_{n \in \mathbb N} Var[\frac{X_n}{n}] = \sum_{n \in \mathbb N} \frac{1}{n^2} < \infty \ \text{by integral test}$$

$\therefore,$ by theorem in Chapter 12.2 of David Williams' Probability with Martingales, $$\sum_{n \in \mathbb N} \frac{X_n}{n} < \infty \ \text{a.s.}.$$ QED

Answer 4

Here's a way with summation by parts: $$ \sum_{k=1}^n \frac{X_k}{k} = \frac{S_n}{n} + \sum_{k=1}^{n-1} \frac{S_k}{k(k+1)} $$

where $S_k =\sum_{i=1}^k X_i$. Note that $(S_k)_k$ is the symmetric random walk on $\mathbb Z$. It's an irreducible recurrent Markov chain, so $(S_k)_k$ is unbounded a.s. and the Dirichlet test does not apply.

However, by the law of the iterated logarithm, $\limsup_k \frac{|S_k|}{\sqrt{2k\log \log k}} = 1$ a.s, hence $\left(\frac{|S_k|}{\sqrt{k\log \log k}}\right)_k$ is bounded a.s. As a result, $S_k=O(k^{1/2+\epsilon})$ a.s., thus the series $\sum_{k\geq 1} \frac{S_k}{k(k+1)}$ converges a.s and simultaneously $\frac{S_n}{n}\to 0 $ a.s.

For more elementary asymptotics, note that for any $\epsilon, \epsilon'>0$, Hoeffding's inequality yields $$P\left(\frac{S_n}{n^{1/2+\epsilon}}\geq \epsilon'\right) \leq \exp\left(-\frac{\epsilon'^2n^{2\epsilon}}{2}\right)$$ so $\sum_n P\left(\frac{S_n}{n^{1/2+\epsilon}}\geq \epsilon'\right)$ converges, thus $\displaystyle \frac{S_n}{n^{1/2+\epsilon}}\xrightarrow[]{a.s.} 0$. Hence $S_n=o(n^{1/2+\epsilon})$ a.s. and one concludes similarly.