Limit of Probability and Probability of Limit

3.7k Views Asked by At

Let $\{x_k\}$ and $x^*$ be a sequence and a point in $\mathbb{R}^n$, respectively. Can we conclude that $$\lim_{k\to\infty} \mathrm{Prob}(x_k=x^*)=1$$ and $$\mathrm{Prob}(\lim_{k\to\infty} x_k=x^*)=1$$ are equivalent or that one implies the other?

I think the first one implies the second, but not vice versa since $x^*$ might not be part of the sequence.

3

There are 3 best solutions below

2
On BEST ANSWER

Neither implies the other.

To see why the first does not imply the second, I'll describe a sequence of random variables defined on $\Omega = [0, 1)$. These variables will all be either $1$ or $0$ with different probabilities. I'll outline where they're $1$, and they're $0$ elsewhere.

  • $X_1(a) = 1$ on $[0, 1/2)$
  • $X_2(a) = 1$ on $[1/2, 1)$
  • $X_3(a) = 1$ on $[0, 1/4)$
  • $X_4(a) = 1$ on $[1/4, 1/2)$
  • $X_5(a) = 1$ on $[1/2, 3/4)$
  • $X_6(a) = 1$ on $[3/4, 1)$
  • $X_7(a) = 1$ on $[0, 1/8)$

etc. Notice the pattern; the next few variables will be $1$ on a set of measure probability $1/8$, and that set will shift to the right until it hits $1$; then, the next few variables will be $1$ on a set of probability $1/16$, and so on.

Note that these random variables satisfy your first condition; specifically, they converge to $0$ in probability. That is,

  • $\mathbb P(X_1 = 0) = 1/2$
  • $\mathbb P(X_3 = 0) = 3/4$
  • $\mathbb P(X_7 = 0) = 7/8$
  • $\mathbb P(X_{15} = 0) = 15/16$

and that $\mathbb P(X_k = 0)$ is a nondecreasing sequence that tends to $1$. However, for no fixed $a \in [0, 1)$ is it the case that $X_k(a) \to 0$, because that sequence of numbers will oscillate infinitely many times between $0$ and $1$.

As you noted, the reverse implication doesn't hold either; pick a deterministic sequence that converges to something that's not in the sequence, e.g. $x_k = 1/k$ and $x^* = 0$.

0
On

Let $x_k=\frac 1 k$ a.s., then $P(x_k=0)=0 \to 0$ but $P( \lim_{k \to \infty} x_k=0)=1$.

Now let $(x_k)$ being a sequence of independent r.v. being $0$ with probability $1-1/k$ and 1 with probability $1/k$. Then :

$$\lim_{k\to\infty} P(x_k=0)=\lim_{k\to\infty} 1-\frac 1 k = 1$$

But $$P(\lim_{k\to\infty} x_k=0)=P(\exists N, \forall k>N, x_k=0)$$

And

$$P(\exists N, \forall k>N, x_k=0)=P(x_k=1 \text{ finitely often })$$

But since $ \sum P(x_k=1)=\sum \frac 1 k = \infty$, by second Borel-Cantelli lemma :

$$P(\lim_{k\to\infty} x_k=0)=0.$$

0
On

Imagine you had a bunch of sample trajectories of $x_k$—an infinite bunch, in fact—and further imagine that we treat $k$ as a kind of time index.

Intuitively, the first statement says that eventually (i.e., as $k$ increases without bound), almost all of the trajectories at any given moment will be at $x^*$—not just arbitrarily close to $x^*$, but equal to it. However, it is possible for the exceptions to change from one $k$ to the next, with each trajectory momentarily leaving $x^*$ infinitely often, so that none of the trajectories actually has a limiting value.

$$ \begin{array}{|c|c|c|c|c|c|c|c|c|c|c|} \hline \text{Trajectory} & k = 1 & k = 2 & k = 3 & k = 4 & k = 5 & k = 6 & k = 7 & k = 8 & k = 9 & \ldots \\ \hline A & 1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & \ldots \\ \hline B & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & \ldots \\ \hline C & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & \ldots \\ \hline D & 1 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & \ldots \\ \hline E & 1 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & \ldots \\ \hline F & 0 & 1 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & \ldots \\ \hline G & 0 & 1 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & \ldots \\ \hline H & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & \ldots \\ \hline I & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 1 & \ldots \\ \hline J & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & \ldots \\ \hline \end{array} $$


In contrast, the second statement says that almost all of the trajectories will eventually get arbitrarily close to $x^*$. However, it may happen that none of the trajectories ever reaches $x^*$.

$$ \begin{array}{|c|c|c|c|c|c|c|c|c|c|c|} \hline \text{Trajectory} & k = 1 & k = 2 & k = 3 & k = 4 & k = 5 & k = 6 & k = 7 & k = 8 & k = 9 & \ldots \\ \hline A & 1 & 1/2 & 1/3 & 1/4 & 1/5 & 1/6 & 1/7 & 1/8 & 1/9 & \ldots \\ \hline B & 1 & 1/2 & 1/4 & 1/8 & 1/16 & 1/32 & 1/64 & 1/128 & 1/256 & \ldots \\ \hline C & 1 & -1/3 & 1/5 & -1/7 & 1/9 & -1/11 & 1/13 & -1/15 & 1/17 & \ldots \\ \hline D & 1 & 1/2 & 1/2 & 1/4 & 1/4 & 1/4 & 1/4 & 1/8 & 1/8 & \ldots \\ \hline E & 1 & 1/2 & 1/3 & 1/5 & 1/7 & 1/11 & 1/13 & 1/17 & 1/19 & \ldots \\ \hline F & 1 & 1/2 & 1/3 & 1/5 & 1/8 & 1/13 & 1/21 & 1/34 & 1/55 & \ldots \\ \hline G & 1 & 2/3 & 1/2 & 1/3 & 1/4 & 1/6 & 1/8 & 1/12 & 1/16 & \ldots \\ \hline H & 1 & 1/2 & 1/6 & 1/12 & 1/20 & 1/30 & 1/42 & 1/56 & 1/72 & \ldots \\ \hline I & 1 & 1/4 & 1/10 & 1/20 & 1/35 & 1/56 & 1/84 & 1/120 & 1/165 & \ldots \\ \hline J & 1 & 1/4 & 1/9 & 1/16 & 1/25 & 1/36 & 1/49 & 1/64 & 1/81 & \ldots \\ \hline \end{array} $$


So neither of them implies the other.