Notions of continuity for stochastic processes

Question

I would like to receive some clarification regarding the difference between continuous in probability and continuous almost surely. Using the definition of the wikipedia page (that match the one I have seen in other references), we have

Continuous in probability for all $\varepsilon>0$ \begin{equation} \lim_{s\rightarrow t} \mathbb{P}\left(\left\{\omega \in \Omega : |X_s(\omega) - X_t(\omega) | \geq \varepsilon\right\} \right)= 0 \end{equation}

and

Continuous with probability 1 (almost surely) \begin{equation} \mathbb{P}\left(\left\{\omega \in \Omega : \lim_{s\rightarrow t} |X_s(\omega) - X_t(\omega) | = 0 \right\} \right)= 1 \end{equation}

Now, I can see that the second condition is stronger than the first one, but to me they seem analogous, since the first should be valid for arbitrarily close $\varepsilon$. Obviously, my intuition is wrong, but I cannot understand how the difference can be significant. Could you please explain how the two definitions give raise to different processes? Thank you!

Answer 1

[Note: I am answering this question because I see no replies and I was also interested, like the OP, in understanding better this difference and the intuitive aspects. Take this as just an attempt and please, feel free to correct, add, make precise, as appropriate, so that I can also learn from my errors.]

The almost sure continuity is stating that each of the functions $X_\omega(s)$ that you have, for each one of the ω in the space of outcomes $\Omega$, is continuous (in the usual math sense). And that the possible exceptions $\omega's$ have an overall probability equal to $0$.

The continuity in probability does not guarantee the continuity of "almost all" (i.e., with probability $1$) the sample paths $X_\omega(s)$, as we have seen before. On the contrary, each sample-path, for each $\epsilon>0$ you use, and no matter how close you get $s$ to $t$, is allowed to have still one index $s < s' < t$ (and, therefore, infinite of them) where it keeps violating the inequality $| X_\omega(t) - X_\omega(s) | < \epsilon$ (so that the continuity limit does not hold). However, if we consider, at each index $s$, all the $\omega$ where the corresponding paths, on that index, are having the "violation" $| X_\omega(t) - X_\omega(s) | < \epsilon$, their probability must tend to zero, as $s$ tends to $t$.

Example (please, correct me if I got this one wrong):

    X(s)= Be( min(1, t-s) ) , for s < t   
    X(t) = 0 
    X(s)= U(0, s-t)         , for s > t

where $Be(p)$ denotes a Bernoulli with success probability $p$ and $U(a,b)$ denotes a Uniform in $(a,b)$.

From the right, this indexed family $X(s)$ has a limit of $0$, surely.

From the left, the limit does not hold, because infinite $1's$ will keep preventing the function from getting definitely close to $0$. However, the probability of getting those sporadic "violations" is tending to $0$, as $s$ tends to $t$.

[UPDATE: more details]

Thank you for the upvotes. I am encouraged to put in the answer some more details, which I previously glossed over because I was mostly concerned with focusing on the "intuitive" aspects for the OP and myself. Again, feel free to jump in and correct/make precise where I am exaggerating with "intuition" and hurting rigor.

The reason for the infinite $1's$ which keep popping out in the left limit to $t$ in my example above (Bernoulli with decreasing success probability), is essentially the fact that:

1.) for each $s$, the probability of having a "violation" $| X_\omega(s) - X_\omega(t) | < \epsilon$ is non zero

and ($1.$ alone would not suffice, because the probability might decrease, as it does in this case)

2.) as $s$ tends to $t$, the probability of having a violation decreases, but not fast enough" to quit "fuelling" the infinite appearances of the value $1$. More precisely, the limit of the sum of the violation probabilities, over all $s < t$, is exploding to infinity, instead of converging (a continuous analog of the harmonic series).

If, for instance, instead of the $1/(t-s)$ probability, we consider its square, the overall "violation" probability from $s$ to $t$ would instead converge (a continuous analog of Basel problem) to a constant and there would be no probability left to "fuel" an infinite sequence of $1's$ to cause the limit inequality violations.

And this is, essentially, at least as I see it, the intuition with the Borel-Cantelli Lemma (which originally addressed the discrete case, though).

This is also the reason why the fact that the "individual" (for each $s$) violation probability goes to $0$ is not enough to prevent the phenomenon of occurrence of "infinite violations" (which will prevent the continuity limit to happen). In other words, the reason why convergence in probability does not cause convergence almost sure.

What you will need to make the convergence in probability imply the convergence almost sure, is that the integral, over all $s < t$, of the violation density (the overall probability to get a violation on $(s,t)$) to be finite, in order to stop "fuelling" the infinite appearance of those violations and thus allow the continuity limit to take place.

Answer 2

1. We first examine a much easier variant, i.e., discrete-time process. Let $Y = (Y_n)_{n\in\mathbb{N}_1}$ be a sequence of independent random variables such that

$$ \mathbb{P}(Y_n = 1) = \frac{1}{n} \qquad\text{and} \qquad \mathbb{P}(Y_n = 0) = 1 - \frac{1}{n}. $$

Then it is clear that, for each $\epsilon > 0$,

$$ \lim_{n\to\infty} \mathbb{P}( \left| Y_n - 0 \right| > \epsilon) \leq \lim_{n\to\infty} \mathbb{P}( Y_n \neq 0) = \lim_{n\to\infty} \frac{1}{n} = 0. $$

So $Y_n \to 0$ in probability. However, $\sum_{n=1}^{\infty} \mathbb{P}(Y_n = 1) = \infty$, and so, by the second Borel-Cantelli's lemma,

$$ \mathbb{P}(Y_n = 1 \text{ infinitely often}) = 1. $$

So it follows that

$$ \mathbb{P}\Bigl(\limsup_{n\to\infty} Y_n = 1\Bigr) = 1 \qquad\text{and}\qquad \mathbb{P}\Bigl(\liminf_{n\to\infty} Y_n = 0\Bigr) = 1. $$

So $Y_n$ does not converge as $n\to\infty$ with probability one.

Here, the difference between two notions is that the probability $\mathbb{P}(\left|Y_n - 0\right| > \epsilon)$ describes what happens to a 'typical observer $\omega$', and telling that the exceptional event occurs only to a vanishing fraction of observers for each large $n$. However, for a 'fixed observer $\omega$', this exceptional event can happen infinitely often.

2. Now coming back to continuous-time case, we can borrow the above example as follows:

Let $Y=(Y_n)_{n\in\mathbb{N}_1}$ be as above, and define $X = (X_t)_{t\geq 0}$ by

$$ X_t = \begin{cases} Y_{\lceil 1/t \rceil}, & \text{if } t > 0, \\ 0, & \text{if } t = 0. \end{cases} $$

Then $X_t \to 0$ as $t \to 0$ in probability, whereas $X_t$ does not converge as $t \to 0$ almost surely.