Let $(X_t)_{t \in \mathbb{R}}$ be a square-integrable real-valued process with a continuous mean value function $\mu:\mathbb{R}\rightarrow\mathbb{R}$ and a continuous covariance function $\Sigma:~\mathbb{R} \times \mathbb{R} \rightarrow \mathbb{R}$.
I want to show, that this process is stochastic continuous ($\lim_{s \rightarrow t} \mathbb{P}(|X_t - X_s| > \epsilon)=0~ \forall t \in \mathbb{R})$.
My first idea was using the markov inequality, then I would get
$\mathbb{P}(|X_t-X_s|>\epsilon) \leq \dfrac{\mathbb{E}(|X_t-X_s|^2)}{\epsilon^2} = \dfrac{\mathbb{E}(X_t^2-2X_tX_s+X_s^2)}{\epsilon^2} = \dfrac{\mathbb{E}(X_t^2)-2\mathbb{E}(X_tX_s)+\mathbb{E}(X_s^2)}{\epsilon^2}$.
Is this a helpful beginning of the proof or is there a better way? And how can I proceed?
Thanks in advance.
Define $m(t)$ and $f(t,s)$ as the mean and covariance functions: \begin{align} m(t) &= E[X_t]\\ f(t,s) &= E[X_tX_s] - E[X_t]E[X_s] \end{align} We are told that the functions $m(t)$ and $f(t,s)$ are continuous. In general, if a function $h(t)$ is continuous, we know that $\lim_{s\rightarrow t} h(s) = h(t)$. Thus: \begin{align} &\lim_{s\rightarrow t} m(s) = m(t)\\ &\lim_{s\rightarrow t} f(t,s) = f(t,t) \end{align}
Note that once you have the functions $m(t)$ and $f(t,s)$, there is no randomness involved (and we can take limits as usual). This is in contrast to $\lim_{s\rightarrow t} X_s$, which is the limit of a random process. The limit may not exist. If it does exist, the limit is itself a random variable. We are not told anything about continuity of the random process $X_t$. It could be discontinuous even if its mean and covariance functions are continuous. See example below.
Example 1: Here is an example of a discontinuous process $X_t$ with continuous mean and covariance functions. Let $Y$ be a random variable that is exponentially distributed with rate $\lambda$. Define $X_t$ for all $t \geq 0$ as follows: $$ X_t = \left\{ \begin{array}{ll} 1 &\mbox{ if $t < Y$} \\ 0 & \mbox{ if $t \geq Y$} \end{array} \right.$$ Thus, $X_t$ starts out with the value 1, but flips discontinuously to 0 when $t$ crosses the random threshold $Y$.
Then for $t \geq 0$ we have: \begin{align} m(t) &= E[X_t] \\ &= Pr[X_t=1]\\ &= Pr[t < Y] \\ &= e^{-\lambda t} \end{align} Thus, $m(t)$ is continuous over $t \geq 0$.
Now for $t \geq 0$ and $s \geq 0$ we have: \begin{align} f(t,s) &= E[X_tX_s] - m(t)m(s) \\ &= Pr[X_t=1, X_s=1] - m(t)m(s) \\ &= Pr[\max[t,s] < Y] - m(t)m(s) \\ &= e^{-\lambda\max[t,s]} - e^{-\lambda t}e^{-\lambda s} \end{align} Thus, $f(t,s)$ is continuous over $t \geq 0$ and $s \geq 0$.
Example 2: Here is an example of a process $X_t$ that satisfies $E[\lim_{t\rightarrow\infty} X_t] \neq \lim_{t\rightarrow\infty} E[X_t]$. Again let $Y$ be exponentially distributed with rate $\lambda$. Define $X_t$ for all $t \geq 0$ by:
$$ X_t = \left\{ \begin{array}{ll} e^{\lambda t} &\mbox{ if $t < Y$} \\ 0 & \mbox{ if $t \geq Y$} \end{array} \right.$$ Thus, $X_t$ increases exponentially until it crosses the random threshold $Y$, at which point it (discontinuously) jumps down to $0$ and stays there. Since $Y$ is finite with probability 1, we know with probability 1 that $\lim_{t\rightarrow\infty} X_t = 0$. Thus, $E[\lim_{t\rightarrow\infty} X_t] = 0$. However, for all $t\geq 0$ we have: $$ E[X_t] = e^{\lambda t} Pr[t < Y] = 1 $$ Thus, $\lim_{t\rightarrow\infty} E[X_t] = 1$. And $1 \neq 0$.
Example 3: Here is an example of a process $X_t$ such that $\lim_{t\rightarrow 0} X_t$ does not exist, but $\lim_{t\rightarrow 0} E[X_t] = 0$. Let $A$ be a random variable that takes values in the set $\{-1,1\}$ and satisfies $Pr[A=1]=Pr[A=-1]=1/2$ (and so $E[A]=0$). Define: $$ X_t = \left\{ \begin{array}{ll} A\cos(1/t) &\mbox{ if $t \neq 0$} \\ 0 & \mbox{ if $t =0$} \end{array} \right.$$ Then $E[X_t]=0$ for all $t$, so $\lim_{t\rightarrow 0} E[X_t]=0$. But $\lim_{t\rightarrow 0} X_t$ does not exist because $X_t$ infinitely oscillates between $-1$ and $1$ as $t\rightarrow 0$.