Is $R_2 =\max\{n \geq 0 : X_n = 2\}$ a stopping time?

Question

Consider a discrete time Markov chain with state space $\{1,2,3,4,5\}$ with the following transition matrix:

$$ \begin{bmatrix} 0 & 0 & 1 & 0 & 0 \\ 0.3 & 0.2 & 0.1 & 0.4 & 0\\ 0 & 1 & 0 & 0 & 0\\ 0 & 0 & 0 & 0.3 & 0.7 \\ 0 & 0 & 0 & 0.2 & 0.8 \\ \end{bmatrix} $$

1. Suppose $R_2 =\max\{n \geq 0 : X_n = 2\}$, is $R_2$ a stopping time?
2. Calculate $\mathbb{P}(X_{R_2+1}=4 \mid X_{R_2} = 2, R_2 = 8)$.

For question 1, I am kinda confused about "stopping time" in this case. In my textbook, stopping time is defined as "$T$ is a stopping time if the occurrence (or nonoccurrence) of the event “we stop at time $n$,” $\{T = n\}$, can be determined by looking at the values of the process up to that time."

My understanding for $R_2 =\max\{n \geq 0 : X_n = 2\}$ is that $R_2$ is the last time we visit state 2. I think this event cannot be determined by just looking the prior cases. Instead, we need to know if the next state is 4 (recurrent state). Am I right?

For question 2, I guess we can write $\mathbb{P}(X_{R_2+1}=4 \mid X_{R_2} = 2, R_2 = 8)$ as $\mathbb{P}(X_{9}=4 \mid X_{8} = 2)$, so it is $0.4$? Suppose $R_2$ is not a stopping time, then can we just drop the information $R_2 = 8$?

This is a problem from my stochastic process homework, so any hints will be appreciated.

Answer 1

Let $\{\mathcal F_n\}$ be the natural filtration of the Markov chain, that is, $$ \mathcal F_n = \sigma\left(\bigcup_{i=0}^n\sigma(X_i)\right). $$ A nonnegative integer-valued random variable $\tau$ is a stopping time with respect to $\{\mathcal F_n\}$ if and only if $\{\tau=n\}\in\mathcal F_n$ for all $n$. Here $R_2$ is not a stopping time, since for example $\{R_2 = 1\}\notin\mathcal F_1$, as $$ \{R_2=1\} = \{X_1=2\}\cup\bigcup_{n=2}^\infty \{X_n\ne 2\}. $$ Intuitively, the information gained from observing the process up to time $n$ does not allow us to determine the value of $R_2$.

For part 2, observe that $X_n=1$ implies that $\mathbb P(X_{n+1}=3)=\mathbb P(X_{n+2}=2)=1$, and $X_n=3$ implies that $\mathbb P(X_{n+1}=2)=1$. It follows that conditioned on $\{R_2=8\}$ and $\{X_{R_2}=2\}$, $X_{R_2+1}=4$ with probability one.