Let $X = (X_{n})_{n \geq 0}$ be a homogeneous irreducible Markov chain on the state space $E := \{1,\dots ,N\}$, $N \in \mathbb{N}$, $N \geq 2$ and transition probability Matrix $P$. Let $B \subset E$ be a proper subset of the state space $E$. For simplicity we assume that $B = \{1,2,\dots,L\}$, $1 \leq L < N$.
We call "sojourn of $X$ in $B$" any sequence $X_{m},X_{m+1},\dots,X_{m+k}$ where $k \geq 1$, $X_{m},X_{m+1},\dots,X_{m+k-1} \in B$, $X_{m+k} \notin B$ and if $m > 0$, $X_{m-1} \notin B$. This sojourn begins at time $m$ and finishes at time $m+k$. It lasts $k$.
Now let $V_{n}$, $n \geq 1$ be the random variable "state of $B$ in which the $n^{\text{th}}$ sojourn of $X$ begins".
Maybe the answer is obvious. I want to show that $(V_{n})_{n\geq 1}$ is a homogeneous Markov chain on the state space $B$. Hope someone can help me. Thanks!
Let S stands for the sequence of X's states up to (and including) the beginning of the nth sojourn and T be the sequence after S up to (and including) the beginning of the (n+1)th sojourn. You have $P(V_{n+1}/V_n,V_{n-1},...)=\sum_S[P(S/V_n,V_{n-1},...)\sum_TP(T,V_{n+1}/S,V_n,V_{n-1},...)]$ (E)
The inner sum is equal to $\sum_TP(T,V_n+1/V_n)$ due to the memoryless property so it does not depend on $S$ and the right part of (E) can be rewritten :
$P(V_{n+1}/V_n,V_{n-1},...)=\sum_SP(S/V_n,V_{n-1},...).\sum_TP(T,V_{n+1}/V_n)]=\sum_TP(T,V_{n+1}/V_n)=P(V_{n+1}/V_n)$