Equality of probability of finite hitting time for irreducible states in Markov Chain

134 Views Asked by At

Suppose I have a finite state Markov Chain with state space $S=\{1,2,3,4,5,6\}$. Suppose I further have that $\{1,2\}$,$\{3,4\}$ and $\{5,6\}$ are irreducible classes where $\{1,2\}$ and $\{3,4\}$ are closed.

Suppose I want to calculate $P_5[T_{\{1,2\}}<\infty]$ and $P_6[T_{\{1,2\}}<\infty]$. Here $T_C$ is the first hitting time at $C\subset S$. Now, why can I not give the following argument:

$5$ leads to $\{1,2\}$ in finite number of steps if and only if $6$ leads to $\{1,2\}$ in finite number of steps, because $6$ and $5$ communicate with each other. Hence $P_5[T_{\{1,2\}}<\infty]=P_6[T_{\{1,2\}}<\infty]$.

This argument is clearly not valid (an exercise in Hoel Port Stone shows I am wrong). But I can't find any flaw with it.

Also, when are these equal?

Suppose I had a different problem where the state space $S=\{1,2,3,4,5\}$ and $\{1,2\}$ and $\{3,4\}$ are irreducible and closed respectively, and we want to find $P_5[T_{1}<\infty]$ and $P_5[T_{2}<\infty]$. Are they equal, by my previous "argument"? Here by abuse of notation, consider $T_j=T_{\{j\}}$.

1

There are 1 best solutions below

11
On

To see a counterexample to your argument, let for example the transition probabilities:

  1. $6 \to 5= 0.01$ and $6 \to \{3,4\}=0.99$,
  2. $5 \to 6= 0.01$ and $5 \to \{1,2\}=0.99$.

Now, a simple calculation shows that $$P_6[T_{1,2}<+\infty]\le 0.01 < 0.99 \le P_5(T_{1,2}<+\infty)$$

For them to be equal, you need certain symmetry conditions on the transitions probabilities from states $5$ to $6$ as you can see from the above counterexample. So, actually equality would be a special case and not the rule. But in your second question, this is different. The probabilities $P_5[T_1<+\infty]$ and $P_5[T_2<+\infty]$ are indeed equal, since the following events are equivalent $$(T_2 \mid X_0=5)=+\infty \iff (T_1 \mid X_0=5)=+\infty$$ Namely, if starting from $5$ your chain visits $1$ then this implies that the chain will visit $2$ by the "closedness" of $\{1,2\}$. And conversely, if the chain visits $2$ this implies that the chain will certainly visit $1$.