Markov chains - first hitting times between transient states satisfy either $\mathbb{P}^i(T_j < \infty) < 1$ or $\mathbb{P}^j(T_i < \infty) < 1$

226 Views Asked by At

We have a irreducible Markov chain $X=(X_n)_{n\geq0}$ on a state space $I$ with transition matrix $P$ and we denote by $T_i=\inf\{n\geq1: X_n=i\}$ the first hitting time of state $i$.

I have to prove the following:

Let $i\neq j \in I$ be connected transient states. Then either $\mathbb{P}^i(T_j < \infty) < 1$ or $\mathbb{P}^j(T_i < \infty) < 1$.

First I need to show that $\mathbb{P}^i(T_j < \infty) < 1$ or $\mathbb{P}^j(T_i < \infty) < 1$ hold. And then if one of them holds, the other doesn't.

I have a lack of intuition here. If $i$ is a tranient state, that means, starting in $i$, we visit $i$ only a finite number of times ($\mathbb{P}^i(V_i < \infty) =1$). I have already shown, that an equivalent definition of transience is that $\mathbb{P}^i (T_i < \infty) <1$. If I now assume that both $\mathbb{P}^i(T_j < \infty) < 1$ and $\mathbb{P}^j(T_i < \infty) < 1$ hold, this would mean that starting in $i$ there is a possibility to never hit $j$ and that starting in $j$, there is the possibility to never hit $i$. I don't see, where this gives a contradiction. Maybe this is the point where it kicks in that $i$ and $j$ are connected, so there has to be a positive probability of going from $i$ to $j$ or (and?) from $j$ to $i$.

Could you please help me to concretize my intuition and give me hints for a formal proof of the statement?

UPDATE: I came in contact with the person stating the result which should be proven. The "either ... or..." is not an exclusive or as I thought, it is just the normal inclusive or.

1

There are 1 best solutions below

7
On

Having stared at this for a while, I honestly can't see why we need to assume that $i$ and $j$ are communicating. If $i$ and $j$ are not communicating, then either $j$ is inaccessible from $i$ (in which case $\mathbb P(T_j < \infty | X_0 = i) = 0$) or $i$ is inaccessible from $j$ (in which case $\mathbb P(T_i < \infty | X_0 = j) = 0$).

For our proof by contradiction we should assume that $i$ and $j$ are transient, and that $$\mathbb P(T_j < \infty | X_0 = i) = 1, \ \ \mathbb P(T_i < \infty | X_0 = j) = 1.$$ Intuitively, these equations are saying that:

  • If we're currently in state $i$, then we will definitely visit state $j$ at some point in the future.
  • If we're currently in state $j$, then we will definitely visit state $i$ at some point in the future.

But then, if we're currently in state $i$, then we will definitely visit state $j$ at some point in the the future, and we will definitely return to state $i$ at some point after having visited state $j$. Therefore, if we start from state $i$, we are guaranteed to eventually return to state $i$. Which contradicts state $i$ being transient.