A discrete time Markov chain with such a transient state that $\mathbb P(T_i<\infty \ | \ X_0=i) \neq 0$

103 Views Asked by At

All examples of discrete time Markov chains my text provides are where $S$ is finite, and as far as I can tell, it makes all transient states have $$\mathbb P(T_i<\infty \ | \ X_0=i) = 0.$$

Are there any simple examples of a transient state with the above probability being strictly greater than zero?

1

There are 1 best solutions below

0
On BEST ANSWER

If the transition matrix is $P=\pmatrix{p&1-p\cr 0&1}$, then $\mathbb{P}(T_i<\infty\,|\, X_0=i)=p$ for any $0\leq p\leq 1$. The state $i$ is transient provided $p<1$.