Show that this Markov chain is recurrent or transient

633 Views Asked by At

Consider the Markov chain $(X_n)_{n\geq 0}$ with state space $E=\left\{1,2,3,4,5\right\}$ and transition matrix $$ T=\frac{1}{2}\begin{pmatrix}0 & 1 & 1 & 0 & 0\\0 & 0 & 0 & 1 & 1\\0 & 0 & 0 & 2 & 0\\2 & 0 & 0 & 0 & 0\\2 & 0 & 0 & 0 & 0\end{pmatrix}. $$ Is this Markov Chain recurrent or transient?

First of all, it is to say that this Markov chain is irreducible, i.e. only has one communicating class $C=E$. Because recurrence and transience are class properties, it sufficies to examine for example whether$\mathbb{P}_1(t(1)<\infty):=\mathbb{P}(t(1)<\infty|X_0=1)=1$ (in which case the MC would be recurrent) or $\mathbb{P}_1(t(1)<\infty)<1$ (in which case the MC would be transient)}. Here for $i\in E$ it is $$ t\colon\Omega\to\mathbb{N}\cup\left\{+\infty\right\}, t(\omega):=\inf\left\{n\in\mathbb{N}: X_n(\omega)=i\right\}. $$

Another way to examine this is to look whether $$ \mathbb{P}_1(\left\{\exists n\in\mathbb{N}: X_n=i\right\})=1 $$ or $$ \mathbb{P}_1(\left\{\exists n\in\mathbb{N}: X_n=i\right\})<1. $$

I think I should use this. So set $A:=\left\{\exists n\in\mathbb{N}: X_n=1\right\}$. Then it is $\mathbb{P}_1(A)=$ \begin{align} &\mathbb{P}_1(A,X_1=2)+\mathbb{P}_1(A,X_1=3)\\ &\geq\mathbb{P}_1(X_1=2,X_2=4,X_3=1)+\mathbb{P}_1(X_1=2,X_2=5,X_3=1)+\mathbb{P}_1(X_1=3,X_2=4,X_3=1)\\ &=\mathbb{P}(X_1=2,X_2=4,X_3=1|X_0=1)+\mathbb{P}(X_1=2,X_2=5,X_3=1|X_0=1)+\mathbb{P}(X_1=3,X_2=4,X_3=1|X_0=1)\\ &=\frac{\mathbb{P}(X_0=1,X_1=2,X_2=4,X_3=1)}{\mathbb{P}(X_0=1)}+\frac{\mathbb{P}(X_0=1,X_1=2,X_2=5,X_3=1)}{\mathbb{P}(X_0=1)}+\frac{\mathbb{P}(X_0=1,X_1=3,X_2=4,X_3=1)}{\mathbb{P}(X_0=1)}\\ &=\frac{\mathbb{P}(X_0=1)p_{12}p_{24}p_{41}}{\mathbb{P}(X_0=1)}+\frac{\mathbb{P}(X_0=1)p_{12}p_{25}p_{51}}{\mathbb{P}(X_0=1)}+\frac{\mathbb{P}(X_0=1)p_{13}p_{34}p_{41}}{\mathbb{P}(X_0=1)}\\ &=p_{12}p_{24}p_{41}+p_{12}p_{25}p_{51}+p_{13}p_{34}p_{41}\\ &=\frac{1}{2}\cdot\frac{1}{2}+\frac{1}{2}\cdot\frac{1}{2}+\frac{1}{2}=1 \end{align}

So it follows that $$ \mathbb{P}_1(A)=1, $$ meaning that the Markov chain is recurrent.

Am I right? Is my "strategy" okay?