$(X_n)_n$ is a discrete-time, time-homogenous Markov chain. I have have the following transition matrix and want to show whether the chain is ergodic.
$$P = \begin{pmatrix} \frac{1}{2} & 0 & 0 & \frac{1}{2}\\ 0 & \frac{1}{2} & \frac{1}{2} & 0\\ 0 & \frac{1}{2} & \frac{1}{4} & \frac{1}{4}\\ \frac{1}{2} & 0 & 0 & \frac{1}{2}\end{pmatrix}$$
I tried to compute $P^2 P^3 $.... I don't have all coefficients $>0$.
There is a unique stationary distribution [$\frac{1}{2},0,0,\frac{1}{2}$] so I can't prove this is not ergodic.
More over, I can't prove that the distribution is periodic for example $P=P^5$ .
I tried to show that the matrix is irreducible with $A=(A+ \, \text{Id})^n$ ( find a n with all coefficients >0).
I am quite stuck now, thanks for your help
The Markov chain is not ergodic.
Consider a (more general) matrix of the form
$$P = \begin{pmatrix} p_{11} & 0 & 0 & p_{14} \\ p_{21} & p_{22} & p_{23} & p_{24} \\ p_{31} & p_{32} & p_{33} & p_{34} \\ p_{41} & 0 & 0 & p_{44} \end{pmatrix}.$$
We are interested in $Q := P^2$. By definition (of matrix multiplication), we have
$$q_{12} = \sum_{j=1}^4 p_{1j} p_{j2} = p_{11} \underbrace{p_{12}}_{0} + \underbrace{p_{12}}_{0} p_{22} + \underbrace{p_{13}}_{0} p_{32} + p_{14} \underbrace{p_{42}}_{0}=0.$$
Exactly the same argumentation yields
$$q_{13} = q_{42} = q_{43}=0.$$
This means that $Q=P^2$ is also of the form
$$Q= P^2 = \begin{pmatrix} q_{11} & 0 & 0 & q_{14} \\ q_{21} & q_{22} & q_{23} & q_{24} \\ q_{31} & q_{32} & q_{33} & q_{34} \\ q_{41} & 0 & 0 & q_{44} \end{pmatrix}.$$
Iterating this argumentation, we find that for any $n \in \mathbb{N}$ $P^n$ is of this form, i.e.
$$(P^n)_{1,2} = (P^n)_{1,3} = (P^n)_{4,2} = (P^n)_{4,3}=0.$$
This shows that $P$ is not ergodic.