Calculate $P_{BB}^{n}$ using the periodicity of a Markov chain

50 Views Asked by At

I want to compute the probability of $P_{BB}^{n}$ with the following transition matrix: \begin{equation} \begin{pmatrix} 0 & 1/2 & 0 & 1/2 & 0 \\ 0 & 0 & 2/3 & 0 & 1/3 \\ 0 & 0 & 0 & 1 & 0 \\ 1 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 \end{pmatrix} \end{equation} I know this is an irreducible Markov chain and the periodicity of every state is 2 (it is a class property and all states belong to the same class with it being irreducible). I've been trying to compute the probability without recurring to the eigenvalues method given that the problem becomes very complex and I've been given the hint that I should be able to get it using the period, but I don't really know how to approach it, since I started decomposing the probability but kinda got stuck with it. \begin{equation} \mathbb{P}(X_n = B|X_0 = B)=\frac{1}{3} \mathbb{P}(X_n = B|X_1 = E)+\frac{2}{3} \mathbb{P}(X_n = B|X_1 = C)=...=\frac{1}{3} \mathbb{P}(X_n = B|X_2 = B)+\frac{2}{3}(\frac{1}{2}\mathbb{P}(X_n = B|X_4 = B)+\frac{1}{2}\mathbb{P}(X_n = B|X_4 = D) \end{equation} And with D I would reach again a cyclic solution in another cyclic solution. I also know that if $n=2·k$ then the $\mathbb{P} > 0 $ and $0$ otherwise.