Limiting distribution of a Markov Chain with period 3

67 Views Asked by At

Question: Fix $\alpha \in (0,1)$. Let $(Y_n)_{n=0,1,2,...}$ be a Markov Chain with state space $S=\{1,2,3,4,5\}$ and transition matrix $$P= \begin{bmatrix} 0 & \alpha & 1 - \alpha & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 & 0 \\ \end{bmatrix}. $$ Define a new matrix $Q$ from $P$ such that $q_{ii} = 1- \delta$, where $\delta \in (0,1)$, and $q_{ij} = \delta p_{ij}$ for $i \neq j$. The following fact is given: for a transition matrix $P$ with $p_{ii} = 0$, if $\pi$ is stationary for the chain with matrix $P$, then $\pi$ is stationary for the chain with matrix $Q$.

The question is to find the limiting distributions of $P$ and $Q$ if they exist.

Attempt: I started by finding the stationary distribution of $P$, which I got as $$\pi = \frac{1}{3} \begin{bmatrix} 1 & \alpha & 1-\alpha & \alpha & 1-\alpha \end{bmatrix}.$$

I have managed to verify this calculation using Python code.

I also noted that the chain with matrix $P$ is irreducible and has period $3$. It is also positive recurrent as the chain is closed and finite. Since it is not aperiodic, it does not have a limiting distribution.

For $Q$, I note that it is aperiodic, irreducible and positive recurrent. It therefore has a unique limiting distribution. I write $\pi Q = Q$, I deduced the same equations as I had obtained for $P$. So $\pi$ is stationary for $Q$ as well, and it is the unique limiting distribution.

Can someone please let me know if this is correct? Thank you for your time!

1

There are 1 best solutions below

0
On BEST ANSWER

Your conclusions are correct, but the limiting distribution of $Q$ can be deduced without need for further computation.

Since $P_{ii}=0$ for all $i$, we can write $Q=(1-\delta)I+\delta P$, hence $$ \pi Q = \pi(1-\delta)I + \delta\pi P = \pi-\delta\pi + \delta \pi = \pi, $$ so that $\pi$ is a stationary distribution for $Q$ (as was given in the question). Moreover, $Q$ is irreducible and recurrent since $P_{ij}>0$ implies $Q_{ij}>0$, and $Q$ is necessarily aperiodic as $Q_{ii}=\delta>0$ for all $i$. It follows that $\pi$ is the unique limiting distribution of $Q$.