I have the following question in which I need some help. A mark of chain on states {0,1,2,3,4,5} has the transition probability matrix
\begin{bmatrix} 1/3 & 2/3 & 0 & 0 & 0 & 0 \\ 2/3 & 1/3 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1/4 & 3/4 & 0 & 0 \\ 0 & 0 & 1/5 & 4/5 & 0 & 0 \\ 1/4 & 0 & 1/4 & 0 & 1/4 & 1/4 \\ 1/6 & 1/6 & 1/6 & 1/6 & 1/6 & 1/6 \end{bmatrix}
Find all classes. Compute the limiting probabilities $\lim_{x\to\infty} P^{n}_{ji}$ for all i,j = 0,1,2,3,4,5.
My approach so far:-
I have identified that there would be three classes:- $C_{1}$ = {0,1}, $C_{2}$ = {2,3} and $C_{3}$ = {4,5}. Out of which $C_{1}$ and $C_{2}$ are recurrent and $C_{3}$ is transient. Also, $C_{1}$ and $C_{2}$ are absorbing classes. I then calculate the absorption probabilities $\pi_{4}(C_{1})$, $\pi_{5}(C_{1})$, $\pi_{4}(C_{2})$, $\pi_{4}(C_{2})$. What should I do after that?
Any help would be appreciated. Thanks.
Since the last two states are transient, you know that the last two columns of $P^\infty$ will be zero. Also, the nonzero entries of the last two rows will be the corresponding absorption probabilities. Now you need to fill in the two $2\times2$ blocks corresponding to the two absorbing classes. You should be able to work those out by isolating the corresponding blocks of the original transition matrix. Each row will, of course, be the steady-state distribution for that absorbing class.
You can also compute $P^\infty$ without first explicitly computing the absorption probabilities by computing the left eigenvectors of $1$, i.e., a basis for the null space of $P^T-I_6$. The usual row-reduction method will give you, after normalization, the steady-state distributions of the two absorbing classes. The remaining two rows of the matrix are affine combinations of these vectors. By inspection or otherwise, you can find that the probabilities of ending up in either absorbing class are equal, so the last two rows of $P^\infty$ are the averages of the two absorbing class distributions.