Consider the probability transition matrix
$$\begin{pmatrix} 0.5 & 0.5 & 0 & 0 & 0 & 0 & 0 \\ 1 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0.5 & 0.2 & 0.3 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0.2 & 0 & 0.8 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0.5 & 0 & 0.5 & 0\end{pmatrix}.$$
Calculate $\lim_{n\to\infty} p_{n}$ where $p_{0} = \begin{pmatrix}1 & 0 & 0 & 0 & 0 & 0 & 0 \end{pmatrix}.$
This is a given transition state matrix for a $7$-state Markov chain. I am asked to find the limit as n approaches infinity of $p_{n}$.
I was able to compute the answer using a program as follows: Let the transition state matrix equal $M$
I believe that the correct answer is $\begin{pmatrix}0.67 & 0.33 & 0 & 0 & 0 & 0 & 0 \end{pmatrix}$, which I got using Mathematica. I pretty much just computed a large matrix power of the transition matrix and multiplied it with $p_{0}$.
So I have the following two questions:
1) Is this correct?
2) Is there a way to do this by hand? For instance, if I were doing this on an exam, I wouldn't be able to use Mathematica. So how can I do this by hand?
Your method is correct, but I would like to point out a simplification. Note that $p_0$ essentially tells you that we start off the markov chain in state 0. Based on the transition matrix, we can only ever be in states 0 and 1. Hence, we can reduce the problem to a markov chain problem with 2 states with probability matrix
$P = \begin{bmatrix} 0.5 & 0.5 \\ 1 & 0\end{bmatrix}$
and $p_0 = (1,0)$. Now, you can diagonalize $P = Q \Lambda Q^{-1}$ and it is easy to see that $L = \lim\limits_{n\to \infty} P^n = Q \lim\limits_{n\to \infty} \Lambda^n Q^{-1}$. Then, multiply $p_0^T L$ to get the final result.
From my calculation, $\Lambda = diag(1,-0.5)$, and
$Q = \begin{bmatrix} 1 & 1\\ 1 & -2\end{bmatrix}$
Another possibly more elegant solution is that in the above simplified problem, the chain is ergodic and hence every row vector of $L$ is the stationary distribution. One way to solve for the stationary distribution is to solve for the eigenvector (corresponding to eigenvalue 1) of $P^T$.