Exercise 1.3 Introduction to Stochastic Processes Georgy.F Lawler :
Consider a Markov chain with state space {1,2,3} and transition matrix
$$ P=
\begin{pmatrix}
.4 & .2 & .4 \\
.6 & 0 & .4 \\
.2 & .5 & .3 \\
\end{pmatrix} $$
what is the probability in the long run that the chain is in state 1?
My Answer: I want to use the invariant probability vector as a left eigenvector. But according to Perron-Frobenius Theorem, The matrix P ( Markov matrix) doesn't satisfy the condition to use the invariant probability distribution.( all of entries must be strictly positive, one of them is zero ), so I am thinking to use the square of the matrix P instead of P .
My question: How can I use the invariant probability distribution here?
The solution of the following system of equations will give the left eigenvector (assuming that there is a corresponding eigenvalue: $1$)
$$ \begin{pmatrix} P_1& P_2& P_3 \end{pmatrix}= \begin{pmatrix} P_1& P_2& P_3 \end{pmatrix} \begin{pmatrix} .4 & .2 & .4 \\ .6 & 0 & .4 \\ .2 & .5 & .3 \\ \end{pmatrix} .$$
If we want he eigenvector(s) to be long run state probabilites the we have to add the equation that
$$P_1+P_2+P_3=1.$$
So, we have
$$0.4P_1+0.6P_2+0.2P_3=P_1$$ $$0.2P_1+0.5P_3=P_2$$ $$0.4P_1+0.4P_2+0.5P_3=P_3$$ $$P_1+P_2+P_3=1.$$
This system of equations happens to have a unique solution:
$$P=\left(\frac{25}{66} \ \frac{17}{66}\ \frac{4}{11 }\right).$$
The long run probability that the process is in state $1$ is then
$$\frac{25}{66}.$$
The mere existence of this solution shows that there is an eigenvalue: $1$.