Limit distribution of Markov Chain

Question

I have to determine the limiting distribution for the MC so I determine the equations first:

$\Pi_0 +\Pi_1 + \Pi_2 + \Pi_3 +\Pi_4 =1$

$\Pi_0 = \Pi_0q + \Pi_1q + \Pi_2q +\Pi_3q +\Pi_4$

$\Pi_1 = \Pi_0p$

$\Pi_2 = \Pi_1p$

$\Pi_3 = \Pi_2p$

$\Pi_4 = \Pi_3p$

and then I tried to solve the system and I got this:

$0= \Pi_0 (-1+q+pq+p^2q+p^3q+p^4) $

So I conclude $0= \Pi_0$ therefore $\Pi_1 = \Pi_2 = \Pi_3 =\Pi_4 =0$

but this doesn't match the answer of the book.

Can somebody tell me where is my mistake?

Answer 1

In this Markov chain, you need find the vector ${\bf \Pi}$ that satisfies the following:

$$ \bf \Pi\ {\bf{P}} = \Pi$$

To start, compute $\bf\Pi\ P$, and we get

$$ {\bf\Pi\ P} = \left[\Pi _5+\Pi _1 q+\Pi _2 q+\Pi _3 q+\Pi _4 q,\ p \Pi _1,\ p \Pi _2,\ p \Pi _3,\ p \Pi _4\right]. $$

Then we set up the linear equations, as shown below:

$$ \begin{eqnarray} \Pi _5+\Pi _1 q+\Pi _2 q+\Pi _3 q+\Pi _4 q & = & \Pi _1 \\ p \Pi _1 & =& \Pi _2 \\ p \Pi _2 & =&\Pi _3 \\ p \Pi _3& = &\Pi _4 \\ \Pi _1+\Pi _2+\Pi _3+\Pi _4+\Pi _5& =& 1 \end{eqnarray} $$

An then once we solve them, we get:

$$\Pi _1 = -\frac{1}{p \left(p^2+p+1\right) (q-1)+q-2}$$

$$\Pi _2=-\frac{p}{p \left(p^2+p+1\right) (q-1)+q-2}$$

$$\Pi _3 = -\frac{p^2}{p \left(p^2+p+1\right) (q-1)+q-2}$$

$$\Pi _4= -\frac{p^3}{p \left(p^2+p+1\right) (q-1)+q-2}$$

$$\Pi _5= -\frac{p^4}{p \left(p^2+p+1\right) (q-1)+q-2}$$

Answer 2

You were correct in your setup, and getting to $$0= \Pi_0 (-1+q+pq+p^2q+p^3q+p^4).$$

However, you need to use the fact that $p+q=1$ to rewrite this. You get \begin{align*} -1+q+pq+p^2q+p^3q+p^4 &= -p+pq+p^2q+p^3q+p^4\\ &=p(q-1)+p^2q+p^3q+p^4\\ &=-p^{2}+p^2q+p^3q+p^4\\ &=p^{2}(q-1)+p^3q+p^4\\ &=-p^{3}+p^3q+p^4\\ &=p^{3}(q-1) + p^{4}\\ &=-p^{4}+p^{4}\\ &=0. \end{align*}

Hence, we can choose any non-zero value for $\Pi_{0}$ and normalize later. By normalize I mean enforce the condition that $\Pi_0 +\Pi_1 + \Pi_2 + \Pi_3 +\Pi_4 =1.$ Without this condition we are simply finding a left eigenvector for $P$ corresponding to an eigenvalue of $1$. Choosing $\Pi_{0} = 1$ we get \begin{align*} \Pi_{0}&=1\\ \Pi_{1}&=p\\ \Pi_{2}&=p^{2}\\ \Pi_{3}&=p^{3}\\ \Pi_{4}&=p^{4}. \end{align*}

Note that we now have a vector $\Pi$ such that $\Pi P = \Pi$, however $\Pi$ as it is now is not a probability vector because we relaxed the condition that $\Pi_0 +\Pi_1 + \Pi_2 + \Pi_3 +\Pi_4 =1.$ To normalize $\Pi$ so it satisfies this condition, we divide $\Pi$ by the sum of its components so that the resulting vector is a probability vector. Since currently $\Pi_0 +\Pi_1 + \Pi_2 + \Pi_3 +\Pi_4 =1 + p + p^{2} + p^{3} + p^{4}$, we divide each component of $\Pi$ by this to obtain the limiting distribution (which I'm also calling $\Pi$):

\begin{align*} \Pi_{0}&=\frac{1}{1+p+p^{2}+p^{3}+p^{4}}\\ \Pi_{1}&=\frac{p}{1+p+p^{2}+p^{3}+p^{4}}\\ \Pi_{2}&=\frac{p^{2}}{1+p+p^{2}+p^{3}+p^{4}}\\ \Pi_{3}&=\frac{p^{3}}{1+p+p^{2}+p^{3}+p^{4}}\\ \Pi_{4}&=\frac{p^{4}}{1+p+p^{2}+p^{3}+p^{4}}. \end{align*}

This division didn't change the fact that $\Pi P = \Pi$, but now if we add the components we get $1$ which solves the system of equations.